Photo: Bogdan Popa/autoevolution/USPTO
Ford has developed a system to read signs no matter how damaged or obstructed they are, allowing its cars to see behind any obstacle the critical data drivers care about.
The company has filed a patent application whose name says it all: “generation of artificial images using road signs”.
Before we get into the details, it’s important to understand why this is a major issue for drivers around the world.
While many people rely on navigation apps to read speed limits, authorities in several regions point out that the data they see on the screen may not always be accurate. Temporary restrictions and changes to speed limits could make your navigation app inaccurate, so police say paying attention to road signs is always crucial.
The biggest problem with road signs is that they are sometimes missing or damaged. While authorities are responsible for the first part, the second is something that technology can solve. If you can’t see a road sign because it’s blocked by a tree, damaged, rusted, bent, or covered in graffiti, Ford’s new system could help.
The company created this technology specifically to determine a road sign as you approach it, using machine learning and a system to create an artificial image that then feeds the correct data into the vehicle. ADAS Systems.
Photo: USPTO
It all starts with the realization that a road sign up ahead cannot be read properly.
Many new-generation cars on the road can already look ahead to read speed limits and display the information on the instrument panel. This makes it easier for the driver to avoid exceeding the speed limit, although I have always told everyone that they should also pay attention to road signs because you never know when the cameras read the data incorrectly.
Ford says the images for its system can be acquired from sensors, but these also include cameras installed on your vehicle. A special traffic sign detection system allows the hardware to spot a sign, take a picture, and feed it into a computer to extract the traffic sign information presented to the driver.
As I mentioned earlier, some of these signs can’t be read properly, so the computer tries to generate a new image that includes the correct information. In other words, if trees are blocking the road sign and the speed limit can be read correctly, the system generates a new image where the data is not obstructed. It uses this image for a variety of purposes, including presenting it to the driver on the dashboard or sending it to other systems that rely on the data being analyzed — for example, an autonomous vehicle could use the road sign data to continue a journey.
Photo: USPTO
Ford's system uses image manipulation to create correct road signs whose data can be clearly read; the company says its role is to “simulate the physical changes in road signs.”
The technology can capture multiple images of the same road sign as you drive toward it, so you can properly read the information it’s supposed to display. In some cases, these systems can compare a newly captured image to an existing database to determine what a road sign should display.
Furthermore, the system can be further improved by allowing vehicles on the road to exchange images and creating a database with the exact location of each road sign, thus allowing faster processing when a new vehicle approaches.
Photo: USPTO
The machine learning system can use all sorts of photo manipulation techniques and can generate clear images by adjusting image settings, including brightness, exposure, and colors. The system can simulate environmental conditions, such as lighting, weather, and time of day, and can even reproduce raindrops added to a road sign.
The benefits of such systems are clear, as they make it easier for drivers to read critical data when road signs are obstructed or damaged.
However, the other part of the problem still needs to be solved. The lack of road signs is still a major headache around the world, and this is where car manufacturers and the technology installed on them cannot help. That is why drivers are relying more on navigation software to make it easier to find a specific destination, even if this increases the likelihood of ending up in a different place or getting the wrong information.
Photo: USPTO
Ford’s idea needs some further refinement and possibly a simplified implementation to make it to production models, but for now it’s important to remember that it’s still in the patent phase. Sometimes car companies file patents in an attempt to prevent others from having the same idea, so I wouldn’t be surprised if that’s the case. The company has been tight-lipped about its plans for this technology, so you’d be wise not to hold your breath for it to make it into production.
If you are interested in all the technical details, including how the system can manipulate images to simulate real-life conditions, I have attached the patent application below.