John Deere
R4f167304 Rrd 1x 624b020d32bc4

Perception Systems Guide the Path to a Fully Autonomous Tractor

April 5, 2022
Creation of robust and reliable perception systems enables fully autonomous operation of the John Deere 8R tractor.

Automation is making its way into many applications, from the smallest of components and systems to large pieces of heavy-duty mobile equipment. It can take many forms such as controlling a single movement of a part or enabling an entire vehicle to drive without interaction of a human being.

The technologies which make automation possible are varied; sensors and software are key among them as well as perception systems.

For John Deere’s autonomous 8R tractor unveiled at the Consumer Electronics Show (CES) 2022, Noah Schwartz, lead perception engineer at Bear Flag Robotics—an autonomous agriculture technology company acquired by John Deere in 2021—said the company leveraged many of its existing automation technologies. These include StarFire GPS, which provides reliable localization to enable the tractor to know where it is at when traversing the field. Such technology is vital to ensuring safe operation of the fully autonomous tractor.

Robust, Reliable Technology Development

In addition to its StarFire GPS technology, John Deere utilizes a robust control stack for path planning. “It handles all of the navigation and obstacle intelligence [necessary] to get through the field in a safe and effective way,” says Schwartz.

The autonomous 8R’s perception system feeds into this control stack to ensure safe movement through a field. It is comprised of six pairs of stereo cameras which provide a 360-deg. view around the tractor. The cameras are able to capture the environment surrounding the tractor and send data about it to the control stack, ensuring the safest path is chosen.

Not only does the perception system need to monitor the tractor’s surroundings, but also the implement being pulled by the tractor, the health of the implement health as well as the field state. All of this information needs to be fed to the tractor so it can appropriately determine how best to do its job. As such, Schwartz says there are a lot of robust AI (artificial intelligence), models and algorithms built into the system to ensure the tractor has a constant feed of high-quality information about what is going on in its environment. 

Schwartz goes on to say the company relied on the use of deep learning and computer vision for its perception system because of their ability to accurately take in the amount of information necessary for this application. “It’s really rich technology, and we built models specifically to work with the cameras,” which are collecting full color imagery as well as near infrared and depth information. 

He notes there is a suite of sensors currently undergoing research and development which John Deere plans to bring to market as necessary and as it brings automation to more agricultural equipment. 

Perception and autonomy in agriculture is less like autonomous driving through a city and more like perception on Mars or the moon, says Schwartz. “We have sparse environments where there is not a lot of information, or it is the same thing over and over again. How do we develop models that are sensitive to things, because once they do detect [an object] it is a big deal. There should not be anything [in a farm field] but if there is we need to deal with it correctly,” he said.  

Agricultural environments are harsh, which can also present challenges when developing perception systems for autonomous equipment. Farmers spend about a third of the day driving into the horizon and into the sun; they may also be operating at night or in fog or dust and in a variety of temperatures. “We have to make sure the system is robust and reliable throughout all of it,” says Schwartz.

He says it has been a fun challenge for the design team to determine the best sensors, models and techniques to utilize to ensure a robust and accurate system is developed. “It has been an amazing journey, and I'm really proud of what's coming out in the fall.”

The autonomous 8R tractor is scheduled to be available to customers starting in fall 2022.

Watch our full interview with Noah Schwartz to learn more about the development of the autonomous 8R tractor.

Autonomy’s Benefits in Agriculture

Like many industries, agriculture is facing a shortage of skilled workers entering the market. This is causing many challenges for farmers, many of whom are already working long days. And as many start to reach retirement age, there will need to be younger generations entering the field to ensure the growing global population can continue to be fed.

It is estimated the global population will reach close to 10 billion people by 2050, increasing the demand for food 50%. This will put a lot of strain on the farming industry.

Autonomy is seen as a way to help overcome these challenges. A fully autonomous machine could be completing a task in the field, freeing up a farmer’s time to focus on other necessary tasks. Use of autonomous equipment enables them to do more with less. “There are always competing priorities [on a farm],” said Deanna Kovar, VP of Production and Precision Ag Production Systems at John Deere, during the company’s press conference unveiling the autonomous 8R at CES 2022. “And if you don't manage them, they're likely to not get done well or not get done at all.” 

The autonomous tractor can help with this as it can run 24 hours a day if necessary, greatly benefiting a farmer’s productivity. In addition, autonomy can help ensure work is done correctly and efficiently as the precision needed to do so is built into the machine and there is no chance of human error. And like an assembly line, the autonomous tractor is able to perform the same operation repeatedly and precisely; no matter the conditions, the operation is done the same every time says Schwartz.

To start, John Deere’s autonomous 8R will focus on tillage. This application helps prepare fields for planting which is integral to the growing season. “It is a core operation and the market suggested that was a great place for us to start,” says Schwartz.

He notes the company is working with its other divisions—specifically construction and forestry—to evaluate and develop autonomous systems for those industries as well, as they are also facing similar labor challenges and need for increased productivity.

In the near term, Schwartz says John Deere is focusing on getting the autonomous 8R into customers hands and scaling out the technology as much as possible. There is a roadmap for expanding the autonomous system out to other tillage implements and eventually other types of agricultural equipment.        

Take a deeper dive into some of the technology utilized in the 8R tractor by watching Farming AI: Self-Driving Tractors from our colleagues at Electronic Design who spoke with Gaurav Bansal of John Deere's Blue River Technology group about the electronics, machine learning and other solutions used to enable fully autonomous operation of the tractor. 

About the Author

Sara Jensen | Executive Editor, Power & Motion

Sara Jensen is executive editor of Power & Motion, directing expanded coverage into the modern fluid power space, as well as mechatronic and smart technologies. She has over 15 years of publishing experience. Prior to Power & Motion she spent 11 years with a trade publication for engineers of heavy-duty equipment, the last 3 of which were as the editor and brand lead. Over the course of her time in the B2B industry, Sara has gained an extensive knowledge of various heavy-duty equipment industries — including construction, agriculture, mining and on-road trucks —along with the systems and market trends which impact them such as fluid power and electronic motion control technologies. 

You can follow Sara and Power & Motion via the following social media handles:

X (formerly Twitter): @TechnlgyEditor and @PowerMotionTech

LinkedIn: @SaraJensen and @Power&Motion

Facebook: @PowerMotionTech

Continue Reading

Handbook Simplifies Advanced Aspects of Mobile Hydraulic Controls

May 10, 2017
Author Brendan Casey lays out his hydraulics know-how that he acquired over 27 years in a simple handbook made for people that want to improve their skills in troubleshooting ...

The Impacts of Electrification on Fluid Power Systems

May 15, 2023
Electrification presents challenges as well as opportunities to re-evaluate and improve upon the design of hydraulics and pneumatics.

Sponsored Recommendations

MONITORING RELAYS — TYPES AND APPLICATIONS

May 15, 2024
Production equipment is expensive and needs to be protected against input abnormalities such as voltage, current, frequency, and phase to stay online and in operation for the ...

Circuit Protection Devices & Busbars

March 13, 2024
With experienced Product Engineers and Customer Service personnel, Altech provides solutions to your most pressing application challenges. All with one thought in mind - to ensure...

All-In-One DC-UPS Power Solutions

March 13, 2024
Introducing the All-In-One DC-UPS, a versatile solution combining multiple functionalities in a single device. Serving as a power supply, battery charger, battery care module,...

Motor Disconnect Switches

March 13, 2024
With experienced Product Engineers and Customer Service personnel, Altech provides solutions to your most pressing application challenges. All with one thought in mind - to ensure...