Third Place Solution and Experience in 2020 ROAR S1 Series Game
Written by Michael Wu, Flaviano Christian Reyes, James Cheney, Marleah Puckett, and Jonathan Wong describing their experience and solution for their EECS 106A/206A final project at UC Berkeley during Fall 2020.
Table of Contents
This project is part of a larger research project called ROAR. ROAR stands for Robot Open Autonomous Racing and is Berkeley’s first AI race car competition. Our project is primarily concerned with making racing agents in the virtual environment. In the future, once we finalize our algorithms through computer simulations, we can port our algorithms to actual cars and try out strategies in reality. For a more detailed overview, please visit our project page.
For this project, we built four agents that use different control strategies. These controllers include:
- A traditional PID controller
- A gym-tuned PID controller where K values were determined by a learned policy with current state as input
- A controller with latitudinal control via PID and longitudinal control via roll calculated using depth camera data
- A traditional Stanley controller
We test these agents using Carla, an autonomous driving software that acts as a framework for testing driving algorithms, and our results are reported in the results section of this website. Furthermore, we were able to test the traditional PID controller agent in a physical car at Richmond Field Station.
Our project also implements and showcases a calibration of the ROAR car, which is ultimately helpful for the next step in the ROAR research pipeline since we would like our live trials to match up with our simulations (i.e. if we tell a car to go straight, it goes straight, just like in a virtual simulation).
In the end, our project not only contributes (1) different types of agents for students and researchers to compare to and (2) improves the simulation-to-reality transition, but we also:
- Set up the Vive Tracker system in Richmod Field Station for GPS/State messages
- Built a gym environment for researchers in ROAR to train their RL agents
These contributions were necessary for us to develop our virtual agents and eventually test them in real life.
Goal of Project
Our end goal was to achieve the fastest lap, whether that be in physical or virtual simulations. For the purposes of this project and the ROAR race this semester, we were primarily focused on getting the fastest lap time in the virtual race simulation that was open to students. We sent our RL PID agent to represent our group in the race because at the time, this was our fastest agent based on our experiments.
Motivation and Questions behind Project
This is an interesting project for a number of reasons. First of all, the field of autonomous driving is a hot topic in research today, and companies have invested a lot of time and money into creating reliable and performant systems that could be used in production level environments. The public as a whole has not embraced automous driving strategies due to (but not limited to) (1) saftey concerns and (2) economic impact (automous vehicles could take away jobs). Although our research group is not actively solving this latter systemic issue, we are aware of the consequences of self-driving vehicles.
What our project is primarily concerned with is speed and reliability, which relates to the safety concerns that the general public has about autonomous vehicles and the performance goals that car companies seek. We want to answer the following question: how do we drive as fast as possible and avoid obstacles or collisions? From this main question comes related subproblems such as:
- How do we know where to go? (A sensing and planning problem)
- How do we get from point A to point B? (A sensing, planning, and actuation problem)
For the purposes of our project, we drive around the virtual and physical environment and collect our vehicle’s coordinates along the way. These waypoints give us the path we need to traverse. And in regard to the second question, we implement our variety of controllers and see which one yields the fastest lap without colliding into other objects.
Real-World Robotics Application
Our project centers around autonomous vehicle research, which car companies have invested time and money in to develop production level systems. Our project work could be useful in figuring out which autonomous vehicle strategies to use in production, but utlimately, its up to others to figure out how they could use our work. For instance, our project could be useful in figuring out autonomous driving capabilities within personal cars, or it could be useful for deploying a feet of autonomous rideshare or delivery vehicles. In the end, it’s all about imagination!
For this project, we built four agents that use different control strategies. The path that each car is following is an alloted set of way points collected from driving a car around a track. For virtual simulations, we use the location data from Carla. For real life simulations, we installed a VIVE tracking system in Richmond Field Station to keep track of where the car is on the track.
Traditional PID Controller
The Real PID controller was difficult to implement, but works very well. Through testing, laptime was drastically decreased and the car didn’t run into any barriers (although there was one close call). Given the challenges we faced dealing with the physical car, we are very pleased with the output of the Real PID.
One interesting observation is that the car was found to speed up at the start of its lookahead length and then slow down until the end of its lookahead length. We implemented a short lookahead since it was beneficial around corners, reducing crashes and creating smoother movement. However, this behavior, combined with a short lookahead length made the car perform poorly on straight runs that can handle faster speeds. To improve the Real PID controller in the future, the lookahead could be improved so that there is a longer lookahead on straight paths that can handle increased speeds. Likewise, as a driver it is known that it is desirable to enter a curve slowly and then speed up while exiting the curve and, when done in reverse, this could be detrimental. Improving the system so that the car never enters a curve at its fastest speed would also be beneficial.
Results shown below. Here is a demo of our PID controller at Richmond Field Station.
Gym-Tuned PID Controller
The video demonstrates an agent using the model trained using observation of current steering, throttle, and speed. We made our own gym environment that adapts well to our existing ROAR codebase and the Carla autonomous driving software that we use for prototyping.
It is worth noting that the trajectory of the vehicle is obviously not smooth, however, it never collides. We think that it is due to the extremely high penalty that we have imposed on collision and therefore the model has learnt to not collide. We suspect that the shaking is due to the fact that the waypoint being read in is not smooth, or in other words, the trajectory we passed in is not exactly smooth.
It is also worth noting that the agent learnt to increase its speed on straight lines and decrease speed at turns. That is a desired behavior because as humans, we would also opt for such actions.
Here is a demo of our RL PID agent.
Traditional Stanley Controller
|Lap Time (Seconds)
|Lap Time (Min:Sec)