During the winter of 2021, I worked in collaboration with Ashwin Gupta, a fellow member of the Michigan Mars Rover Perception Team, to develop a proof of concept for a GPU accelerated version of our current obstacle detection pipeline. We researched and used CUDA to create custom GPU implementations of our current obstacle processing algorithms: Pass Through Point Filtering, RANSAC Plane Segmentation, Euclidean Cluster Extraction, and Path Planning. After our proof of concept was completed and tested, we put together a proposal documenting the enhancements we made and the results that we found. Our conclusion was that our GPU proof of concept was 25% faster than our current CPU based detection algorithm. We proceeded to work on the project throughout the spring and summer of 2021, making optimizations and implementing an additional algorithm, Voxel Grid Point Sorting, to further boost our pipeline’s performance. We performed several tests at the end of the summer and found that our optimized GPU pipeline was able to achieve speeds 3x faster than the current CPU implementation. Though it was a tremendous amount of work to develop, our custom system proved to have significantly boosted the performance of our sub-team and was an incredible learning experience for myself. The code for the project, a demo of our pipeline without voxel grid processing, a demo of our pipeline with voxel grid processing, and a playlist of videos we made walking through how the code works.

Over the summer of 2020, I was tasked with developing a prototype obstacle detection that utilized the Point Cloud Library (PCL) to detect distinct obstacles in 3D space. I spent a lot of time researching the capabilities of the library and various possible pipelines, until I settled on the one detailed in this diagram. I went to work implementing various filters provided in the library as well as writing several of my own, for calculating the outline of obstacles and finding a clear path around obstacles. After doing a lot of unit testing for each stage of the pipeline, I had successfully created a working prototype that was able to find a clear path around obstacles, with considerable accuracy. After reviewing the code and the demo, I decided to move forward with the project and have Perception more fully develop the project in the fall of 2020. I created an in-depth code walkthrough to depict exactly what each element of the pipeline does, so the team could quickly understand how the prototype was structured and what each element of the pipeline does. Since then, we have been focused on restructuring and refining the prototype, which can now be found in our current code base.

The development environment for the Mars Rover Perception sub-team has previously been an issue. Our code is designed for an Ubuntu 18 operating system, so each member that wishes to develop our code needs access to this operating system. The development environment setup process can be tedious and painful, especially for people who do not have any experience working in a Linux environment, and was a major inhibitor for new members. I made it a top priority to make the sub-team’s setup process as streamlined as possible, so that new members could get coding as quickly as possible.

To achieve my goal, I first made sure that our environment could be setup in a virtual machine on both Windows and Mac computers, since historically we have only every dual-booted. I experimented with Windows Subsystem for Linux (WSL) and Oracle VirtualBox to make this a reality. I successfully configured these virtual machines to run the Ubuntu 18 operating system on Windows and Mac. I then created a bash script to install all the packages that our code depends on, so that I did not have to manually go through the installation process.

Once I documented the setup process for each of the virtual machines, I helped worked with the sub-team to get them setup with the new development environment. However, Perception’s recent use of the Point Cloud Library has made its coding package much more computationally intensive, and many members who used VirtualBox or did not have high-performance computers were not able to get useful debugging output from our software, because their code ran incredibly slow.

I wanted our sub-team to be accessible to anyone interested in joining and did not want to make owning a high-performance computer a pre-requisite for my members. To tackle this issue, I started researching cloud computing and Amazon Web Services (AWS). AWS was quite overwhelming, so I reached out to the University of Michigan IT Department for assistance in understanding Amazon’s platform. The UM IT department understood my predicament and said that they could accommodate our team on one of their remote computing clusters that runs CentOS 7. They instructed me on how I could create a Singularity container for Perception’s development environment and run Ubuntu 18 on their cluster. I created the container for our development environment, and now all of the members of the Perception sub-team are able to develop and debug their code on a powerful remote computing cluster, that takes minimal time and knowledge of Linux to setup.

Developing a website has always been on my bucket list. I developed this site over the course of the Coronavirus pandemic starting in March 2020 and finished the the bulk of the front-end code in August 2020. It was a very large project and development was definitely slow at times, but in the end it was very rewarding. Front end development can be very tedious and making a site look stylish and be functional is a difficult and sometimes tedious thing to do. I read a couple books to teach me the basics of HTML and Javascript and figured the rest out by googling and experimenting.

Over the summer of 2022 I began learning Rust and after working through all of the documentation I was deeply impressed by the promises it made, the minimalism of the language, and the amount of power it exposes. After reading through those docs I knew I had to gain some experience with this langauge. Consequently, I went ahead and wrote my own Rust back end for this website. This back end is a work in progress as there are various features I would like to add such as in memory file caching and greater functionality beyond that of just a static server. For right now, however, it is functioning as a static server running on Heroku and is providing performance which is identical to the performance provided when the site was hosted on GitHub. The codebase for the website can be found on my GitHub repository.

My first project on the Perception sub-team was developing an obstacle detection that could identify distinct obstacles in the environment, since our current system could only identify blocked regions. In the Spring of 2020, I started researching and prototyping a method called U-V Disparity Obstacle Detection. The idea was to generate two plots of the collected depth data: the U plot, which had depth values 0-7 meters on the x-axis and the rows 0-720 (image height) on the y-axis, and the V plot, which had depth values 0-7 meters on the y-axis and columns 0-1280 (image width) on the x-axis. In the U plot, we can identify the ground as being a slanted linear line, while any obstacles we detect would appear as vertical lines shooting off the slanted one. OpenCV’s Hough Transform could be used to detect the lines and then identify the corresponding obstacle lines in the V plot, which appeared as horizontal lines. The topmost and bottommost points for a vertical line in the U plot told us the topmost and bottommost rows that the obstacle was bounded in and the leftmost and rightmost points for a horizontal line in the U plot indicated the leftmost and rightmost columns that bounded the obstacle. From this data we could identify specific obstacles in our frame. I developed the code and created a demo for this prototype. It was decided that this system showed promise; however, I was asked to prototype a Point Cloud Based Obstacle Detection system before it was decided, which system the team would use in competition.