Hardware/software codesign of real time obstacle detection system using stereovision

Abstract

Obstacle detection and localization are crucial abilities for a mobile robot to navigate in an environment. Therefore, a highly developed environment perception system is necessary. Systems based on stereo vision are strongly recommended due to their high resolution, relatively wide range of measurement, and low cost. Such systems generate a disparity map from two stereo images (left and right), the generation process and disparity map quality depend on the stereo matching technique. In this project, we built a software system for a real-time obstacle detection using stereovision. For the stereo vision module, we used the RANK non-parametric transformation and the Sum of Absolute Differences (SAD) correlation matric to estimate the disparity between the two images, the obstacle detection phase was carried forward using the U-V disparity approach. This approach makes use of the disparity map to build the V-disparity and U-disparity maps, from which obstacles can be easily localized after extracting vertical and horizontal lines from the maps respectively. The software design processes 240×320 pixels images at a rate of 1/3 fps, with a single thread Central Processing Unit (CPU) rate of 667 MHz and a Double Data Rate (DDR) memory running at a frequency of 533 MHz, this result reveals the inefficiency of our system for real-time applications. To overcome this limitation, we involved the hardware/software codesign methodology, where the system is split into hardware and software partitions. The choice of the best partitioning depends on the characteristics required to be present at the final system implementation. For our project, the speed was the main factor to meet the real-time application requirements. In order to get a better partitioning, we used profiling tools to determine the most time-consuming functions in our software system. The profiling shows that the stereo vision module consumes 90% of the total execution time, as a result, we replaced the stereo vision module in our application by a hardware stereo vision module. The software/hardware codesign solution built with the same software characteristics and a hardware stereo vision module running at 7.8 MHz led to a huge improvement in the system’s speed, for 240×320 pixels images the processing rate reaches 78 fps achieving a 234 times acceleration from the original software design.