| Press release (English) | link |
| Press release (Spanish) | link |
| Read the paper | link |
Together with AMD, Intel, Ford, Harvard, Klagenfurt University, Georgia Institute of Technology, Boston University, Johannes Kepler University Linz, Barnard College, Columbia University and Carnegie Mellon University we are thrilled to introduce the beta release of RobotPerf™ Benchmarks, an advanced benchmarking suite crafted specifically to evaluate robotics computing performance using ROS 2 as its baseline. In this beta release, we not only showcase new benchmarks and results but also introduce novel visualization capabilities. The complete release is available at https://github.com/robotperf/benchmarks/releases/tag/beta.
You can download our paper here.
RobotPerf™ - Why?
The myriad combinations of robot hardware and robotics software make assessing robotic-system performance challenging, specially in an architecture-neutral, representative, and reproducible manner. RobotPerf™ addresses this issue delivering a reference performance benchmarking suite that is used to evaluate robotics computing performance across CPU, GPU, FPGA and other compute accelerators. The benchmarks are designed to be representative of the performance of a robotic system and to be reproducible across different robotic systems. For that, RobotPerf™ builds on top of ROS 2, the de facto standard for robot application development.
Roboticists can use performance data to develop more efficient robotic systems and choose the appropriate hardware for each robotic application with the help of performance data. It can also aid in comprehending the trade-offs between algorithms implementing the same skill.
RobotPerf™ - A Quick Overview
RobotPerf™ provides an open reference benchmarking suite centered around ROS 2, making it an essential tool for robotic architects. Through these benchmarks, decision-making about hardware and software components of robotic systems becomes a breeze. The suite ensures that evaluations are:
- Architecture-neutral
- Representative of real-world robotic system performance
- Reproducible across different systems
The benchmarking initiative is backed by a consortium of industry, academia, and research lab leaders, promising technology-agnostic, vendor-neutral, and unbiased evaluations.
New Benchmarks and Results
Our benchmarks cover the entirety of the robotics pipeline, including:
- Perception
- Localization
- Control
- Manipulation
While these form the core of our suite, be on the lookout! New categories may emerge as we continually enhance the platform.
Introducing New Visualization Capabilities
With this release, RobotPerf™ elevates the evaluation experience by introducing innovative visualization capabilities. For robotic architects, understanding computing performance intricacies is fundamental. Recognizing this, RobotPerf™ brings forth a set of visualizations tailored for utmost clarity and depth.
Through these visualizations, RobotPerf™ transmutes the multifaceted realm of robotics computing performance into tangible insights, simplifying and enhancing the architect's role.
Grey-Box and Black-Box: Two Benchmarking Approaches
We're not just about results; we're also about flexibility. RobotPerf™ offers two distinct benchmarking approaches:
- GREY-BOX - Tailored for real-world applications, this detailed approach incurs a minimal average latency of only 3.3us.
- BLACK-BOX - Ideal for quick prototyping, it offers a simpler but slightly less detailed analysis.
Both approaches ensure that every robotic architect finds a benchmarking solution aligned with their specific needs.
Wrapping Up
The beta release of RobotPerf™ Benchmarks marks a milestone in robotics computing performance assessment. With novel benchmarks and groundbreaking visualization capabilities, the platform is set to redefine how robotic architects approach system design and optimization.
We invite you to dive into the world of RobotPerf™ and experience firsthand the revolution it brings to the robotics landscape.
Stay tuned for more updates at https://robotperf.net, and happy benchmarking!
