These lessons are derived from working towards the Amazon Picking Challenge and interacting with other teams during the final competition at RoboCup in Leipzig, Germany.
Our software stack consisted of modules for perception, grasping and motion planning tied together using ROS and the SMACH state controller.
Codebase & Testing
- Discuss and write down a list of code style rules and naming conventions. Preferably have a pre commit hook for clang format. Could have borrowed Googles style guide. This improves uniformity and readability.
- Have a basic code review system to maintain code quality.
- Design the architecture so that the the modules can be tested independently.
- Don’t abuse ROS: Use it only when passing messages to other modules or for visualization in RViz. Prefer using function calls otherwise. We used it everywhere.
- Better state machine and data management: A lot of time was wasted tweaking SMACH to pass data (pointclouds, trajectories, poses) between the states as we didn’t have a separate data store.
- Feature freeze 1 month before competition would have helped us isolate failures in certain combinations of items (eg: low classification confidence). We could have then modified the state machine to skip those combinations.
- Strategic ordering: Focus on maximizing score instead of trying to pick every item. We could have optimized for picking items with most points and grasping confidence first.
- An visual introspection tool to help record and debug grasps and cached trajectories would have been useful. UBonn’s Nimbro had some superb rqt plugins to do this.
Motion Planning
- Trajectory caching: Trajectories to move the end effector between bins and common poses were pre-cached for speed. Only in-bin planning was done online and it was hard to tune its parameters for consistently good paths.
- Interestingly, no team used in-bin collision checking. They treated the shelf bin as a convex hull for faster planning and preferred trajetories near the top.
Robots & Actuators
- Early on, we made a decision to migrate from the PR2 robot platform (which was already available to us) to the UR5. The PR2 had a moving base and a slow telescopic spine which made planning more complex.
- The UR5 was an amazing robot - it had tight ROS integration and was force compliant. We were up and running in no time. Universal Robots loaned us one. Thanks guys!
- However, with the UR5, we had reachability issues - We could not easily plan paths to all the corners of the shelf bins as our end-effector geometry limited the workspace. At the competition, we were the only team using the UR5. Many of the other teams had a UR10 robot with a linearly actuated end-effector - resulting in a much larger workspace, better reachability and easier planning. For next year, UR10 with a linearly actuated arm would be the way to go!
End effector
- Due to the geometry and weight of majority of the items, most teams prefered to use suction grasping end-effectors. This eliminates the need for accurate perception and planning force-closure finger grasps.
- The MIT team’s end-effector was bulky - it had finger graspers and suction. Delft’s was more elegant - suction cup and a clamp. These enabled them to grasp all items - even the 3lb dumbbell and black mesh pencil cup.
- Our end-effector and suction system design was finalized early on. If we had more resources, we would have focussed on improving the design - the better performing teams had a smaller suction cup and a vacuum pump. We used a shop-vac!
Perception
- Controlled lighting: The top teams had a lighting setup mounted inside their workspace. This allows them better control for using CNNs for item segmentation and classification. It would also allow them to match lighting conditions of their testing and training data.
- Camera selection: We were one of the few teams using the large Kinect 2. Many
teams used the smaller Intel RealSense SR300 depth cameras. A huge drawback of
the Kinect2 was that it used time-of-flight compared to the rest which used
Structured light. The ceiling Halogens lamps at the event venue emitted IR
radiation which interfered with our sensors causing noisy pointclouds. Some
teams put a tarp over their setup to mitigate this.
- A smaller sensor with a shorter range would have helped increase the search space making in-bin planning easier allowing us to grasp items in tighter spaces.
- We could have explored fusing multiple depth images to give denser pointclouds for grasp pose estimation.
Shipping to event venue
- We packed our robots and computers in Pelican cases and transported it as checked baggage. When we received it, the computers were all banged up.
- Tips for transporting computers: Remove the GPUs and fill the computer case with anti-static bubble wrap. Wrap the entire computer with bubble wrap to keep them cushioned.
- Be mentally prepared for Murphy’s Law to upset your best-laid plans. Have backup components - relays, dev-boards, computers, monitors. If something goes wrong, stay calm. Panic leads to more mistakes.
- If the event is in a different country, check their AC power sockets, voltage and frequency rating. If different, buy the socket adapters and transformers beforehand. We accidentally blew up the shop-vac by plugging it into a different AC source. Thankfully, computer power supplies can handle a variety of AC voltages.
Teamwork
- It was my first time working in a 5 person team for months together. This MRSD project course was structured similar to the industry and I learned about various engineering processes and workflows.
- Initially I explored new skills like CAD modeling and 3D printing for prototyping end-effectors and suction gripper mechanisms. As deadlines got closer, I shifted to tasks which were my strengths.
- If you have prior experience and feel strongly about a design choice or architecture, be assertive and speak up. Don’t be too afraid of being confrontational.
- I did way too many things that semester - heavy courseload, side projects, research and extracurriculars. I learned to triage tasks and manage time effectively.
Comments