The main motivation for designing a humanoid robotic platoform was the need to delve into and contribute progress towards the existing problem of
autonomous robotic manipulation and grasping. Having seen robotic platforms like Baxter in demonstrations of major academic robotics labs around
the world we decided to develop iForce (a 14 DOF humanoid robot with two 6 DOF ambidextrous robotic arms) as a barebone humanoid robotic platform to kick-start experimentation of hypothesis towards improving
the the state-of-the-art in robotic manipulation and grasping.
This project is what has led me down the road of utilizing robotics, computer vision and
machine learning with a focus on embodied intelligence for emulating (and ultimately exceeding) human-level dexterity for non-prehensile multi-step manipulations in an
unstructured cluttered workspace along with employing methods for distilling experiences from learned models and mapping them to newer tasks (analogous to human cognition
of life-long learning).
It is a well known fact that manipulation and fine-manipulation [1] is
achievable using tele-operation, this was time and time again demonstrated robustly, and is largely a solved problem in my opinion. But, instilling
human-level dexterity and human-level cognition using intelligence is something that's largely an unsolved or open problem. And this project aimed
at improving upon the 6D pose estimation without using expensive stereo cameras or Lidars (largely due to lack of funds and non-availability of such
equipment in the lab :) ). We designed and fabricated a humanoid robotic platform from scratch, all the way from purchasing the material and elelctronics,
machining the links and joints, assembly, electronic desging, kinematic modeling and then finally experimenting a novel approach towards 6D Pose
Estimation for a detected object for grasp point. The robot had three modes of operations i.e.
- Dependent Mode: This is a teleoperated mode in which the limbs of the robot was controlled using a custom-designed controller box.
- Semi-independent Mode: In this mode of operation the robot would perform pre-defined actions based on the authorized human voice command.
- Independent Mode: In this mode of operation the robot would detect a predefined object (e.g. a bottle) in a cluttered space and localize its 6D Pose relative to the camera frame and using then inverse kinematic equation pick and place the object to a predefined location.
We utilized the existing Tensorflow's Object Detection API, and rather than regressing points for 2D bounding box (BBX) location in the image plane, the
architecture was modified to regress 2D projected vertex locations of object's 3D bounding box. As an analogous measure for 2D confidence for 2D object detection,
we calculated the 3D convex hull for the IOU (intersection over union). Then the object's 6D pose relative to the camera frame was determined by a PnP algorithm.
For this purpose, we created a custom toy dataset that consisted of 5 object classes i.e. a PET bottle, a cup, a shampoo bottle and a glue bottle. The object classes
selected had regular geometrical features, and this was done by design (i.e. on purpose) to avoid complexities of irregular or transparent surfaces. In addition to that,
no deformable objects were selected on purpose to keep the experiment as simple as possible. Once the 6D pose of the object in the clutter was determined relative to the
camera frame, the inverse kinematic model was employed for adjusting the joint parameters to achieve optimal motion to reach that specific object, pick it and place it in a
pre-programmed bin.
The major drawbacks of our approach were:
- The perception-control system was open-loop i.e. any changes in the location after the object 6D pose was determined would not be able to be adapted.
- Lack of exploring with deformable objects.
- The gripper mechanism used force closure, with no tactile sensing, which meant no feedback in case the object slipped from the gripper after a successful grasp.
Despite the scarcity of resources and proper equipment, we refused to let these trivial obstacle hinder our progress, and for that we were able to secure a funding of
80000/= PKR (amounting to 1000 USD in 2018) from the Ministry of IT, Pakistan.
