This project is to develop a computer vision assistant pick and place game app for Android phone to play with the 3D printed robotic arm.  An example of pick and place game is chess game. You can propose any kind of game using the robotic arm.

In general, the app will use smart phone camera to find an object position and control the robotic arm to pick and place it.  The user interface of the app will include UI to control the robotic arm movements such as placing the gripper to particular X,Y,Z or home position, start/stop stepper torque. Connection between the smart phone and the robotic arm is via BlueTooth LE for passing G-code to it to perform a pick and place action given by the robotic arm firmware.

The robotic arm is required to enhance to do auto home position. That's to move X,Y,Z axis to the starting coordinate when it is powered on. That can be done by first modifying mechanical structure for adding end stop switches (see figure 2) using 3D printing and modelling, then to modify the firmware of  the robotic arm to read end stop switch signal. The firmware is Arduino program. It can be found in MakerLab's github

For this project, familiarity in C/C++ and Java programming is definitely a plus. Tutorials will be given on 3D printing & modelling,  Arduino programming platform and Android app development. This project is best for a group of 2 students.

The project deliverable includes:

  • the game app
  • enhanced robotic arm with automatic home positioning