search

HybridThingsLab / robodada

http://hybridthings.com/
MIT License
9 stars 1 forks source link
arduino face-api-js node-js robotics

readme

version 1.0.0

ROBODADA

Table of Contents

ROBODADA is an open-source toolkit to map facial expressions to body language of a robot. A web-based interface allows you to move a two-axis robot, record different movement patterns and assign them to face expressions. Activating your webcam, ROBODADA tries to interpret your emotion based on a pre-trained model and activates the robot's recorded movement.

Find examples of ROBODADA in use here.

We use the face-api.js API for face expression detection. We are aware, that machine based emotion detection has to be discussed very critically. ROBODADA underlines, that a machine is not really able to detect human emotions. It just makes decisions on values, which can be measured by a machine.

We see ROBODADA as a playful approach for further discussion:

Our goal is to offer a tool as simple as possible, no coding skills are required. The robot module offers endless possibilites to add different analog materials and extensions. ROBODADA also provides multi-user and robots support in a local network. Users do not need to install any packages or hardware drivers. Just open your browser and select your robot.

Our server is implemented with Node.js, interface elements and emojis are based on OpenMoji. For the hardware part we use a NodeMCU ESP8266 module to control the servo motors wirelessly.

Requirements

To start with ROBODADA you need:

Nice to have

Installation

First of all you need to install Node.js from here. We recommend using the installer.

Check your working Node.js installation by opening your terminal and type

node --version

which should show something like v12.18.3. If your terminal tells you Command not found or similar, something went wrong with your Node.js Installation.

Download the ROBODADA archive from here and unzip it to a suitable directory. There will be a folder called robodada-master.

Next, we will install the neccessary packages: Go with your terminal inside the robodada-master folder and into the folder Serverland. In this folder you see things like app.js and package.json. The following command will automagically install all the needed things for you, while amazing you with nice terminal animations:

npm install

Startup

After a successful installation, again with your terminal and inside the folder robodada-master/Serverland start the server with this command:

node app.js

The server is starting up, telling you:

Searching robots on <Your interface and ip here>
Listening on 3000 in <Your Path>robodada-master/Serverland/controller

It will search robots every few seconds - so don't mind the messages. Congratulations! This wasn't too hard, was it? If you don't have a robot yet, there is a simulation view available, see How to use.

How to use

Take your favourite browser, we recommend Firefox and Chrome and go to http://localhost:3000.

Multi-user and robots support

To access the webinterface from another machine in the local network use the IP and port number of the machine running the server. For example http://192.168.188.24:3000

Right now we have some issues to access the webcam if not running on localhost. For now this can be fixed in the settings of your webbrowser. For example in Chrome open chrome://flags/ and add IP & port of the machine running the server.

security

Build your own Robot

Basic module Customized module
robo1 robo2

So you decided to build your own robot - in the future this will be done by robots for you but until then you have to do it yourselves.

We assume that you have worked with Arduino or similar in the past and you know what you are doing when plugging in soldered stuff into your beloved Laptop/Computer.

Our robot module is just a proposal, and you can modify it in any way you like. Let us know when you did something cool!

It consists of three main components:

Additionally to the items listed in the components you will need:

Pan/Tilt module

pan/tilt

We recommend this module from DaguRobot. You will find it at your favourite distributor over the internet. Make sure you get the long version! Build the module as described in the manual. When fixing the servo hubs, mount them in a position that the servers are in a 50% position when pointing upright/centered.

Servostand
SVG file Cutting
servo_mount_svg stand_cut
Parts Build the stand
stand_parts stand_build

Our servostand is a lasercutted and perfectly fits the pan/tilt module. There are fixing holes for the PCB and power connector. You can cut it out of a 3mm thick material of your choice - we recommend MDF board or something similar, non-conductive and glueable.

Circuit board

Top Bottom With NodeMCU
board_top board_bottom board_controller

The circuit board holds the electric heart of your robot: a NodeMCU wireless controller (free the robots!). Additionally it provides a voltage converter and a (huge) capacitor to provide the servos with power, while supporting a wide range of input voltages. It also has mounting holes and a terminal connector for the powerboard too keep everything organized.

Building the board

circuit

To build the circuit you need:

We use a breadboard, labeled on the top side and cut with a lasercutter. Make sure you hit the board on its holes when labeling your conductor paths with a lasercutter. Drill 3mm holes at the correct positions.

Solder the board as shown in the diagram. Start with the wire for the long connections, then add the other components.

Flashing the Robot

flashing We do this with the Arduino software. Don't worry - it's straight forward.

Before programming the robot, you need to add the board to the Arduino's board library and do some minor configurations. It is well documented here. For NodeMCU boards we recommend to choose "NodeMUC 1.0 (ESP-12E Module)" in the Arduino IDE.

You also need to add the ArduinoOSC library.

After setting up your Arduino environment, go to robodada-master/Roboland/NodeMCU_ESP8266/osc_servo and open up the osc_servo.ino file with your Arduino programming environment. You will notice that the patch consists of two files: beside the osc_servo.ino there is also a config.h.

Open the config.h and fill the following information:

#define ROBO_NAME "Bender"

#define SSID_NAME "name of your WiFi"
#define PWD "password of your WiFi"

Save your changes and upload the patch to your NodeMCU. Your Arduino Software should show the upload progress and something like "completed" in the end. If it does not finish/start the upload, most probably something went wrong with the NodeMCU driver installation or the board configuration.

Put it all together

Nearly there..

Solder two cables (6cm) to your power connector, isolate, and seal the open ends by soldering or crimping sleeves. Fix your power connector in the prepared mounting hole in the servostand.

Fix your circuitboard with M3 screws and washers in the four holes inside the servostand. Add a distance ring between the stand and the board. Connect the power connector to the terminal of the board. Take care of the polarity!

Fix the pan/tilt module in the stand with M3 screws and washers. Guide the servocables through the holes and connect them to the board. Again: Care the polarity! You don't want to fry your robots limbs at the first powerup.

Powering up your Robot

Congratulations when you made it till here! It is time to awake your new robo-friend!

Disconnect the NodeMCU module from your computer, make sure you are in the same WiFi with your computer like your robot.

Check your power supply: It has to provide 6V to 9V direct current the power rail at the connectors pin and ground at the ring. Watch the diagram at the motorstand.

Power up your robot by plugging in the power supply. No white smoke of enlightment? Congrats! The servos should move a little bit and you should feel restistance when you try to manipulate them. Inside the servostand you should see the led of the NodeMCU beeing lit.

Start your ROBODADA server like described here. Visit your server on http://localhost:3000. Now let's choose your robot by clicking the robo-head with the "?" on it. Follow instructions here. After a while your robot should show up. Click it to connect.

When you are painting motionpaths your robot should move accordingly.

It does? You did it!

Good to know

If it feels somehow strange with the directions when painting movement paths, try switching the servos on their connector at the circuit board. Here you can change the mapping left/right/up/down - pan/tilt.

The servos are driven by the voltage converter on the board. If you plan to lift heavy loads with your servos or keep them under permanent stress, consider adding a heatspreader.

Team

ROBODADA is an open source project of the HYBRID THINGS LAB at the University of Applied Sciences Augsburg, Faculty of Design.

concept, design & development

Andreas Muxel & Elias Naphausen

design & development

Fabian Pitzer

development

Andreas Kraus

contact

Any questions or feedback? Get in contact with us.

Credits

ROBODADA is built upon elements of other open source projects and libraries.

Thanks to:

License

The modified emojis from OpenMoji are licensed under CC BY SA 4.0. Changes are documented here.

The source code is published under the MIT license