Replicate Y2M2 demo offline with robot showing the exercise in simulation

vvasco commented 5 years ago

We want to replicate the Y2M2 demo offline with:

a disembodied camera;
a simulated R1 in gazebo showing the exercise.

This will allow us to do a back-to-back comparison between the interaction with a real robot and its visual counterpart, in order to evaluate the importance of the robot's embodiment on the user.

pattacini commented 5 years ago

According to the protocol we would need to include an "occlusion" within the environment. Also, it's important to set up the point of view of the robot programmatically.

pattacini commented 5 years ago

It is important to make the speech as well as the facial expression working on the simulated R1.

vvasco commented 5 years ago

The virtual demo is now working and the related code can be found in the branch feat/virtual-robot.

Assistive-rehab installation

Instructions for installing assistive-rehab can be found here, with related dependencies. Optional dependencies including TensorFlowCC, fftw3 and GSL are required.

Replace <install-prefix> the absolute path where you want to install the project and do:

git clone https://github.com/robotology/assistive-rehab.git
git checkout feat/virtual-robot
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=<install-prefix>
make
make install

Additional dependencies

RealSense
cer
gazebo
gazebo-yarp-plugins
cer-sim, branch feat/face-expressions:
- after cloning do: git checkout feat/face-expressions
speech

Requirements

RealSense camera.
NVIDIA graphics card for running yarpOpenPose and actionRecognizer.

Run the demo

The folder app/scripts contains the template AssistiveRehab-virtual.xml.template including all the relevant modules. The template is installed in <install-prefix>/share/ICUBcontrib/templates/applications. You can copy the template in Yarp data directories, remove the .template extension and customize the xml by defining the cuda-machine node.

To run the demo, first run yarpserver. Connect the RealSense to your laptop. Open yarpmanager, run the Assistive_Rehabilitation_Virtual_App and connect. The simulation environment with R1 should appear, as following:

You can start the interaction using the API, as described in #199.

cc @ddetommaso @PCH313

robotology / assistive-rehab