O-CNN repository for local installation

[movefasta]

Greetings, Andrej! Thanks for impressive and well documented work!

In README.md you mention that for local installation should be installed your O-CNN fork, but in drl_grasping.repos specified official repo URL https://github.com/AndrejOrsula/drl_grasping/blob/3d9fa038dc2855b9ca97d9d1570b06abe17a0b89/drl_grasping.repos#L40

What repo i should use?

Thanks!

[AndrejOrsula]

Hello, the fork specified in README.md is the correct one.

You can also follow docker/Dockerfile that has all instructions, in case there are some problems. https://github.com/AndrejOrsula/drl_grasping/blob/3d9fa038dc2855b9ca97d9d1570b06abe17a0b89/docker/Dockerfile#L96-L98

Please let me know if you encounter any other issues :smile:.

---

... but in drl_grasping.repos specified official repo URL https://github.com/AndrejOrsula/drl_grasping/blob/3d9fa038dc2855b9ca97d9d1570b06abe17a0b89/drl_grasping.repos#L40

This line is actually commented out (#) because it is not a project compatible with colcon. https://github.com/AndrejOrsula/drl_grasping/blob/3d9fa038dc2855b9ca97d9d1570b06abe17a0b89/drl_grasping.repos#L33

But thanks for letting me know! I will update it to reduce the confusion.

---

Btw. If you only want to test this repo, I still recommend using the pre-built Docker image (more info in README.md). Local installation is needed only if you are planning to do some modifications that are currently not exposed as parameters.

docker pull andrejorsula/drl_grasping:latest

[movefasta]

Yep, i already installed project via docker image and it's working fine. So test is OK!) After that i decided to understand how it works more deeply, install locally and (maybe) change some parameters and algorithms. Is it possible for docker variant ? Where exposed parametes listed ?

[AndrejOrsula]

... change some parameters and algorithms. Is it possible for docker variant ?

Yes, it is possible to change parameters also inside docker container - however, it is not ideal. I generally used this approach for few very small changes (e.g. use of different algorithms, different robot, and combinations of hyperparameters). The workflow can look something like this:

## Spin up docker container with `bash` cmd
<drl_grasping dir>/docker/run.bash andrejorsula/drl_grasping:latest /bin/bash

## Edit files (any file)
## Inside docker, this repository is awkwardly in `./repos/drl_grasping/src/drl_grasping` relative to WORKDIR
## You can install editor of your choice if you do large changes (`nano` is used here)
# Selection of env, algo, robot, ... (or you can use custom `ros2 run drl_grasping train.py $YOUR_TRAIN_ARGS`)
nano ./repos/drl_grasping/src/drl_grasping/examples/ex_train.bash
# Hyperparams for algo, feature extraction, randomiser, ...
nano ./repos/drl_grasping/src/drl_grasping/hyperparams/tqc.yml
# Environment config
nano ./repos/drl_grasping/src/drl_grasping/drl_grasping/envs/tasks/__init__.py
# Further environment config (ugly right now)
nano ./repos/drl_grasping/src/drl_grasping/drl_grasping/envs/tasks/grasp/grasp_octree.py

## Run commands/examples
ros2 run drl_grasping ex_train.bash
ros2 run drl_grasping <executable>

So, you can do it all inside docker. This is just an example - you could instead modify the Dockerfile to include some persistent volumes or do something completely different. But I generally used local setup when trying out more complicated edits.

---

Where exposed parameters listed ?

I listed the files above, but I will give some more detailed description here.

In ex_train.bash, you can set the desired environment (including robot) and algorithm combination. It also allows you to continue training of previous agents, store replay buffer and others. It is pretty much just an interface for train.py that also spins up MoveIt2 action server. When run, it should show you what is the corresponding ros2 run drl_grasping train.py $YOUR_TRAIN_ARGS script.

https://github.com/AndrejOrsula/drl_grasping/blob/3d9fa038dc2855b9ca97d9d1570b06abe17a0b89/examples/ex_train.bash#L1-L49

The ex_enjoy.bash is almost the same. You just select whatever trained agent you want to evaluate. Datasets stay the same, so you need to change them manually if you want different for training/testing.

https://github.com/AndrejOrsula/drl_grasping/blob/3d9fa038dc2855b9ca97d9d1570b06abe17a0b89/examples/ex_enjoy.bash#L1-L41

All hyperparams/*.yml contain hyperparameters for algorithms (e.g. SAC), see stable-baselines3 docs for more info. It also contains hyperparameters for feature extractor and some config for randomizer (for domain randomization).

https://github.com/AndrejOrsula/drl_grasping/blob/3d9fa038dc2855b9ca97d9d1570b06abe17a0b89/hyperparams/sac.yml#L187-L233

In drl_grasping/envs/tasks/__init__.py, you can find config of the registered environments. Some additional environment config for each environment is also directly in their corresponding file, e.g. drl_grasping/envs/tasks/grasp/grasp_octree.py.

https://github.com/AndrejOrsula/drl_grasping/blob/3d9fa038dc2855b9ca97d9d1570b06abe17a0b89/drl_grasping/envs/tasks/__init__.py#L145-L187

https://github.com/AndrejOrsula/drl_grasping/blob/3d9fa038dc2855b9ca97d9d1570b06abe17a0b89/drl_grasping/envs/tasks/grasp/grasp_octree.py#L15-L53

There are many many parameters, some of which were experimental and do not provide any/much improvement, e.g. stages in curriculum are disabled by default. As I wrote somewhere else, I might clean it up later if I have time for it. For now, feel free to ask if there are certain parameters that are unclear.

If you want to create your own tasks inside Ignition Gazebo, I recommend you taking look at Gym-Ignition first. Then Reach task and #87 list all that needs to be added for each task to make them work directly with this repo.

[janisa9]

@AndrejOrsula I also encountered an issue. I installed everything mostly by following DockerFile instructions, however pyoctree module wasn't found: File "/home/janisa/drl_grasping_rep/drl_grasping/install/lib/python3.8/site-packages/drl_grasping/utils/conversions.py", line 7, in <module> import pyoctree ModuleNotFoundError: No module named 'pyoctree' Any Ideas?

[AndrejOrsula]

@janisa9 It seems that pyoctree is not installed together with ocnn for some reason. Can you try the following?

export PATH=<path_to>/O-CNN/octree/build:${PATH}
export PYTHONPATH=<path_to>/O-CNN/octree/build/python:${PYTHONPATH}

Please let me know if that helps.

---

I totally forgot to mention it in README, but I added such workaround inside the entrypoint. I will try to look into a proper fix. https://github.com/AndrejOrsula/drl_grasping/blob/3d9fa038dc2855b9ca97d9d1570b06abe17a0b89/docker/entrypoint.bash#L10-L12

[janisa9]

@AndrejOrsula Unfortunately this didn't work.

[AndrejOrsula]

Hmm. Does import ocnn work for you?

python3
>>> import ocnn

---

And just to be sure, these are the instructions that you followed. Right? Everything went fine without any error? https://github.com/AndrejOrsula/drl_grasping/blob/3d9fa038dc2855b9ca97d9d1570b06abe17a0b89/docker/Dockerfile#L96-L103

[janisa9]

Thank you for support. Yes I was following the instructions, I didn't have permissions for some folders and therefore installation was incomplete and thus the way I tried to solve the permission issue wasn't the right way :smile:

`unning install error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the installation directory:

[Errno 13] Permission denied: '/usr/local/lib/python3.8/dist-packages/test-easy-install-120112.write-test'

The installation directory you specified (via --install-dir, --prefix, or the distutils default setting) was:

/usr/local/lib/python3.8/dist-packages/

Perhaps your account does not have write access to this directory? If the installation directory is a system-owned directory, you may need to sign in as the administrator or "root" account. If you do not have administrative access to this machine, you may wish to choose a different installation directory, preferably one that is listed in your PYTHONPATH environment variable. `

Installing convert-caffe2-to-onnx script to /usr/local/bin error: [Errno 13] Permission denied: '/usr/local/bin/convert-caffe2-to-onnx'

Now by giving permissions to these folders it and adding thepaths, it can find the pyoctree. :partying_face:

[AndrejOrsula]

@janisa9 Yes, the instructions in Dockerfile assume that you have sudo permissions.

If not, sudo apt-get install ..., sudo cmake install/cmake --build . --target install, sudo python3 setup.py install... and similar are required if you want to install on system-level. Although for some of them, user installation or target install directory can also be specified.

[janisa9]

@AndrejOrsula Yes!

There was one more issue, the stable-baselines3 has new release 1.1.0 which got installed by following instructions, but I suppose you used different version as I got following error:

drl_grasping/install/lib/python3.8/site-packages/drl_grasping/envs/tasks/grasp/curriculum.py", line 482, in _log_curriculum logger.record("curriculum/current_stage", AttributeError: module 'stable_baselines3.common.logger' has no attribute 'record' I installed 1.1.0a7 version which seems to work.

[AndrejOrsula]

Closing this issue due to inactivity (two weeks).

Feel free to re-open if anyone encounters this issue again and suggestions above do not provide a solution.