Environment setup fixes

Simple-Robotics / cosypose

Code for "CosyPose: Consistent multi-view multi-object 6D pose estimation", ECCV 2020.

MIT License

69 stars 15 forks source link

Environment setup fixes #11

Closed swtyree closed 1 year ago

swtyree commented 1 year ago

The current conda environment is a bit brittle and did not work for my system (Ubuntu 22.04, CUDA 11.8, RTX 3090). So I've suggested three changes in this pull request:

Installing the most recent torch and torchvision via pip
Installing the latest pybullet instead of building ./deps/bullet3
Adding urdfdom=1.0.4 to support pinocchio

I have tested this setup using cosypose.scripts.run_cosypose_eval and cosypose.scripts.run_bop_eval, but it's certainly possible that other scripts may fail. If you have other tests that you would like me to perform to validate this PR, please let me know!

ylabbe commented 1 year ago

Thank you!

Which datasets did you test the more recent pybullet version with ? Also, did you have access to a display in your experiment ? I remember that upstream had bugs for rendering certain textures with EGL. This is pretty criticial for rendering and performance so I would first ensure that the bugs I fixed in the deps/bullet3 are properly fixed in upstream. On the long term I think the real solution is to use another renderer like Panda3D in megapose.
Can we relax the constraint on pinocchio to use the latest version from conda? It should install the urdfdom dependency.

swtyree commented 1 year ago

Hi Yann,

Here are the two datasets that I tested:

TLESS via cosypose.scripts.run_cosypose_eval; verified that there are no crashes and the scores seem reasonable, but didn't scrutinize the accuracy
HOPE (as described in #13) via the BOP scripts; verified the accuracy by submitting the test predictions on the BOP evaluation server; metrics were actually slightly better than my previous CosyPose submission 🤷‍♂️

Is there a dataset that you know failed with prior versions of PyBullet? I can try to test a known failure case. As noted in a reply to #2 (https://github.com/Simple-Robotics/cosypose/issues/2#issuecomment-1397890766), I wasn't able to build the fork in deps/bullet3, which motivated trying the latest version.

I will try to install the latest pinocchio and report back about the urdfdom dependency...

KushnirDmytro commented 1 year ago

Thank you for your contribution, @swtyree! These environment setup fixes are very useful indeed. The previous setup update was working fine only on Ubuntu 20.04, while yours indeed fixes the setup for Ubuntu 22.04. So I have tested it and the setup is functional. Not only evaluation scripts, but data visualization notebooks are working well as well (which is even more painful in package setup).

ylabbe commented 1 year ago

The modifications that differ between my fork of bullet and upstream were:

EGL device selection for handling the case with multiple gpus. This has been fixed in upstream https://github.com/bulletphysics/bullet3/pull/4024
Improper rendering when a link in the urdf has multiple visual textures. This is not actually not a problem for BOP objects. It affects the renderer for some robots I used in another project but this does not affect cosypose.
An bug in the b3ComputeViewMatrixFromYawPitchRoll function that I had fixed here in my fork. It looks correct in upstream.

I am quite confident we can merge your PR now. Thanks again @swtyree !