Downsample meshes? (Suggestion)

[markusgft]

Hi, and first and foremost, thanks for the great work! I checked out the repo and noted that the robots/urdf* folder is roughly 900 MB. This seems to be the case because of extremely high-resolution meshes. Would it be an option to downsample the meshes a bit? For users, that would mean a more lean repo, faster collision checking in classical motion planning, faster rendering in visualization, etc.!

[bdelhaisse]

Hi,

Welcome! :) The robots/urdfs/ folder is indeed the largest one in the (PyRoboLearn) PRL framework. This is because it contains the meshes of all the robots that are accessible through PRL. When I started PRL, I downloaded, cleaned and edited several URDF files from different repositories, and converted several mesh files into formats that were accepted by Pybullet. However, I did not downsample the meshes at that time because I thought that:

1. I wanted to avoid people having different results (and possible issues) because of the meshes; the meshes are the same ones that are available in the original repositories (except they might have been converted to different formats).
2. Depending on the task, it might require to use high-resolution meshes, and I don't think having different resolution mesh files for each robot on the repository is a viable option.

However, here are possible solutions that I thought that I (or someone) could do:

1. You can use meshlab to downsample the meshes yourself and create a new URDF pointing to these created meshes. For each robot in PRL, you can specify the urdf_path to the new URDF file that included these simplified meshes. This is probably the easiest option.
2. I thought about for each robot creating a separate URDF file <robot_name>_simplified.urdf that would use primitive collision shapes for faster collision checking. This would avoid to create simplified meshes for each robot and increase the size of the repository unnecessarily. However, for some non-convex meshes this would require to estimate them using a collection of primitive shapes. Also, using simple primitive collision shapes might not suit everybody. Some people might want a resolution between a primitive shape and an original mesh.
3. I recently implemented several parsers for robotic files (such as URDF, MuJoCo, etc) as I am currently integrating other simulators into PRL, and sometimes I need to convert one type of file (e.g. Mujoco file) to another one (e.g. URDF file), and convert meshes to a certain format at runtime such that they can used by the specified simulator. Note that right now, the simulator that is fully supported in PRL is pybullet, I am currently finishing the integration of the mujoco simulator. Anyway, I could probably provide an argument for each robot class that specifies in some way if I should downsample the meshes or not (and at what resolution), and if it is the case, I could parse the URDF file and check the size of each mesh at runtime and downsampled them using the trimesh library. However, I need to check in more details the documentation of this trimesh library (if an algorithm like the quadratic edge collapse decimation algorithm is implemented or not) to see if this is possible in the first place, and how to do it. If it is not possible in trimesh, I might need to have a look at the open3d library. If possible, I think this solution would be the most general one, and based on some values for the resolution argument, I would also probably implement the solution described in point 2; i.e. replace meshes by primitive shapes (or a collection of it) at runtime. Having said that, this solution might take some time to implement based on how much the other mesh libraries provide the required features or not.

Let me know what you think.

[dbonattoj]

Hi @markusgft, @bdelhaisse

I would like to propose something if the solution 2 above is chosen: Maybe you could have full, medium and low precision zip files and ask the user which version they want in a custom build step in the setup.py file[1] ?

python setup.py meshes --mesh_precision=[high,medium,low,all]

Depending on the precision, those zip files can be automatically extracted in the right folder without any overlapping in the names during the build step. if someone decide to download all the precision levels, a robot folders may look like this: /robotxy/mesh_low.urdf /robotxy/mesh_med.urdf /robotxy/mesh_high.urdf

Downloading a separate zip file and extracting it let us have a smaller file to download from a faster server :)

[1] https://jichu4n.com/posts/how-to-add-custom-build-steps-and-commands-to-setuppy/

[markusgft]

Hi all, thank you for the detailed responses! I actually like @dbonattoj 's proposal very much! However, I want to emphasize again that I opened this issue only because it might be a good future addition to PRL, and not because it is a pressing urgent problem. From my personal experience with open-source projects, I had some projects where I got feedback from users that the repository was not "lean" enough ... and we were talking about something like 50 MB total size! For critical users like that, it may be worth having a feature where they can pull small-size version of the URDFs. Alternatively, would it be an option to only pull the meshes on-demand ... if someone's only interested in working with fix-base robots, he won't benefit from having all the mobile robots locally available, for instance. But again ... just writing down thoughts, no urgency!

[dbonattoj]

I don't know the status of partial cloning in github but in the future maybe this could provide a solution?

https://about.gitlab.com/blog/2020/03/13/partial-clone-for-massive-repositories/