Scene Builder

[Jmeyer1292]

After talking with @Levi-Armstrong , I'd like to remove the working assumption that there is one "core" URDF file that generates a scene and everything else is just "attached".

I propose that we make our own domain specific language around workcell modeling (that could closely follow URDFs example) that offers the following features:

• The ability to construct the scene entirely through code
• The ability to parse a URDF into a scene "assembly"
• The ability to compose, mutate, and delete links and assemblies to form desired workspaces.
• Perhaps the ability to specify a "solver" for a particular sub-graph of the scene

Our modeling might allow for parallel kinematic chains - the user would need to provide a solver for these components.

A good use case to build toward would be a first-class representation of "tool changing" - the frames and geometry of the attached tool would join a chain.

There's a lot of stuff to figure out here, but I think it would be worth it.

Food for thought.

[gavanderhoorn]

Pixar USD (emphasis mine):

USD provides for interchange of elemental assets (e.g. models) or animations. But unlike other interchange packages, USD also enables assembly and organization of any number of assets into virtual sets, scenes, and shots, transmit them from application to application, and non-destructively edit them (as overrides), with a single, consistent API, in a single scenegraph. USD provides a rich toolset for reading, writing, editing, and rapidly previewing 3D geometry and shading. In addition, because USD's core scenegraph and "composition engine" are agnostic of 3D, USD can be extended in a maintainable way to encode and compose data in other domains.

Something I've been eying for some time now.

I propose that we make our own domain specific language around workcell modeling [..]

Using something existing has the advantage of being able to reuse whatever tooling, experience and assets are available. In an industrial setting, that is a huge thing.

USD doesn't necessarily have that, but other formats do.

---

Edit: just noticed these guys. They appear to already have quite a bit of infrastructure (plugins, parsers, etc) for USD and various modelling tools.

[johnwason]

OpenRAVE uses COLLADA with extensions for robot-specific information. I don't know if it would be as powerful as USD, however COLLADA is widely supported and ROS already has some support for COLLADA based models

http://openrave.org/docs/0.8.2/collada_robot_extensions/

[gavanderhoorn]

While I'm definitely in favour of using something like Collada, there has been (quite) some discussion about it in the past in the context of the "next generation" description format for ROS 2. See the ROS Discourse: robot description formats category for some context.

tesseract is not ROS 2 of course. So perhaps it's perfectly ok to use Collada here.

[johnwason]

I did a quick skim of the discussions. I agree that we should try to harmonize with the 3D modeling community as much as possible rather than having a separate file formats. Being able to use Blender or Maya with the same workflows as the gaming industry would greatly simplify asset management. There will need to be some extensions to the existing formats, but it should be pretty straightforward to create plugins for existing 3D editors that handle the extra data. For my own near term projects, I'd like to have the ability to embed the 3D model in the robot firmware so it can be read by end users. Right now COLLADA seems to be the best fit for my requirements as it already has a set of robot extensions defined, is widely supported in 3D editors, and has the *.zae format for combining multiple xml files and textures into a compressed archive. I am open to being persuaded otherwise though.

[gavanderhoorn]

Right now COLLADA seems to be the best fit for my requirements as it already has a set of robot extensions defined

Well .. that's the thing: it hasn't. Unless you are talking about the elements that are actually part of the specification, the extensions that OpenRAVE uses are just that: extensions.

The problem with those is that it's undefined what editors and viewers should do with such extensions if they encounter them and they don't have support for them.

Some editors ignore them, others discard them on-write and again others will refuse to load files with extensions they don't support. That is a big problem, as you don't know what will happen to your robot metadata if you load it into an editor.

Support for other things that are actually part of the spec is also spotty, such as kinematics, dynamics and simulation properties.

Right now COLLADA seems to be the best fit for my requirements as it [..] is widely supported in 3D editors

This is somewhat true, but with the caveats I mentioned above. Support is rather spotty and often not very consistent. SolidWorks for instance does not natively support it and plugins available are inconsistent in their support for it.

As much as I'd like Collada to become/be the standard for robot geometry and other metadata, it may not be the best candidate.

---

Edit:

and has the *.zae format for combining multiple xml files and textures into a compressed archive

that is a nice feature, but unfortunately also not universally supported (and if it is supported, there are differences in how various loaders treat paths inside .zaes).

[johnwason]

I agree that COLLADA currently has lots of problems. The question is does it make sense to start from scratch with something else, or would it be easier to fix the problems in COLLADA. We can always propose a new revision of the COLLADA standard with the required features and update Blender to support them. Convincing the commercial software developers to support the new features will be tricky, but this is something the ARM Institute could probably help with.

Writing a plugin for solidworks to export COLLADA wouldn't be overly difficult if the current ones are inadequate, although it would be time consuming. This problem will exist for any newer file format.

[gavanderhoorn]

Writing a plugin for solidworks to export COLLADA wouldn't be overly difficult if the current ones are inadequate, although it would be time consuming.

Don't forget maintaining it. That is the bigger problem in that case.

Without Dassault being problem owner, it's going to be a community thing and that is not going to work - in my experience.

But that is of course only one of the tool vendors.

[gavanderhoorn]

The question is does it make sense to start from scratch with something else, [..]

As you can read in the posts in the robot description formats category, I'm a proponent of fixing things and reusing what is already there (I proposed looking at Collada I believe).

or would it be easier to fix the problems in COLLADA

I can't really gauge how easy it would be to fix things: some of it would be in the standard, but a big part of it would be the implementations and support in modeling and design products. It would take some effort to get all of them behind a goal like this, especially because, I believe, the 3D industry has "moved on" (to FBX for instance).

Interestingly though: Collada is an Industry 4.0 standard (in Germany and the rest of Europe that participates in Industry 4.0). Perhaps that could be exploited somehow.

If the ARM institute could help in getting the various stakeholders together and get things moving, that may be an option as well.

[johnwason]

One of the funded topic areas at the ARM is "Software Interoperability," and this seems like it would be a very relevant project. I'll ask about it next week at the SWG meeting. I am releasing a package industrial_payload_manager very shortly as part of an ARM project that ran into some of these file format issues.

@shaun-edwards any thoughts?

[johnwason]

I pushed the industrial_payload_manager package that contains functionality to load payloads from URDF files and manipulate them in the planning scene/RViz separate from the robot_description parameter. This implements some of the functionality discussed here, but is still an incomplete solution for mutable environments.

https://github.com/arminstitute/industrial_payload_manager

The industrial_payload_manager currently only supports RViz and MoveIt. I will be adding support for Tesseract in the near future.

[Levi-Armstrong]

I have been looking into this and I am thinking about creating a simple tesseract_scene_graph using boost graph where links will be the vertexes and joints will be edges. Any thoughts?

[gavanderhoorn]

USD too complex eh? :)

[Levi-Armstrong]

I pulled it down and built it but it would take some time to get familiar with it before I think I could have anything useful. Also It currently has to be built from source. I am not opposed to USD but just don't have the time at the moment. Do you know anyone that is familiar with it?

[Levi-Armstrong]

A scene graph has been implemented leveraging boost graph and is merged in to master.