Roadmap – feedback welcome!

[osanseviero]

TensorFlow Graphics was first released at Google I/O in 2019, and since, the library grew a lot, both in terms of core components, but also in terms of infrastructure which was put in place to facilitate collaboration with the community. We are excited to share with you what we believe would be great next steps for TensorFlow Graphics, and would love to hear your thoughts & suggestions!

Top level goals

• Build standard structures and implementations for differentiable 3D and graphics development.
• Provide a central place to access tooling and implementations.
• Make it easier for research results to be implemented and replicated.

Our current efforts focus in the following areas:

• A Distill.pub article on the importance of differentiable graphics and dynamics.
• Packaging mainstream 3D datasets to support research.
  • Examples include ModelNet, ShapeNet, Pix3D, etc.
• Providing complete implementations of state of the art papers.
  • CvxNet, PointNet, Neural Voxel Renderer, Point-based convolutions, and more!
• More Colab demos!
• Begin Google Summer of Code projects!
  • Implicit representations using Occupancy Networks, DepSDF and IM-NET.
  • Point-based convolutions.
  • Mesh R-CNN implementation.
• Infrastructure work.
  • Google <-> Github pipeline to facilitate open-source development.
  • Pip packaging of our differentiable rasterizer.

Completed

• Launch of a Special Interest Group! Read our approved charter to read more about this effort. If interested, you can browse the slides and watch the recording of the meeting.
• Launched a submodules folder for the community to raise awareness about their work on the themes of differentiable 3D and graphics. The first submodule is redner!
• A complete deployment pipeline between the Github and the Google infrastructure!
• Launched a projects folder to host implementations of your work! The first project is PointNet 1.0!
• Started implementing loaders for mainstream datasets. The first datasets available as TF datasets are modelnet40 and shapenet!
• Added several losses and metrics.
• Implemented differentiable triangle mesh sampler.
• Added matting Colab demo.
• Implemented trilinear interpolation.
• Implemented linear blend skinning.

[jackd]

Meeting recording is private :(

[julienvalentin]

@jackd I'll look into this.

[sam598]

The face estimation gif looks very interesting. Is there an explanation or example of this somewhere?

[julienvalentin]

@sam598 we have a demo ready. @yifeif is currently working on the open-sourcing aspects of the OpenGL rasterizer this is based on. At that point we will be able to publish the demo which contains all the code to answer your questions.

[julienvalentin]

@jackd unfortunately I can't make the video public, but I can manually grant access to it. Can you give me your gmail?

@osanseviero we should mention that people should ask here + add an email so that we can grant them access. I tried with my personal gmail and I am not receiving the notification to grant access. Are you aware of an easier way?

[sam598]

Thanks @julienvalentin looking forward to it.

[sam598]

@julienvalentin @yifeif any update on the face alignment demo?

[bhack]

Is this project still in active development/progressing on the roadmap?

It Is quite strange that It was not mentioned in the next Neurips 2020 workshop https://montrealrobotics.ca/diffcvgp/ but I see that @taiya Is a speaker.

[osanseviero]

Hi @bhack. We're in active development. We expect to have exciting updates soon. And yes, Andrea will be in Neurips, don't miss it! The workshop will cover many topics

[bhack]

https://github.com/NVlabs/nvdiffrast.

[jackd]

Does the scope of tensorflow-graphics extend to other applications of geometric deep learning? e.g. would it consider projects of non-graphics-specific GNN layer/project/datasets?

[flynnamy]

Any updates about the face rasterizer demo?@julienvalentin @yifeif