Define basic data structure for inter-component communication

erak commented 4 years ago

After some research, the following two options seem to feasible:

a) Raw keypoints formated as a CSV:
- simple, every programming language has CSV reader and writer
- custom rendering components need to be written for either Blender (Python script), Unreal (C++ Blueprint) or plain Rust applications (https://git.borg.sh/to-share/renderer)
b) FBX file format
- proprietary, but openly specified
- encoder / decoder available in Python (could be written from ML component)
- Rendering of animations in Blender or Unreal would basically come for free

Overall, choosing a) results in more effort for custom 3D rendering, and b) probably results in more effort encoding and decoding FBX files.

nikozoe commented 3 years ago

closing since we settled for keypoint format.

nikozoe commented 3 years ago

or maybe this is not solved? We store the 3D keypoints in numpy npz format, but is this useful for rendering? Or do we need a different format? Reopening because this might need discussion when we start working on the visualization part...

erak commented 3 years ago

Let's see what @LeaLazer thinks is best for the 3D visualizer.

erak commented 3 years ago

I think we settled on 3D keypoints in numpy array as basic data-structure for inter-component communication. Closing this issue since https://github.com/deep-dance/core/issues/35 will bring JSON export to all components.

deep-dance / core

Define basic data structure for inter-component communication #5