Can I visualize stream data?

[dx118]

I have a smpl motion which comes frame by frame instead of a whole. Is there any way to visualize it in aitviewer?

[ramenguy99]

Hi, the viewer expects all data to be created upfront and added to the scene before running. If you want to visualize streaming data there is the remote viewer API which allows you to connect to a running viewer and send data to it.

Here is a small example of what streaming SMPL sequences could look like; we are loading a set of SMPL frames from disk and sending them one by one to a viewer in a new process:

import time

import joblib
import numpy as np

from aitviewer.remote.renderables.smpl import RemoteSMPLSequence
from aitviewer.remote.viewer import RemoteViewer
from aitviewer.utils.so3 import aa2rot_numpy

if __name__ == "__main__":
    # Load camera and SMPL data from the output of the VIBE demo from https://github.com/mkocabas/VIBE
    data = joblib.load(open("resources/vibe/vibe_output.pkl", "rb"))
    poses = data[1]["pose"]
    betas = data[1]["betas"]

    # Create a new viewer in a separate processs.
    with RemoteViewer.create_new_process() as v:
        smpl_sequence = None

        for i in range(poses.shape[0]):
            # Do processing here...
            time.sleep(1)
            print(f"Sending frame {i}")

            # Send data to the viewer
            if smpl_sequence is None:
                # Create a new renderable with the first frame of data.
                smpl_sequence = RemoteSMPLSequence(
                    v,
                    model_type="smpl",
                    gender="neutral",
                    poses_body=poses[i, 3 : 24 * 3][np.newaxis],
                    poses_root=poses[i, 0:3][np.newaxis],
                    betas=betas[i][np.newaxis],
                    rotation=aa2rot_numpy(np.array([1, 0, 0]) * np.pi),
                )
            else:
                # For the following frames just add new poses and betas
                smpl_sequence.add_frames(poses_body=poses[i, 3 : 24 * 3], poses_root=poses[i, 0:3], betas=betas[i])

                # Advance the current frame on the remote viewer.
                v.next_frame()

You can run this from the examples directory or update the path to the data (which is in examples/resources/vibe).

For more details you can check the documentation or any of the remote examples.

[dx118]

thank you, it helps