This plugin allows one to record all users in a session. It is able to record:
This plugin is what the NeosVR metagenbot is running (minus a trick to connect to sessions that aren't running the plugin). This release is inteded to be used as a normal Neos plugin, i.e. either locally or with other users running the plugin. Metagenbot is provided for convenience, but recording locally usually gives the best results for single-person recordings. If you want to use metagenbot for multi-person recordings, you can either invite it if it's online, or contact me @guillefix if it isn't online, or if you want maximum performance or other special requirements for your usecase.
You can visit http://metagen.ai/ to find more about the vision and motivation behind this tool beyond helping people make awesome stuff :)
metagenneos_v0.9.2b.zip
and run_neos_metagen.bat
and grpc_csharp_ext.dll
from the Releases tab. metagenneos_v0.9.2b.zip
into the Neos installation folder, and copy run_neos_metagen.bat
and grpc_csharp_ext.dll
into the Neos installation folder also.bin
to the System32 folder (C:\Windows\System32
).data
inside the NeosVR folder.run_neos_metagen.bat
(double click on it). This will start Neos with the plugin loaded, and it will open your local homeAAAA
. In it you will find MetaGenLoader
. Equip it with the Logix tip and spawn it in the world. I recommend saving the world now, as doing so (and assuming the save has worked), will mean you can skip this step in the future, as the logix node being present in the world is enough for the system to load.metagenneos_headless_v0.9.0.zip
and run_neos_metagen_headless.bat
and grpc_csharp_ext.dll
and Config.json
from the Releases tab. metagenneos_headless_v0.9.0.zip
into the Neos HeadlessClient installation folder, and copy run_neos_metagen_headless.bat
and grpc_csharp_ext.dll
into the Neos HeadlessClient installation folder also.bin
to the System32 folder (C:\Windows\System32
). (unless you have already done this).data
inside the NeosVR HeadlessClient folder.Config
folder. In the "startWorlds" array add "autoSleep": false,
and "loadWorldUrl": "neosrec:///U-guillefix/R-3979d6e1-c316-48b3-bebf-25739c2b3939",
.FrooxEngine.dll
found in Neos_Data/Managed
in the (main) NeosVR folder, into Neos_Data/Managed
inside the HeadlessClient folder. The later folder may not exist and you should create it if it doesn't.run_neos_metagen_headless.bat
(double click on it). This will start Neos with the plugin loaded, and it will open a world with the MetaGenLoader node.startWorldTemplate SpaceWorld
) and it will load metagen on it.The basic functionality is the same as explained in the release of Metagenbot Beta, you can check the videos:
Note: these videos are not completely up to date now, but should give an overall idea of what can be achieved with the plugin.
You can record yourself and other users by pressing start record. This will record the movement, voice, and hearing (hearing only for one user). Optionally it can also record the vision. Note:
metagen config/users
will be recorded. data
in the Neos installation folder, and are organized by world, and by recording session (each given a unique hash). Within each recording session the recordings are found in numbered folders.You can press Start play to play the last recording made in the current world. You can navigate older recordings by changing the "Recording index" (it's an index increasing from newest to older). Features include:
You can export the recordings as either a native Neos animated mesh (as showcased in the above videos), and/or as a Bvh file. You can do these by selecting the checkboxes. These checkboxes will determine whether these exports are generated while doing a recording, but also while doing a playback! The later is useful in case you want to generate an animation of a previously recorded motion/voice acting, but with a different avatar, or you want to generate a bvh file for a different skeleton! Note:
This is a very cool feature Joris requested. We can record arbitrary value fields and objects.
See example for field recording here https://streamable.com/5eq7nx.
TODO: write docs for object recording.
These are the instructions to puppeteer a Neos avatar from Unity. Here it is driving it from a BVH file, for example: https://twitter.com/guillefix/status/1394045763345334273. Note: this only currently works fot the rig provided. Different rigs need different translation between the rig and Neos proxy conventions. An automated system to adapt to any will be comming in the future (see discussion with lox in the MetaGen discord)
Assets/shadermotion-dev/Example/Example2_motion_stream.scene
You can change the avatar by dropping the avatar slot in the avatar slot field in the MetaGen UI. Some avatar won't work because of some scaling issue. Making the scale in the AvatarRoot component in the avatar to match the scale of the avatar slot sometimes fixes this (need to investigate furhter).
In the Unity scene, in the dancebase gameObject, in the BVHAnimationLoader script, there're two fields, SkeletonScale
ans SkeletonShiftY
which need to be tweaked a bit sometimes when using avatars which are smaller/larger (will try to investigate a way to automate this too in the future).
This is a special case of the above instructions, but for using ThreeDPoseTracker
This is using a version of ThreeDPoseTracker for Unity which I've modified to send pose data to Neos. This allows you to generate an animation from a video. You can check here for guidelines of which videos will give best results, but this is still a technology in development. Check this video for a demo:
To use it:
Credits:
The code to record meshes and export as an AnimX file uses the recording tool by LucasRo7 and jeana https://github.com/jeanahelver/NeosAnimationToolset <3