Thank you for the awesome updates! Recording animations with Vive Trackers support soon?

[ghost]

Hey Blender XR devs,

Just wanted to take a few minutes to let you guys know that you all are doing an awesome job and are one of the most responsive and active teams out there. Your team is making great progress, I have been blown away at how far this plugin has come in such a short time. This team is out performing major companies in VR development progress. I can't thank you enough for making this tool free, and I wish for nothing but great success for your team!

Suggestions Recording animations with Vive Trackers Is there any progress being made towards adding support for recording animations via motion capture using the Vive trackers? This would be an amazing feature that could be a game changer for animation creation. If there is any way I can help contribute, just let me know. Currently I have 2 Vive trackers. If you get a demo together for this, I will be more than happen to test it out and help you all work out the bugs.

Toggling the UI menu while inside my Vive Headset I made a suggestion to add a button that makes your mouse visible while in the headset a while back and your team added it. However, I have not had my computer for a few months and since your most recent update with the new button menu I have not checked where that mouse visibility toggle button is or if there had been any progress on making it appear clearly inside the Vive headset. Last time I tried it, the mouse was distorted in the headset and was hard to see where I was clicking, it almost looked like you were seeing double when you looked at the mouse, and this made it hard to see where I was clicking. I also suggested allowing being able to toggle full UI menu visibility inside the headset so that way we would be able to fully use blenders tools while viewing and editing in the VR headset. There are a lot of features that are not available on the controls yet and probably will not be for a while just because there are limited buttons on the VR controllers, and the fact that your team is working as fast as possible but obviously can't add support for EVERYTHING overnight. Allowing a toggle button to interact and see the full blender UI as well as the mouse would allow Blender to be 100 percent functional and usable while in a VR headset. Also, some features and options in the Blender UI will be much easier to use with a mouse and keyboard for quite some time. Instead of delaying the availability to certain tools and features, you could enable the UI to be visible inside the headset. Until you can create an easier solution, at least consider allowing the ability to toggle the visibility of the UI while inside the VR Window so that we have full access to Blender while in VR!

Keep up the amazing work,

WW

[max-krichenbauer]

Hi winsteadworks,

Thank you for the message and sorry for the late reply!

Development has slowed down a bit over the last month as we are planning next steps and trying to provide an easier way to access to VR/XR settings from the BlenderUI and vice-versa.

Regarding the Vive trackers, getting the tracking data into Blender is not very difficult, but we don't have a concept of how you would want to use it for motion capturing. I guess we could create an object in the Blender scene that follows the tracker's movements. Any idea from your side what kind of integration would be useful is very welcome.

The full UI menu ability is a bit more difficult to achieve, but I agree that it would be great to allow using Blender like this until the VR user interface is developed further. I'll take another look at it.

Best regards, Max

[ghost]

@max-krichenbauer Does your team have a Discord or Slack that I can join to help contribute a little more real time? If so, please send me a link. If you don't have discord or slack, I highly recommend using them so that people can join your community and help contribute towards Blender XR. If you don't have Discord or Slack but decide to make one, I would be able to communicate with your team much quicker and easier. Threads are slow and take days at a time to get responses.

I have typed up a list of suggestions that I hope will be able to help you see my vision for recording animations via motion capture inside Blender.

Use cases for motion capture using the HTC Vive controllers and trackers

1. Recording animations for human avatars So for example lets say I have a avatar in blender, I want to make it to where the body moves with the avatars bone structure. I want to then be able to record myself doing something like waving, crouching, jumping etc... I would also love to be able to record animations with a character such as swinging a sword in a game with Vive trackers. It would save me so much time if I could just record myself swinging a sword and then saving that recording and using it as an animation.

2. Recording animations for machine parts Now this theoretically could be recorded with the Vive trackers too, but it would be cool if I could recording moving parts of a machine. a.) So for example lets say I have a robotic arm and I wanted to quickly record an animation of the arm moving from one point to the next. b.) Another example would be to record the movement of things like a wheel, gear, or motor by hitting the motion capture button and selecting the object and then recording myself twisting my controllers. If there were a lock grid button in the options tab for motion capture, I would be able to freeze my positions of X, Y and Z. I could then enable rotation only and record me twisting my controller or Vive tracker but only record the spinning part, not the full motion of the controller.

3. Recording a moving vehicles So lets say we wanted to record a vehicle moving down the street. I would start by going to the motion capture settings and binding the car to the controller. Let's say the X axis is your horizontal plane in this example. I could freeze the Z and Y axis, and that way when I record the car driving in a straight line down the street, it would one recording motion in a specific direction. If there was no option to freeze the axis then the recording would look terrible and nothing like moving a car just because the chances of you recording yourself moving your controller in a perfectly straight line are slim. With those toggle settings, all sorts of movements and motion could be quick and easy to record.

4. Recording a door opening and closing so for this example I would select a door as my object bound to my controller, and set the pivot point where the hinges would be. Now since I only want to record the door swinging back and forth on my X axis, I would disable movement on Y and Z. Now once I press record, the door will appear to swing from the hinges just by moving my controller left or right.

5. Recording dance moves By assigning avatar to all of my controllers and vive trackers via the tool button, I could record dance moves.

6. Recording emotes By rigging an avatar to my controllers and trackers, I would be able to record different emotes such as spinning in a circle, jumping, waving my hand, swinging my fist, picking something up from the ground, answering my cellphone and much more. This would be amazing for recording NPC characters movements, combat animations, expressions, and much more.

Settings ideas for the Motion Capture button

1. There needs to be a motion capture button on the menu of the Vive controller that is easy to toggle on and off.

2. If there were a motion capture button on the menu, it would need some options such as being able to select where the pivot point would be while in the headset. Would be cool if it gave you a little point at the tip of the controller when you hit the pivot button, so when you have the tip of you controller where you want it you could press the trigger button or something to set the new pivot point.

3. Another useful button in the options menu of the motion capture button would be the ability to select an object and bind it to your controller. Binding the object to the controller basically would mean that you could see the 3D model follow your hand movement. So if I select an object, I can have it following my controllers movements which would allow me to record the actual movement of my controller.

4. The ability to lock positions and rotations on the grid such as the X, Y and Z axis so that you can record specific animations much smoother. Check the Machine parts example for how this could be a great tool

5. A toggle button for a countdown until you record the animation. A countdown from 3 to start recording would be really helpful, that way you get a chance to position yourself. A shortcut button to record might also do the trick, but I think the countdown may be a nice option that you could toggle.

6. A button to allow you to insert a model from your folders. Then once the model is loaded it gives a separate pop up window that can be grabbed and moved around when you press your grip buttons, I suggest you make it to where you controller has to be touching the menu or window in order to grab and move the window in a different place in your virtual space.

7. A button that allows you to either bind the 3d model to your controller or to multiple controllers. So for example lets say I wanted to select an avatar that I wanted to rig to my controllers for full body tracking. I could then select the option to rig the avatar to my body instead of binding it to just one controller to record a simple animation, Now that the avatar is rigged to my controllers and trackers, This tool would allow me to record realistic and fluent character movement very quickly and easily. If you are not already aware of A-Frame's component for motion capture, you should check it out here. It is open source so you might be able to refer to that for guidance on how to setup this button inside Blender XR. There is a GIF on that website that will also give you an idea on how this concept would work. Check out A-Frame here

There are a ton of possibilities for motion capture using the HTC Vive controllers and trackers . Let me know if you have any questions or if you need one of these concepts better explained. I would be glad to help any way I can.

WW

[TheBricktop]

Thats a great bag of features right there. Keep it up.