TheBricktop
commented
5 years ago +1 Amet to that, integrated motion capture using vive trackers would be marvellous.
Open ghost opened 5 years ago
TheBricktop
commented
5 years ago +1 Amet to that, integrated motion capture using vive trackers would be marvellous.
This was originally a response to a thread, but I spent so much time typing it that I decided to re post it as a separate thread. I have typed up a list of suggestions that I hope will be able to help you see my vision for recording animations via motion capture inside Blender.
Networked 3D Modeling Idea Ok I am going to type this up really quickly due to the fact that I am on a crunch for time, but I feel it is worth my time to pitch this. It would be amazing if I could join a virtual space with my friends or coworkers and work on a 3d model with them in real time. Obviously you could start in small steps like adding network support so you can at least see someone else build their model and be able to talk to them. Eventually though, it would be amazing if I could interact with someone else's model. This would be great for a team that is working on a project, they could visualize their ideas, show suggestions for improvements real time, and even build a large project together by all working on separate parts simultaneously. I understand this is no small task, but if you are interested in this concept you should check out possibly using Networked A-Frame or something along those lines to create this!
I have typed up a list of suggestions that I hope will be able to help you see my vision for recording animations via motion capture inside Blender.
Use cases for motion capture using the HTC Vive controllers and trackers
Recording animations for human avatars So for example lets say I have a avatar in blender, I want to make it to where the body moves with the avatars bone structure. I want to then be able to record myself doing something like waving, crouching, jumping etc... I would also love to be able to record animations with a character such as swinging a sword in a game with Vive trackers. It would save me so much time if I could just record myself swinging a sword and then saving that recording and using it as an animation.
Recording animations for machine parts Now this theoretically could be recorded with the Vive trackers too, but it would be cool if I could recording moving parts of a machine. a.) So for example lets say I have a robotic arm and I wanted to quickly record an animation of the arm moving from one point to the next. b.) Another example would be to record the movement of things like a wheel, gear, or motor by hitting the motion capture button and selecting the object and then recording myself twisting my controllers. If there were a lock grid button in the options tab for motion capture, I would be able to freeze my positions of X, Y and Z. I could then enable rotation only and record me twisting my controller or Vive tracker but only record the spinning part, not the full motion of the controller.
Recording a moving vehicles So lets say we wanted to record a vehicle moving down the street. I would start by going to the motion capture settings and binding the car to the controller. Let's say the X axis is your horizontal plane in this example. I could freeze the Z and Y axis, and that way when I record the car driving in a straight line down the street, it would one recording motion in a specific direction. If there was no option to freeze the axis then the recording would look terrible and nothing like moving a car just because the chances of you recording yourself moving your controller in a perfectly straight line are slim. With those toggle settings, all sorts of movements and motion could be quick and easy to record.
Recording a door opening and closing so for this example I would select a door as my object bound to my controller, and set the pivot point where the hinges would be. Now since I only want to record the door swinging back and forth on my X axis, I would disable movement on Y and Z. Now once I press record, the door will appear to swing from the hinges just by moving my controller left or right.
Recording dance moves By assigning avatar to all of my controllers and vive trackers via the tool button, I could record dance moves.
Recording emotes By rigging an avatar to my controllers and trackers, I would be able to record different emotes such as spinning in a circle, jumping, waving my hand, swinging my fist, picking something up from the ground, answering my cellphone and much more. This would be amazing for recording NPC characters movements, combat animations, expressions, and much more.
Settings ideas for the Motion Capture button
There needs to be a motion capture button on the menu of the Vive controller that is easy to toggle on and off.
If there were a motion capture button on the menu, it would need some options such as being able to select where the pivot point would be while in the headset. Would be cool if it gave you a little point at the tip of the controller when you hit the pivot button, so when you have the tip of you controller where you want it you could press the trigger button or something to set the new pivot point.
Another useful button in the options menu of the motion capture button would be the ability to select an object and bind it to your controller. Binding the object to the controller basically would mean that you could see the 3D model follow your hand movement. So if I select an object, I can have it following my controllers movements which would allow me to record the actual movement of my controller.
The ability to lock positions and rotations on the grid such as the X, Y and Z axis so that you can record specific animations much smoother. Check the Machine parts example for how this could be a great tool
A toggle button for a countdown until you record the animation. A countdown from 3 to start recording would be really helpful, that way you get a chance to position yourself. A shortcut button to record might also do the trick, but I think the countdown may be a nice option that you could toggle.
A button to allow you to insert a model from your folders. Then once the model is loaded it gives a separate pop up window that can be grabbed and moved around when you press your grip buttons, I suggest you make it to where you controller has to be touching the menu or window in order to grab and move the window in a different place in your virtual space.
A button that allows you to either bind the 3d model to your controller or to multiple controllers. So for example lets say I wanted to select an avatar that I wanted to rig to my controllers for full body tracking. I could then select the option to rig the avatar to my body instead of binding it to just one controller to record a simple animation, Now that the avatar is rigged to my controllers and trackers, This tool would allow me to record realistic and fluent character movement very quickly and easily. If you are not already aware of A-Frame's component for motion capture, you should check it out here. It is open source so you might be able to refer to that for guidance on how to setup this button inside Blender XR. There is a GIF on that website that will also give you an idea on how this concept would work. Check out A-Frame here
There are a ton of possibilities for motion capture using the HTC Vive controllers and trackers . Let me know if you have any questions or if you need one of these concepts better explained. I would be glad to help any way I can.
WW