Xinyu-Yi / TransPose

A real-time motion capture system that estimates poses and global translations using only 6 inertial measurement units
https://xinyu-yi.github.io/TransPose/
GNU General Public License v3.0
373 stars 72 forks source link

can you share the sensor you are using? #10

Closed StayYouth1993 closed 2 years ago

StayYouth1993 commented 2 years ago

can you share the sensor(module or chip) you are using? where could I buy the sensor? Is it available in taobao or some where?

Xinyu-Yi commented 2 years ago

The sensors we use are the legacy ones from Noitom. I'm not sure whether the Legacy sensors are still on sale. But they may have some new products which also work. Besides, you can use the imu in your phone first (If you can find 6 phones to run the system).

NoLoPhe commented 2 years ago

I found the wireless 6-axis IMU: https://www.seeedstudio.com/Seeed-XIAO-BLE-Sense-nRF52840-p-5253.html

StayYouth1993 commented 2 years ago

ok, we'll have a try, thanks for awesome work!

Xinyu-Yi commented 2 years ago

I found the wireless 6-axis IMU: https://www.seeedstudio.com/Seeed-XIAO-BLE-Sense-nRF52840-p-5253.html

Oh, we need 9-axis IMUs because we use orientation measurements (rather than angular velocity). The orientation measurements in 6-axis IMUs have too many drifts.

Xinyu-Yi commented 2 years ago

The phone imu can be easily read out and sent to the computer using a socket. E.g., using unity3D:


if (!Input.gyro.enabled) Input.gyro.enabled = true;
Input.acceleration;    // get acceleration
Input.gyro.attitude;   // get orientation
StayYouth1993 commented 2 years ago

Just now I was wondering why 6-axes are used, haha. Thanks a lot for the correction! when I invested in hardware before, I have used a number of 9-axis inertial navigation chips and tried inertial navigation. Ordinary sensors placed on the human body are basically unable to integrate to obtain the correct speed and absolute movements (the angle can be solved, but the speed drift is relatively serious), The main reason is that the instantaneous acceleration process of the human body is too fast, and the sampling rate provided by the chip ADC is very limited. The general angle is relatively accurate because of the use of magnetic field sensing.

In addition, I would like to ask, is the main input source of the 9-axis here the most important and useful information source is the angle? Are the auxiliary information for acceleration and angular acceleration important here?

ps: Is it convenient to contact you by email in chinese? mine is:xingquanjiang@qq.com

best!

Xinyu-Yi commented 2 years ago

Yes, I have ever tried to train two networks, one using only orientation measurements to regress human pose, the other using only linear acceleration measurements. A confidence is also estimated from each of the networks. Then based on Gaussian assumption, I multiply two pose distribution and supervise on the final result. When the training finishes, the confidence values of the pose estimated from accelerations are much lower than that from the orientations. It indicates that the network leverages orientation measurements better than the accelerations.

Xinyu-Yi commented 2 years ago

But if you remove the accelerations from the network input, the results will be worse due to the motion ambiguity.

herohuyongtao commented 2 years ago

The sensors we use are the legacy ones from Noitom. I'm not sure whether the Legacy sensors are still on sale. But they may have some new products which also work. Besides, you can use the imu in your phone first (If you can find 6 phones to run the system).

@Xinyu-Yi Hi Xinyu, does your system need time-sync between different IMUs (like the data under data/example folder and the live demo)? Usually it's not easy to sync IMUs precisely between the 6 phones as you suggested here.

Xinyu-Yi commented 2 years ago

The sensors we use are the legacy ones from Noitom. I'm not sure whether the Legacy sensors are still on sale. But they may have some new products which also work. Besides, you can use the imu in your phone first (If you can find 6 phones to run the system).

@Xinyu-Yi Hi Xinyu, does your system need time-sync between different IMUs (like the data under data/example folder and the live demo)? Usually it's not easy to sync IMUs precisely between the 6 phones as you suggested here.

Hi. I'm not sure, but I think we don't need a sync. The Noitom sensor actually works at 96 or 120fps (I forget). So I set a 60Hz clock in the codes and read the most recent measurement of each IMU. I did not do any sync by myself. Actually we only use the orientation and acceleration measurements from the sensors. The orientation is not changing very fast, and our system depends on orientation measurements more than the accelerations. So I think we do not need a sync, although having one will make the system better.

herohuyongtao commented 2 years ago

@Xinyu-Yi Thanks for the clarification. I will try to use them w/o time-sync first.

Oh, we need 9-axis IMUs because we use orientation measurements (rather than angular velocity). The orientation measurements in 6-axis IMUs have too many drifts.

By the way, about the 6-axis or 9-axis IMUs, did you do any experiment about how much performance decrease will be revealed by using 6-axis IMUs instead of default 9-axis ones?

NoLoPhe commented 2 years ago

Hi! @herohuyongtao If you are looking for 9-axis IMUs I can give you a suggestion Hi221 (USB/Wireless) I am learning IMU from their documentation on Zhihu

herohuyongtao commented 2 years ago

Hi! @herohuyongtao If you are looking for 9-axis IMUs I can give you a suggestion Hi221 (USB/Wireless) I am learning IMU from their documentation on Zhihu

@NoLoPhe Thanks for the info. However, I am considering to use 6-axis IMUs as the magnetometer is easily affected by the magnetic field of the environment to make the system more robust.