MichelEkfF and MichelObsF calculation results are inconsistent with the paper

wanXiao1984 commented 3 years ago

Dear Tyrex,

Recently，I download the source code from here, after running the code, what confuses me is some MichelEkfF and MichelObsF calculation results are inconsistent with the paper.

For instance , in "On Attitude Estimation with Smartphones" VII.A Importance of Calibration MichelEkfF( paper) 82.0° 20.1° 20.1° 6.9° 7.0° 18.2° MichelEkfF( mine) 82.0° 51.8° 51.7° 20.1° 20.1° 42.6°

MichelObsF( paper) 82.1° 13.6° 13.5° 5.9° 5.9° 15.1° MichelObsF( mine ) 82.1° 52.3° 52.2° 19.5° 19.3° 42.2°

I run the code by reading the README.md and didn't change anything.

I would be very grateful for your answer.

ThibaudM commented 3 years ago

Hello @wanXiao1984, It's a bit annoying because the benchmark is deterministic. We will try to understand what happened:

Can you give the results of the other algorithms to try to understand if the problem is only related to these two?
Did you run the tests on all the datasets?

wanXiao1984 commented 3 years ago

1. Can you give the results of the other algorithms to try to understand if the problem is only related to these two? results.compare('calibration'); Choukroun rawMag rawGyr rawAcc 95.1° YPR: 82.1° 24.3° 44.5° Choukroun ownMag rawGyr rawAcc 16.5° YPR: 15.2° 4.5° 8.6° Choukroun ownMag rawGyr ownAcc 16.5° YPR: 15.2° 4.5° 8.7° Choukroun ownMag ownGyr rawAcc 9.9° YPR: 9.8° 3.0° 6.0° Choukroun ownMag ownGyr ownAcc 10.0° YPR: 9.7° 3.0° 5.9° Choukroun ownMag osOGyr ownAcc 9.9° YPR: 9.7° 3.0° 5.9° Choukroun osRMag osRGyr osRAcc 17.3° YPR: 16.6° 4.0° 8.0° ChoukrounSn rawMag rawGyr rawAcc 82.5° YPR: 80.6° 6.6° 11.2° ChoukrounSn ownMag rawGyr rawAcc 52.6° YPR: 51.3° 6.1° 10.6° ChoukrounSn ownMag rawGyr ownAcc 52.5° YPR: 51.2° 6.1° 10.6° ChoukrounSn ownMag ownGyr rawAcc 19.3° YPR: 19.2° 1.7° 3.6° ChoukrounSn ownMag ownGyr ownAcc 19.0° YPR: 19.0° 1.7° 3.6° ChoukrounSn ownMag osOGyr ownAcc 17.9° YPR: 18.0° 1.7° 3.6° ChoukrounSn osRMag osRGyr osRAcc 41.2° YPR: 39.8° 3.9° 6.5° Ekf rawMag rawGyr rawAcc 79.8° YPR: 77.5° 8.4° 18.1° Ekf ownMag rawGyr rawAcc 19.4° YPR: 18.6° 3.6° 7.4° Ekf ownMag rawGyr ownAcc 19.4° YPR: 18.6° 3.6° 7.4° Ekf ownMag ownGyr rawAcc 7.9° YPR: 8.0° 2.2° 4.5° Ekf ownMag ownGyr ownAcc 8.0° YPR: 7.9° 2.3° 4.5° Ekf ownMag osOGyr ownAcc 7.9° YPR: 7.9° 2.3° 4.6° Ekf osRMag osRGyr osRAcc 18.2° YPR: 17.3° 3.1° 6.7° EkfExp rawMag rawGyr rawAcc 79.5° YPR: 77.2° 8.4° 18.1° EkfExp ownMag rawGyr rawAcc 19.5° YPR: 18.7° 3.6° 7.4° EkfExp ownMag rawGyr ownAcc 19.5° YPR: 18.7° 3.6° 7.4° EkfExp ownMag ownGyr rawAcc 8.0° YPR: 8.0° 2.2° 4.5° EkfExp ownMag ownGyr ownAcc 8.0° YPR: 8.0° 2.3° 4.5° EkfExp ownMag osOGyr ownAcc 7.9° YPR: 8.0° 2.3° 4.6° EkfExp osRMag osRGyr osRAcc 18.2° YPR: 17.3° 3.1° 6.7° EkfLJ rawMag rawGyr rawAcc 82.3° YPR: 79.6° 8.4° 18.2° EkfLJ ownMag rawGyr rawAcc 19.4° YPR: 18.7° 3.5° 7.4° EkfLJ ownMag rawGyr ownAcc 19.4° YPR: 18.7° 3.5° 7.4° EkfLJ ownMag ownGyr rawAcc 8.0° YPR: 8.0° 2.2° 4.6° EkfLJ ownMag ownGyr ownAcc 8.0° YPR: 8.0° 2.3° 4.5° EkfLJ ownMag osOGyr ownAcc 8.0° YPR: 8.0° 2.3° 4.6° EkfLJ osRMag osRGyr osRAcc 18.0° YPR: 17.1° 3.1° 6.7° EkfRev rawMag rawGyr rawAcc 81.5° YPR: 79.3° 8.4° 17.2° EkfRev ownMag rawGyr rawAcc 17.2° YPR: 16.5° 3.4° 7.1° EkfRev ownMag rawGyr ownAcc 17.3° YPR: 16.6° 3.4° 7.1° EkfRev ownMag ownGyr rawAcc 8.3° YPR: 8.5° 2.2° 4.5° EkfRev ownMag ownGyr ownAcc 8.4° YPR: 8.5° 2.2° 4.4° EkfRev ownMag osOGyr ownAcc 8.4° YPR: 8.5° 2.2° 4.5° EkfRev osRMag osRGyr osRAcc 17.7° YPR: 17.0° 3.0° 6.5° EkfRevSn rawMag rawGyr rawAcc 91.2° YPR: 85.6° 6.6° 18.4° EkfRevSn ownMag rawGyr rawAcc 74.9° YPR: 70.4° 6.3° 16.9° EkfRevSn ownMag rawGyr ownAcc 74.6° YPR: 69.9° 6.2° 16.9° EkfRevSn ownMag ownGyr rawAcc 57.8° YPR: 57.0° 2.5° 6.4° EkfRevSn ownMag ownGyr ownAcc 57.9° YPR: 56.8° 2.3° 6.8° EkfRevSn ownMag osOGyr ownAcc 56.4° YPR: 55.5° 2.3° 6.8° EkfRevSn osRMag osRGyr osRAcc 67.0° YPR: 65.7° 4.3° 8.5° EkfSn rawMag rawGyr rawAcc 88.5° YPR: 82.8° 8.7° 20.1° EkfSn ownMag rawGyr rawAcc 70.0° YPR: 65.0° 8.6° 19.8° EkfSn ownMag rawGyr ownAcc 70.9° YPR: 66.1° 8.4° 19.7° EkfSn ownMag ownGyr rawAcc 36.4° YPR: 33.4° 5.2° 14.8° EkfSn ownMag ownGyr ownAcc 35.8° YPR: 33.0° 5.0° 14.0° EkfSn ownMag osOGyr ownAcc 36.0° YPR: 32.7° 5.1° 15.2° EkfSn osRMag osRGyr osRAcc 58.3° YPR: 53.9° 6.9° 18.0° Fourati rawMag rawGyr rawAcc 83.7° YPR: 81.7° 8.1° 13.9° Fourati ownMag rawGyr rawAcc 15.6° YPR: 14.4° 4.6° 8.4° Fourati ownMag rawGyr ownAcc 15.5° YPR: 14.3° 4.6° 8.4° Fourati ownMag ownGyr rawAcc 10.3° YPR: 10.5° 1.8° 3.8° Fourati ownMag ownGyr ownAcc 10.4° YPR: 10.6° 1.8° 3.7° Fourati ownMag osOGyr ownAcc 10.4° YPR: 10.7° 1.8° 3.7° Fourati osRMag osRGyr osRAcc 16.3° YPR: 15.3° 3.6° 6.1° FouratiExtacc rawMag rawGyr rawAcc 127.6° YPR: 89.4° 37.6° 97.6° FouratiExtacc ownMag rawGyr rawAcc 50.0° YPR: 46.2° 14.6° 25.5° FouratiExtacc ownMag rawGyr ownAcc 49.9° YPR: 46.2° 14.6° 25.5° FouratiExtacc ownMag ownGyr rawAcc 22.5° YPR: 22.0° 7.8° 13.9° FouratiExtacc ownMag ownGyr ownAcc 22.4° YPR: 21.8° 7.8° 13.8° FouratiExtacc ownMag osOGyr ownAcc 21.6° YPR: 21.1° 7.5° 13.5° FouratiExtacc osRMag osRGyr osRAcc 43.3° YPR: 39.3° 13.0° 23.4° Madgwick rawMag rawGyr rawAcc 77.5° YPR: 74.7° 10.0° 18.1° Madgwick ownMag rawGyr rawAcc 18.2° YPR: 17.2° 3.9° 7.8° Madgwick ownMag rawGyr ownAcc 18.2° YPR: 17.3° 3.9° 7.9° Madgwick ownMag ownGyr rawAcc 8.1° YPR: 8.1° 2.2° 4.6° Madgwick ownMag ownGyr ownAcc 8.1° YPR: 8.1° 2.2° 4.6° Madgwick ownMag osOGyr ownAcc 8.1° YPR: 8.1° 2.2° 4.6° Madgwick osRMag osRGyr osRAcc 17.7° YPR: 16.9° 3.4° 6.9° MadgwickB rawMag rawGyr rawAcc 91.6° YPR: 84.0° 15.8° 27.7° MadgwickB ownMag rawGyr rawAcc 13.4° YPR: 13.2° 3.9° 7.7° MadgwickB ownMag rawGyr ownAcc 13.3° YPR: 13.1° 3.9° 7.6° MadgwickB ownMag ownGyr rawAcc 13.1° YPR: 13.0° 3.8° 7.6° MadgwickB ownMag ownGyr ownAcc 13.1° YPR: 12.9° 3.7° 7.4° MadgwickB ownMag osOGyr ownAcc 13.1° YPR: 12.9° 3.7° 7.4° MadgwickB osRMag osRGyr osRAcc 18.7° YPR: 19.0° 3.9° 7.5° Mahony rawMag rawGyr rawAcc 99.6° YPR: 87.9° 21.4° 50.0° Mahony ownMag rawGyr rawAcc 20.2° YPR: 19.6° 5.1° 10.0° Mahony ownMag rawGyr ownAcc 20.2° YPR: 19.6° 5.2° 10.0° Mahony ownMag ownGyr rawAcc 14.2° YPR: 13.8° 4.3° 7.8° Mahony ownMag ownGyr ownAcc 14.2° YPR: 13.7° 4.3° 7.8° Mahony ownMag osOGyr ownAcc 14.2° YPR: 13.7° 4.3° 7.8° Mahony osRMag osRGyr osRAcc 19.1° YPR: 18.9° 4.7° 9.4° MahonyB rawMag rawGyr rawAcc 100.1° YPR: 86.4° 22.1° 53.9° MahonyB ownMag rawGyr rawAcc 24.1° YPR: 23.7° 5.6° 10.3° MahonyB ownMag rawGyr ownAcc 24.1° YPR: 23.7° 5.6° 10.3° MahonyB ownMag ownGyr rawAcc 24.0° YPR: 23.6° 5.6° 10.3° MahonyB ownMag ownGyr ownAcc 24.1° YPR: 23.6° 5.6° 10.2° MahonyB ownMag osOGyr ownAcc 24.0° YPR: 23.6° 5.6° 10.2° MahonyB osRMag osRGyr osRAcc 23.4° YPR: 23.7° 4.9° 9.1° Martin rawMag rawGyr rawAcc 90.3° YPR: 88.9° 5.2° 10.6° Martin ownMag rawGyr rawAcc 39.8° YPR: 39.7° 4.7° 9.8° Martin ownMag rawGyr ownAcc 39.8° YPR: 39.7° 4.8° 9.7° Martin ownMag ownGyr rawAcc 40.1° YPR: 40.0° 4.7° 9.8° Martin ownMag ownGyr ownAcc 39.6° YPR: 39.6° 4.7° 9.6° Martin ownMag osOGyr ownAcc 39.8° YPR: 39.8° 4.7° 9.6° Martin osRMag osRGyr osRAcc 37.8° YPR: 37.8° 4.6° 9.6° MichelEkf rawMag rawGyr rawAcc 79.8° YPR: 77.5° 8.4° 18.1° MichelEkf ownMag rawGyr rawAcc 19.4° YPR: 18.6° 3.6° 7.4° MichelEkf ownMag rawGyr ownAcc 19.4° YPR: 18.6° 3.6° 7.4° MichelEkf ownMag ownGyr rawAcc 7.9° YPR: 8.0° 2.2° 4.5° MichelEkf ownMag ownGyr ownAcc 8.0° YPR: 7.9° 2.3° 4.5° MichelEkf ownMag osOGyr ownAcc 7.9° YPR: 7.9° 2.3° 4.6° MichelEkf osRMag osRGyr osRAcc 18.2° YPR: 17.3° 3.1° 6.7° MichelEkfExtmag rawMag rawGyr rawAcc 82.0° YPR: 81.3° 2.9° 5.8° MichelEkfExtmag ownMag rawGyr rawAcc 51.9° YPR: 51.8° 2.9° 5.7° MichelEkfExtmag ownMag rawGyr ownAcc 51.8° YPR: 51.7° 2.9° 5.7° MichelEkfExtmag ownMag ownGyr rawAcc 20.3° YPR: 20.1° 1.9° 3.8° MichelEkfExtmag ownMag ownGyr ownAcc 20.2° YPR: 20.1° 1.9° 3.8° MichelEkfExtmag ownMag osOGyr ownAcc 19.1° YPR: 19.1° 1.9° 3.8° MichelEkfExtmag osRMag osRGyr osRAcc 42.7° YPR: 42.2° 2.4° 4.8° MichelEkfExtmagWt rawMag rawGyr rawAcc 82.0° YPR: 81.3° 2.9° 5.8° MichelEkfExtmagWt ownMag rawGyr rawAcc 51.8° YPR: 51.7° 2.9° 5.7° MichelEkfExtmagWt ownMag rawGyr ownAcc 51.7° YPR: 51.6° 2.9° 5.7° MichelEkfExtmagWt ownMag ownGyr rawAcc 20.1° YPR: 20.0° 1.9° 3.8° MichelEkfExtmagWt ownMag ownGyr ownAcc 20.1° YPR: 19.9° 1.9° 3.8° MichelEkfExtmagWt ownMag osOGyr ownAcc 19.0° YPR: 19.0° 1.9° 3.8° MichelEkfExtmagWt osRMag osRGyr osRAcc 42.6° YPR: 42.1° 2.4° 4.8° MichelEkfExtmagWtRep rawMag rawGyr rawAcc 82.0° YPR: 81.3° 2.9° 5.8° MichelEkfExtmagWtRep ownMag rawGyr rawAcc 51.8° YPR: 51.7° 2.9° 5.7° MichelEkfExtmagWtRep ownMag rawGyr ownAcc 51.7° YPR: 51.6° 2.9° 5.7° MichelEkfExtmagWtRep ownMag ownGyr rawAcc 20.1° YPR: 20.0° 1.9° 3.8° MichelEkfExtmagWtRep ownMag ownGyr ownAcc 20.1° YPR: 19.9° 1.9° 3.8° MichelEkfExtmagWtRep ownMag osOGyr ownAcc 19.0° YPR: 19.0° 1.9° 3.8° MichelEkfExtmagWtRep osRMag osRGyr osRAcc 42.6° YPR: 42.1° 2.4° 4.8° MichelEkfSn rawMag rawGyr rawAcc 88.5° YPR: 82.8° 8.7° 20.1° MichelEkfSn ownMag rawGyr rawAcc 70.0° YPR: 65.0° 8.6° 19.8° MichelEkfSn ownMag rawGyr ownAcc 70.9° YPR: 66.1° 8.4° 19.7° MichelEkfSn ownMag ownGyr rawAcc 36.4° YPR: 33.4° 5.2° 14.8° MichelEkfSn ownMag ownGyr ownAcc 35.8° YPR: 33.0° 5.0° 14.0° MichelEkfSn ownMag osOGyr ownAcc 36.0° YPR: 32.7° 5.1° 15.2° MichelEkfSn osRMag osRGyr osRAcc 58.3° YPR: 53.9° 6.9° 18.0° MichelObs rawMag rawGyr rawAcc 83.7° YPR: 81.7° 8.1° 13.9° MichelObs ownMag rawGyr rawAcc 15.6° YPR: 14.4° 4.6° 8.4° MichelObs ownMag rawGyr ownAcc 15.5° YPR: 14.3° 4.6° 8.4° MichelObs ownMag ownGyr rawAcc 10.3° YPR: 10.5° 1.8° 3.8° MichelObs ownMag ownGyr ownAcc 10.4° YPR: 10.6° 1.8° 3.7° MichelObs ownMag osOGyr ownAcc 10.4° YPR: 10.7° 1.8° 3.7° MichelObs osRMag osRGyr osRAcc 16.3° YPR: 15.3° 3.6° 6.1° MichelObsExtmag rawMag rawGyr rawAcc 82.1° YPR: 80.9° 4.7° 8.3° MichelObsExtmag ownMag rawGyr rawAcc 52.3° YPR: 51.9° 4.7° 8.3° MichelObsExtmag ownMag rawGyr ownAcc 52.2° YPR: 51.9° 4.6° 8.3° MichelObsExtmag ownMag ownGyr rawAcc 19.5° YPR: 19.5° 1.5° 3.1° MichelObsExtmag ownMag ownGyr ownAcc 19.4° YPR: 19.3° 1.5° 3.1° MichelObsExtmag ownMag osOGyr ownAcc 18.2° YPR: 18.2° 1.5° 3.1° MichelObsExtmag osRMag osRGyr osRAcc 42.3° YPR: 41.3° 3.5° 5.6° MichelObsExtmagWt rawMag rawGyr rawAcc 82.1° YPR: 80.9° 4.7° 8.3° MichelObsExtmagWt ownMag rawGyr rawAcc 52.3° YPR: 51.9° 4.7° 8.3° MichelObsExtmagWt ownMag rawGyr ownAcc 52.2° YPR: 51.8° 4.6° 8.3° MichelObsExtmagWt ownMag ownGyr rawAcc 19.5° YPR: 19.4° 1.5° 3.2° MichelObsExtmagWt ownMag ownGyr ownAcc 19.3° YPR: 19.3° 1.5° 3.1° MichelObsExtmagWt ownMag osOGyr ownAcc 18.2° YPR: 18.2° 1.5° 3.1° MichelObsExtmagWt osRMag osRGyr osRAcc 42.2° YPR: 41.2° 3.5° 5.6° MichelObsExtmagWtRep rawMag rawGyr rawAcc 82.1° YPR: 80.9° 4.7° 8.3° MichelObsExtmagWtRep ownMag rawGyr rawAcc 52.3° YPR: 51.9° 4.7° 8.3° MichelObsExtmagWtRep ownMag rawGyr ownAcc 52.2° YPR: 51.8° 4.6° 8.3° MichelObsExtmagWtRep ownMag ownGyr rawAcc 19.5° YPR: 19.4° 1.5° 3.2° MichelObsExtmagWtRep ownMag ownGyr ownAcc 19.3° YPR: 19.3° 1.5° 3.1° MichelObsExtmagWtRep ownMag osOGyr ownAcc 18.2° YPR: 18.2° 1.5° 3.1° MichelObsExtmagWtRep osRMag osRGyr osRAcc 42.2° YPR: 41.2° 3.5° 5.6° Renaudin rawMag rawGyr rawAcc 82.2° YPR: 79.5° 8.4° 18.1° Renaudin ownMag rawGyr rawAcc 19.5° YPR: 18.7° 3.5° 7.4° Renaudin ownMag rawGyr ownAcc 19.5° YPR: 18.8° 3.5° 7.4° Renaudin ownMag ownGyr rawAcc 8.0° YPR: 8.0° 2.2° 4.6° Renaudin ownMag ownGyr ownAcc 8.1° YPR: 8.0° 2.3° 4.5° Renaudin ownMag osOGyr ownAcc 8.0° YPR: 8.0° 2.3° 4.6° Renaudin osRMag osRGyr osRAcc 18.1° YPR: 17.1° 3.1° 6.7° RenaudinB rawMag rawGyr rawAcc 95.0° YPR: 89.3° 14.4° 27.5° RenaudinB ownMag rawGyr rawAcc 9.5° YPR: 9.9° 2.5° 5.3° RenaudinB ownMag rawGyr ownAcc 9.5° YPR: 9.9° 2.5° 5.3° RenaudinB ownMag ownGyr rawAcc 9.6° YPR: 10.0° 2.5° 5.3° RenaudinB ownMag ownGyr ownAcc 9.6° YPR: 9.9° 2.5° 5.3° RenaudinB ownMag osOGyr ownAcc 9.5° YPR: 9.9° 2.5° 5.3° RenaudinB osRMag osRGyr osRAcc 17.7° YPR: 17.6° 3.0° 6.1° RenaudinBG rawMag rawGyr rawAcc 95.0° YPR: 89.3° 14.3° 27.5° RenaudinBG ownMag rawGyr rawAcc 9.5° YPR: 9.9° 2.5° 5.3° RenaudinBG ownMag rawGyr ownAcc 9.5° YPR: 9.9° 2.5° 5.3° RenaudinBG ownMag ownGyr rawAcc 9.6° YPR: 10.0° 2.5° 5.3° RenaudinBG ownMag ownGyr ownAcc 9.6° YPR: 9.9° 2.5° 5.3° RenaudinBG ownMag osOGyr ownAcc 9.6° YPR: 9.9° 2.5° 5.3° RenaudinBG osRMag osRGyr osRAcc 17.7° YPR: 17.7° 3.0° 6.1° RenaudinBGExtaccExtmag rawMag rawGyr rawAcc 97.6° YPR: 90.5° 15.7° 30.1° RenaudinBGExtaccExtmag ownMag rawGyr rawAcc 10.2° YPR: 10.6° 2.6° 5.6° RenaudinBGExtaccExtmag ownMag rawGyr ownAcc 10.2° YPR: 10.6° 2.6° 5.6° RenaudinBGExtaccExtmag ownMag ownGyr rawAcc 10.4° YPR: 10.8° 2.7° 5.7° RenaudinBGExtaccExtmag ownMag ownGyr ownAcc 10.4° YPR: 10.7° 2.7° 5.7° RenaudinBGExtaccExtmag ownMag osOGyr ownAcc 10.3° YPR: 10.7° 2.7° 5.7° RenaudinBGExtaccExtmag osRMag osRGyr osRAcc 18.6° YPR: 18.3° 3.4° 6.9° RenaudinExtacc rawMag rawGyr rawAcc 86.2° YPR: 80.8° 11.1° 24.7° RenaudinExtacc ownMag rawGyr rawAcc 20.2° YPR: 19.1° 4.1° 8.4° RenaudinExtacc ownMag rawGyr ownAcc 20.2° YPR: 19.1° 4.1° 8.4° RenaudinExtacc ownMag ownGyr rawAcc 8.9° YPR: 8.8° 2.6° 5.4° RenaudinExtacc ownMag ownGyr ownAcc 8.9° YPR: 8.8° 2.6° 5.4° RenaudinExtacc ownMag osOGyr ownAcc 8.9° YPR: 8.8° 2.6° 5.4° RenaudinExtacc osRMag osRGyr osRAcc 19.5° YPR: 18.1° 3.9° 8.0° RenaudinExtaccExtmag rawMag rawGyr rawAcc 84.6° YPR: 79.5° 10.4° 23.3° RenaudinExtaccExtmag ownMag rawGyr rawAcc 20.2° YPR: 19.1° 4.0° 8.3° RenaudinExtaccExtmag ownMag rawGyr ownAcc 20.2° YPR: 19.2° 4.0° 8.3° RenaudinExtaccExtmag ownMag ownGyr rawAcc 8.7° YPR: 8.7° 2.5° 5.2° RenaudinExtaccExtmag ownMag ownGyr ownAcc 8.8° YPR: 8.7° 2.5° 5.2° RenaudinExtaccExtmag ownMag osOGyr ownAcc 8.7° YPR: 8.7° 2.5° 5.2° RenaudinExtaccExtmag osRMag osRGyr osRAcc 19.4° YPR: 18.0° 3.8° 8.0° RenaudinExtmag rawMag rawGyr rawAcc 81.3° YPR: 78.8° 7.8° 16.8° RenaudinExtmag ownMag rawGyr rawAcc 19.5° YPR: 18.8° 3.4° 7.3° RenaudinExtmag ownMag rawGyr ownAcc 19.5° YPR: 18.8° 3.4° 7.3° RenaudinExtmag ownMag ownGyr rawAcc 7.8° YPR: 7.9° 2.1° 4.4° RenaudinExtmag ownMag ownGyr ownAcc 7.9° YPR: 7.8° 2.1° 4.4° RenaudinExtmag ownMag osOGyr ownAcc 7.8° YPR: 7.8° 2.1° 4.4° RenaudinExtmag osRMag osRGyr osRAcc 18.0° YPR: 17.1° 3.0° 6.6° RenaudinSn rawMag rawGyr rawAcc 81.7° YPR: 80.3° 5.0° 9.0° RenaudinSn ownMag rawGyr rawAcc 53.1° YPR: 52.4° 5.0° 9.0° RenaudinSn ownMag rawGyr ownAcc 52.9° YPR: 52.2° 5.0° 9.0° RenaudinSn ownMag ownGyr rawAcc 20.3° YPR: 20.2° 1.5° 3.3° RenaudinSn ownMag ownGyr ownAcc 20.1° YPR: 20.1° 1.5° 3.2° RenaudinSn ownMag osOGyr ownAcc 19.1° YPR: 19.0° 1.5° 3.2° RenaudinSn osRMag osRGyr osRAcc 42.4° YPR: 41.4° 3.2° 5.4° Sabatini rawMag rawGyr rawAcc 79.8° YPR: 77.5° 8.4° 18.1° Sabatini ownMag rawGyr rawAcc 19.4° YPR: 18.6° 3.6° 7.4° Sabatini ownMag rawGyr ownAcc 19.4° YPR: 18.6° 3.6° 7.4° Sabatini ownMag ownGyr rawAcc 7.9° YPR: 8.0° 2.2° 4.5° Sabatini ownMag ownGyr ownAcc 8.0° YPR: 7.9° 2.3° 4.5° Sabatini ownMag osOGyr ownAcc 7.9° YPR: 7.9° 2.3° 4.6° Sabatini osRMag osRGyr osRAcc 18.2° YPR: 17.3° 3.1° 6.7° SabatiniExtacc rawMag rawGyr rawAcc 129.2° YPR: 90.3° 38.2° 98.0° SabatiniExtacc ownMag rawGyr rawAcc 48.4° YPR: 44.5° 14.2° 25.2° SabatiniExtacc ownMag rawGyr ownAcc 48.3° YPR: 44.4° 14.2° 25.1° SabatiniExtacc ownMag ownGyr rawAcc 21.5° YPR: 20.8° 7.8° 14.0° SabatiniExtacc ownMag ownGyr ownAcc 21.4° YPR: 20.6° 7.8° 13.9° SabatiniExtacc ownMag osOGyr ownAcc 20.7° YPR: 19.9° 7.6° 13.7° SabatiniExtacc osRMag osRGyr osRAcc 41.7° YPR: 37.5° 12.9° 23.3° SabatiniExtaccExtmag rawMag rawGyr rawAcc 129.2° YPR: 90.3° 38.2° 98.0° SabatiniExtaccExtmag ownMag rawGyr rawAcc 48.4° YPR: 44.5° 14.2° 25.2° SabatiniExtaccExtmag ownMag rawGyr ownAcc 48.3° YPR: 44.4° 14.2° 25.1° SabatiniExtaccExtmag ownMag ownGyr rawAcc 21.5° YPR: 20.8° 7.8° 14.0° SabatiniExtaccExtmag ownMag ownGyr ownAcc 21.4° YPR: 20.6° 7.8° 13.9° SabatiniExtaccExtmag ownMag osOGyr ownAcc 20.7° YPR: 19.9° 7.6° 13.7° SabatiniExtaccExtmag osRMag osRGyr osRAcc 41.7° YPR: 37.5° 12.9° 23.3° SabatiniExtmag rawMag rawGyr rawAcc 81.0° YPR: 80.3° 3.1° 6.1° SabatiniExtmag ownMag rawGyr rawAcc 22.1° YPR: 21.5° 3.1° 6.3° SabatiniExtmag ownMag rawGyr ownAcc 22.1° YPR: 21.5° 3.1° 6.3° SabatiniExtmag ownMag ownGyr rawAcc 8.4° YPR: 8.4° 1.9° 3.9° SabatiniExtmag ownMag ownGyr ownAcc 8.4° YPR: 8.3° 2.0° 3.8° SabatiniExtmag ownMag osOGyr ownAcc 8.3° YPR: 8.3° 2.0° 3.9° SabatiniExtmag osRMag osRGyr osRAcc 19.9° YPR: 19.1° 2.6° 5.5°

2. Did you run the tests on all the datasets? yes , all the datasets , have ran twice

Attach the complete results. On Attitude Estimation with Smartphones_results.md

Thanks.

ThibaudM commented 3 years ago

Thank you for your feedback, From what I see from your logs, it seems that MichelEkfExtmag, MichelEkfExtmagWt, MichelObsExtmag, MichelObsExtmagWt are also impacted. But that is not the case for MichelEkf and MichelObs.

In parallel, I have started the benchmark with obj.toProcess.algorithms = {'Mahony', 'MahonyB', 'MichelObsExtmagWtRep', 'MichelEkfExtmagWtRep'}; to see if I obtain the same results than yours. If it is the case, I will try to run again the benchmark from an old commit (one around the date of the paper submission).

wanXiao1984 commented 3 years ago

OK, I'll wait for your news.

Thanks

ThibaudM commented 3 years ago

My results:

Mahony               rawMag rawGyr rawAcc  99.6  YPR:  87.9 21.4 50.0
Mahony               ownMag rawGyr rawAcc  20.2  YPR:  19.6  5.1 10.0
Mahony               ownMag rawGyr ownAcc  20.2  YPR:  19.6  5.2 10.0
Mahony               ownMag ownGyr rawAcc  14.2  YPR:  13.8  4.3  7.8
Mahony               ownMag ownGyr ownAcc  14.2  YPR:  13.7  4.3  7.8
Mahony               ownMag osOGyr ownAcc  14.2  YPR:  13.7  4.3  7.8
Mahony               osRMag osRGyr osRAcc  19.1  YPR:  18.9  4.7  9.4
MahonyB              rawMag rawGyr rawAcc 100.1  YPR:  86.4 22.1 53.9
MahonyB              ownMag rawGyr rawAcc  24.1  YPR:  23.7  5.6 10.3
MahonyB              ownMag rawGyr ownAcc  24.1  YPR:  23.7  5.6 10.3
MahonyB              ownMag ownGyr rawAcc  24.0  YPR:  23.6  5.6 10.3
MahonyB              ownMag ownGyr ownAcc  24.1  YPR:  23.6  5.6 10.2
MahonyB              ownMag osOGyr ownAcc  24.0  YPR:  23.6  5.6 10.2
MahonyB              osRMag osRGyr osRAcc  23.4  YPR:  23.7  4.9  9.1
MichelEkfExtmagWtRep rawMag rawGyr rawAcc  82.0  YPR:  81.3  2.9  5.8
MichelEkfExtmagWtRep ownMag rawGyr rawAcc  51.8  YPR:  51.7  2.9  5.7
MichelEkfExtmagWtRep ownMag rawGyr ownAcc  51.7  YPR:  51.6  2.9  5.7
MichelEkfExtmagWtRep ownMag ownGyr rawAcc  20.1  YPR:  20.0  1.9  3.8
MichelEkfExtmagWtRep ownMag ownGyr ownAcc  20.1  YPR:  19.9  1.9  3.8
MichelEkfExtmagWtRep ownMag osOGyr ownAcc  19.0  YPR:  19.0  1.9  3.8
MichelEkfExtmagWtRep osRMag osRGyr osRAcc  42.6  YPR:  42.1  2.4  4.8
MichelObsExtmagWtRep rawMag rawGyr rawAcc  82.1  YPR:  80.9  4.7  8.3
MichelObsExtmagWtRep ownMag rawGyr rawAcc  52.3  YPR:  51.9  4.7  8.3
MichelObsExtmagWtRep ownMag rawGyr ownAcc  52.2  YPR:  51.8  4.6  8.3
MichelObsExtmagWtRep ownMag ownGyr rawAcc  19.5  YPR:  19.4  1.5  3.2
MichelObsExtmagWtRep ownMag ownGyr ownAcc  19.3  YPR:  19.3  1.5  3.1
MichelObsExtmagWtRep ownMag osOGyr ownAcc  18.2  YPR:  18.2  1.5  3.1
MichelObsExtmagWtRep osRMag osRGyr osRAcc  42.2  YPR:  41.2  3.5  5.6

are the same as yours. That is weird.

I will try again from an old commit.

ThibaudM commented 3 years ago

Ok, I got it. The problem comes from this recent commit (7181cb97fe17bfebe7070b1a6251526afb1893a5). Explanation: Here, in QMichelEkfExtMag.m, the value of obj.MagRefNorm is 1 instead of ~47. The value is 1 because I use quatrotate in the createContextFromLocationAndDate.m.

When I tried to fix #3, I normalized the output vector instead of the input quaternion... no comment.

I will revert the commit related to #3 and apply the normalization to the quaternion, run some tests again, then push it when it is ok.

wanXiao1984 commented 3 years ago

Ok, that's great！ In inertial navigation system, we usually normalize quaternion after updating it. When using quaternion to rotate a vector, we may not need to normalize quaternion again if we have done it yet. Hope this information is useful for you.

Thanks

ThibaudM commented 3 years ago

Yes, I agree with your approach, I had the same before someone asked me to add it in #3. I did not take the time to benchmark the impact, but it could be null if the quaternion is well normalized before the call of quatrotate.

wanXiao1984 commented 3 years ago

Thank you very much for your reply, now we can close this issue. Your summary and innovative work on attitude estimation with smartphones has benefited me a lot, especially the open source code and data sets, which are admirable.

ThibaudM commented 3 years ago

Mahony          rawMag rawGyr rawAcc  99.6  YPR:  87.9 21.4 50.0
Mahony          ownMag rawGyr rawAcc  20.2  YPR:  19.6  5.1 10.0
Mahony          ownMag rawGyr ownAcc  20.2  YPR:  19.6  5.2 10.0
Mahony          ownMag ownGyr rawAcc  14.2  YPR:  13.8  4.3  7.8
Mahony          ownMag ownGyr ownAcc  14.2  YPR:  13.7  4.3  7.8
Mahony          ownMag osOGyr ownAcc  14.2  YPR:  13.7  4.3  7.8
Mahony          osRMag osRGyr osRAcc  19.1  YPR:  18.9  4.7  9.4
MichelObsExtmag rawMag rawGyr rawAcc  82.1  YPR:  80.9  4.7  8.3
MichelObsExtmag ownMag rawGyr rawAcc  13.4  YPR:  12.2  4.6  8.2
MichelObsExtmag ownMag rawGyr ownAcc  13.3  YPR:  12.2  4.5  8.2
MichelObsExtmag ownMag ownGyr rawAcc   7.7  YPR:   8.0  1.6  3.4
MichelObsExtmag ownMag ownGyr ownAcc   7.7  YPR:   8.0  1.6  3.3
MichelObsExtmag ownMag osOGyr ownAcc   7.7  YPR:   8.0  1.6  3.3
MichelObsExtmag osRMag osRGyr osRAcc  15.5  YPR:  14.3  3.5  5.9

New results are good. Thanks for your feedback :)

tyrex-team / benchmarks-attitude-smartphones

MichelEkfF and MichelObsF calculation results are inconsistent with the paper #4