What is the expected behavior when no marker is found during triangulation ?

[ANaaim]

Hi,,

As I understand it (in the last result I processed), it seems that when no marker is found the last value found is used. It seems pretty strange behavior as you can easily miss that a value was not found. Is it suppose to work like this ?

Thanks again !!

[davidpagnon]

Hi, that's it indeed, see this line: https://github.com/perfanalytics/pose2sim/blob/f287b0e5b4b09ce6bcb21a19842f2dd5e905cb06/Pose2Sim/triangulation.py#L915

    # Fill non-interpolated values with last valid one
    for n in range(nb_persons_to_detect): 
        Q_tot[n] = Q_tot[n].ffill(axis=0).bfill(axis=0)

I found out that OpenSim crashed when I left missing values in the TRC file. So the behavior is:

• if the gap is less than a threshold, it is filled with the method of your choice.
• if it is larger for it to make sense to interpolate it, I fill the Nan with the last valid value. Interpolation does not make sense then, Nan make it crash, zeros don't make sense and mess up IK results, so that's what I came up with under the advice of a colleague.

Do you have a better suggestion?

[ANaaim]

Hello,

I would say If the problem arise when using opensim, I suppose that it should be dealt with when opensim is used not before at the triangulation phase. Or at least, we can make it an option in the config file ==> we can keep the default behavior which adapted for your application but ease the work of people not using Opensim with a value sucha as : value_if_gap_too_large

[triangulation]
to_do = true
reproj_error_threshold_triangulation = 100
likelihood_threshold_triangulation = 0.05
min_cameras_for_triangulation = 2
interpolation = "cubic"
interp_if_gap_smaller_than = 10
value_if_gap_too_large = "last value" # "NaN","zero" using "last value" avoid bug in opensim when no value is found
show_interp_indices = true

I definitely consider that you should carefully investigate your data before putting in opensim. Because shit in... shit out.

Small comment : I think that give such value will also mess with the inverse kinematics in opensim I suppose. It will work (no bug), but the results will be very questioinable.

As a rule of thumb, I consider that it should be NaN, when you do not know the value as it is really easy to find in file and you stay kind of aware that you data have problem. I prefer that my code crash because there is NaN and investigate that there is some NaN due to other problem to sweep it under the rug. That kind of "bug" can save your analysis of the data.

[davidpagnon]

I guess it makes sense! The reason I did so is that some users got stuck before at the OpenSim stage because of the NaNs, and finding out what the problem was was very time-consuming 😅 When filling large gaps with the last valid value, the person just appears to be frozen for a few frames before they get correctly triangulated again.

In any case, I agree that the advantage of NaNs is that they can be spotted more easily, and then you can decide what you do with them. I just added the option in the Config file: https://github.com/perfanalytics/pose2sim/blob/86f7088b4ed89e6af2ffabb946491de81ffba094/Pose2Sim/Demo_SinglePerson/Config.toml#L147

Not that you need to install Pose2Sim from source to access that option:

git clone --depth 1 https://github.com/perfanalytics/pose2sim.git
cd pose2sim
pip install .