search

fact-project / aict-tools

Reproducible Artificial Intelligence for Cherenkov Telescopes
MIT License
6 stars 6 forks source link

'join'-function 'left' messes up tel_table order #155

Closed Hckjs closed 3 years ago

Hckjs commented 3 years ago

Since the other tables for gamma_pred and gamma_energy_pred are also sorted by event_id, i think it would be easier for further analyses to sort the disp_pred table also.

https://github.com/fact-project/aict-tools/blob/fa057dcf7aa2fb49554ebfa194c8e3bd3be1a120/aict_tools/io.py#L536-L542

LukasNickel commented 3 years ago

From offline discussion: They are not actually sorted explicitly, it is just the order the input is in. This join operation messes up the order apparently, because astropy doesnt generally preserve the row order. Other joins are done on keys, that happen to be ordered, so its not an issue there

maxnoe commented 3 years ago

From offline discussion: They are not actually sorted explicitly, it is just the order the input is in.

No it's not. It does not preserve order. It sorts by the keys using an unstable sorting algorithm. That basically results in a random order when all keys are identical.

Working on the fix.