Unable to reproduce results

[gguzzy]

Good morning everyone, I wanted to ask if you could share additional information on how IDs have been filtered out in the subset.ipynb, moreover could you provide the used dataset files not currently uploaded within the datasets?

More info on the preprocessing part.

Thanks

[johnnynunez]

Good morning everyone, I wanted to ask if you could share additional information on how IDs have been filtered out in the subset.ipynb, moreover could you provide the used dataset files not currently uploaded within the datasets?

More info on the preprocessing part.

Thanks

Hello, the data were obtained from the original dataset. Finally, we annotate based on a behavioral protocol. https://www.kaggle.com/datasets/ivannikolov/longterm-thermal-drift-dataset

So, first, you need to extract dataset with LTD script. Then, create annotations.csv from the notebook

[gguzzy]

Would reproduce the LTD script the annotations.csv but called as 'metadata_images.csv'? That was maybe why I was not getting it. So, basically from kaggle and using the script, I could reproduce the other steps? Thanks

[johnnynunez]

Would reproduce the LTD script the annotations.csv but called as 'metadata_images.csv'? That was maybe why I was not getting it. So, basically from kaggle and using the script, I could reproduce the other steps? Thanks

exactly. The code from extract_LTD was provided by the owners of the dataset

[gguzzy]

Ok thanks, but one more question please, how would I be able to produce the annotations folder you mentioned in the notebook 'analysis.ipynb' and 'analysis_dataset.ipynb'? Since, if I am not wrong, it is not available from the dataset downloaded from Kaggle. I mean, from the folder 'Data_Annotated_Subset_Object_Detectors' which subfolder should I take? Or should I create a new directory with only the .txt? Thanks a lot, and congrats for the great work!

[johnnynunez]

Ok thanks, but one more question please, how would I be able to produce the annotations folder you mentioned in the notebook 'analysis.ipynb' and 'analysis_dataset.ipynb'? Since, if I am not wrong, it is not available from the dataset downloaded from Kaggle. I mean, from the folder 'Data_Annotated_Subset_Object_Detectors' which subfolder should I take? Or should I create a new directory with only the .txt? Thanks a lot, and congrats for the great work!

I will try to reproduce it again and send you the results and the process. Sorry for that

[gguzzy]

No issues, and thanks a lot for supporting! If you want to collaborate with that improving the model or dataset with additional data we can keep in contact. Thanks a lot, waiting for those

[johnnynunez]

No issues, and thanks a lot for supporting! If you want to collaborate with that improving the model or dataset with additional data we can keep in contact. Thanks a lot, waiting for those

You're right, it was my confusion. The annotations were shared with me by the original authors of the dataset. I ask them if I can share them.

[gguzzy]

Okay thanks a lot!

[johnnynunez]

Okay thanks a lot!

I have been told it will be published soon. Sorry for the waiting times. I no longer work at Milestones

[gguzzy]

Hi thanks a lot, still nothing. Is it possible to get the 'predicitons_NO_IMP.csv' used for evaluation, or get to know where and how it was obtained? Thanks again.

[gguzzy]

Moreover, I would like to ask, since it is not well defined in the paper, how this loitering annotations are perfomed? I mean, which are the criteria to establish and auto-annotate if that trajectory is an anomalous one? For instance, let's suppose we have a new video, as I read we need to keep standard length to 120, but are the trajectory methods applied themselves to detect loitering? among which ones are used? Convex hull or other? Thanks

[gguzzy]

Can you please explain how the annotations were made? Using a weighted vote of the geometric methods? If yes, which threshold did you use and why? How did you obtain those? For geometric methods how did you classify them and give an accuracy, if you used that for annotations? How we can establish the efficiency of geometric methods if so? It makes no sense for me the evaluation.ipynb and the pipeline.ipynb, could you provide please further explanations or provide the scripts which you used to achieve so? 'gt_df' is not defined, 'predictions_IMP' is not defined in dataset folder and moreover it's impossible to retrieve. Could you please look back at the code since it is almost all incomplete or unsufficient? I need that to compare results, but it is impossible to get your results. I hope you can provide those answers asap. Thanks

[johnnynunez]

Moreover, I would like to ask, since it is not well defined in the paper, how this loitering annotations are perfomed? I mean, which are the criteria to establish and auto-annotate if that trajectory is an anomalous one? For instance, let's suppose we have a new video, as I read we need to keep standard length to 120, but are the trajectory methods applied themselves to detect loitering? among which ones are used? Convex hull or other? Thanks

It is defined by a protocol that we made with the members of the company that we consider loitering. We mannually annotated. But it was very hard annotate start/end frames that occurs the loitering, so we decided to annotated all trajectory. So, we decided 4 cases of loitering.

[gguzzy]

These 4 cases corresponds to the geometric methods you used? For instance convex_hull, rectangular etc? Hence to annotate new data you suggest to manually label all the frames of the trajectory where the loitering has been applied? One more question, about the IPM, if we change perspective of camera, we need to define a new IPM? Do you suggest working with trajectory length of 120 in any case? Thanks a lot.

[johnnynunez]

Can you please explain how the annotations were made? Using a weighted vote of the geometric methods? If yes, which threshold did you use and why? How did you obtain those? For geometric methods how did you classify them and give an accuracy, if you used that for annotations? How we can establish the efficiency of geometric methods if so? It makes no sense for me the evaluation.ipynb and the pipeline.ipynb, could you provide please further explanations or provide the scripts which you used to achieve so? 'gt_df' is not defined, 'predictions_IMP' is not defined in dataset folder and moreover it's impossible to retrieve. Could you please look back at the code since it is almost all incomplete or unsufficient? I need that to compare results, but it is impossible to get your results. I hope you can provide those answers asap. Thanks

Can you please explain how the annotations were made? Using a weighted vote of the geometric methods? If yes, which threshold did you use and why? How did you obtain those? For geometric methods how did you classify them and give an accuracy, if you used that for annotations? How we can establish the efficiency of geometric methods if so? It makes no sense for me the evaluation.ipynb and the pipeline.ipynb, could you provide please further explanations or provide the scripts which you used to achieve so? 'gt_df' is not defined, 'predictions_IMP' is not defined in dataset folder and moreover it's impossible to retrieve. Could you please look back at the code since it is almost all incomplete or unsufficient? I need that to compare results, but it is impossible to get your results. I hope you can provide those answers asap. Thanks

The final threshold was using finally using optuna to detect loitering. It can be used for example with genetic algorithm like most of trackers papers. The paper was initial exploration, it can be improved a lot, but these dataset it is very challenge. It need super resolution and smoothing. (For example, it was very challenging the angles of the circle to point a closed areas etc).

IMP and normalization finally was not done. Not improve the results. But new IMP it is necessary to do. I didn't get the parameters of the camera etc. 4 cases of loitering were that the team decided. The annotations corresponds with all trajectory. It is binary annotation. It could be improved by the beahavior, seated, abnormal trajectory etc and start/end frames

For the data, they were manually annotated. I no longer work for Milestone Systems, if they publish the dataset, I may update the code. I share you the poster.

Poster Loitering.pptx

FYI: For me, it was very challenging annotated that... The first idea was annotated 4 months, but the majority of annotations were done by me mannually