MQ empty annotation - Githubissues

dairui01 commented 2 years ago

Hello,

I face the following issue while evaluating the prediction:

Retrieval evaluation starts!

a. Generate retrieval! joblib.externals.loky.process_executor._RemoteTraceback: """ Traceback (most recent call last): File "/home/rdai/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/process_executor.py", line 431, in _process_worker r = call_item() File "/home/rdai/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/process_executor.py", line 285, in call return self.fn(*self.args, *self.kwargs) File "/home/rdai/anaconda3/lib/python3.7/site-packages/joblib/_parallel_backends.py", line 593, in call return self.func(args, **kwargs) File "/home/rdai/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 253, in call for func, args, kwargs in self.items] File "/home/rdai/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 253, in for func, args, kwargs in self.items] File "/data/stars/user/rdai/PhD_work/Ego4d/code/episodic-memory/MQ/Evaluation/ego4d/generate_retrieval.py", line 88, in _gen_retrieval_video df = rm_other_category(df, test_anno['annotations'], classes) File "/data/stars/user/rdai/PhD_work/Ego4d/code/episodic-memory/MQ/Evaluation/ego4d/generate_retrieval.py", line 73, in rm_other_category df_v = pd.concat(df_v) File "/home/rdai/anaconda3/lib/python3.7/site-packages/pandas/core/reshape/concat.py", line 225, in concat copy=copy, sort=sort) File "/home/rdai/anaconda3/lib/python3.7/site-packages/pandas/core/reshape/concat.py", line 259, in init raise ValueError('No objects to concatenate') ValueError: No objects to concatenate

Then I check the clip_annotations.json generated by the Convert_annotations.py is empty in the 'annotations'. I am predicting with the Slowfast feature from ego4d and the baseline code in this repo.

fedegonzal commented 1 year ago

Hey, news about this error? I'm facing on the same problem. Thanks!

coolbay commented 1 year ago

It looks like you are using a different version of the annotation file, which has been updated several times. Could you please provide the annotation file (input to the Convert_annotations.py) you are using?

fedegonzal commented 1 year ago

I used this command to download annotations: ego4d --output_directory="./data" --datasets annotations --benchmarks EM --version v2

You can find my annotations folder here: https://drive.google.com/drive/folders/1-BW-ZkFiDoodzRjKFvRwBgFfgH5vzLTQ?usp=sharing

Later I'm running:

Convert_annotation.py
Train.py --use_xGPN --is_train true --dataset ego4d --feature_path data/v2/slowfast8x8_r101_k400/ --checkpoint_path data/v2/moments_models/ --clip_anno data/v2/annotations/clip_annotations.json --moment_classes data/v2/annotations/moment_classes_idx.json --batch_size 4 --train_lr 0.0001
Infer.py --use_xGPN --is_train false --dataset ego4d --feature_path data/v2/slowfast8x8_r101_k400/ --checkpoint_path data/v2/moments_models/ --moment_classes data/v2/annotations/moment_classes_idx.json --clip_anno data/v2/annotations/clip_annotations.json --output_path inferences/ --batch_size 4 --infer_datasplit test
Eval.py --output_path inferences/ --clip_anno data/v2/annotations/clip_annotations.json --moment_classes data/v2/annotations/moment_classes_idx.json --eval_stage all --infer_datasplit test

Eval should generate two files:

detections_postNMS.json seems to be fine (but without annotations at the final part)
retrieval_postNMS.json (facing on an error because it can't found annotations in the previous file)

If you need something else, just tell me. I want to resolve it soon as possible! Thanks

coolbay commented 1 year ago

@fedegonzal

I tried running "Convert_annotation.py" using your annotation files "moments_train.json", and "moments_val.json". But I didn't see the [info_path] "ego4d.json" in your folder, so I prepared it myself. Doing so, the output annotation file "clip_annotations.json" I got is fine and not empty.

Can you check if you got an empty "clip_annotations.json" from your Step 1?
If so, may I ask if you have used all the input files required by "Convert_annotation.py"? 1) [feat_path], 2)[info_path], 3)[annot_path_train], [annot_path_val], [annot_path_test].

fedegonzal commented 1 year ago

This is my generated clip annotations https://drive.google.com/file/d/1qLFFn0wnYIsWCOUEEi1MHMRj4DglQUSP/view?usp=drivesdk thank you by supporting me!

El El dom, 30 de abr. de 2023 a la(s) 09:56, Cathy @.***> escribió:

@fedegonzal https://github.com/fedegonzal

I tried running "Convert_annotation.py" using your annotation files "moments_train.json", and "moments_val.json". But I didn't see the [info_path] "ego4d.json" in your folder, so I prepared it myself. Doing so, the output annotation file "clip_annotations.json" I got is fine and not empty.

Can you check if you got an empty "clip_annotations.json" from you Step 1?

If so, may I ask if you have used all the input files required by "Convert_annotation.py"? 1) [feat_path], 2)[info_path], 3)[annot_path_train], [annot_path_val], [annot_path_test].

— Reply to this email directly, view it on GitHub https://github.com/EGO4D/episodic-memory/issues/36#issuecomment-1528963390, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGJSBWZ2Q5N72PAJMIQJDTXDYLLJANCNFSM6AAAAAAQOPKXCM . You are receiving this because you were mentioned.Message ID: @.***>

fedegonzal commented 1 year ago

Hi @coolbay I couldn't fix my problem

This is my retrieval_postNMS.json result, just empty values:

{ "375e2d5b-31fa-4d21-8aa9-8bdfba124681": [], 
  "365d86c5-3b3c-4fc5-82e0-948b2807b32e": [], 
  ...
}

I tried using the official EM model from the github repositry. I downloaded again annotations, slowfast features, I ran again Convert_annotation.py, Infer.py, but I have everytime the same empty result.

To run Convert_annotation.py I'm using this files as input:

data/ego4d.json
data/v2/slowfast8x8_r101_k400/
data/v2/annotations/moments_train.json
data/v2/annotations/moments_val.json
data/v2/annotations/moments_test_unannotated.json

I must to use moments_test_unannotated.json that's right?

My clip_annotations.json file is here: https://drive.google.com/file/d/1GT3yW2mc8opF9s1bGurBN1S6emrZ7XIJ/view?usp=share_link

And here the moment_classes_idx.json file: https://drive.google.com/file/d/1-NANZXoF6KzjZKzaUEeu7rvDYjo55CRL/view?usp=share_link

Later I'm using this command to Infer.py (may be there is the problem, I don't know):

python episodic-memory/MQ/Infer.py \
  --use_xGPN \
  --is_train false \
  --dataset ego4d --feature_path data/v2/slowfast8x8_r101_k400/ \
  --checkpoint_path data/v2/moments_models_baseline/ \
  --moment_classes data/v2/annotations/moment_classes_idx.json \
  --clip_anno data/v2/annotations/clip_annotations.json  \
  --output_path inferences/ \
  --batch_size 4 \
  --infer_datasplit test

And this for Eval.py

python episodic-memory/MQ/Eval.py \
  --output_path inferences/ \
  --clip_anno data/v2/annotations/clip_annotations.json \
  --moment_classes data/v2/annotations/moment_classes_idx.json \
  --eval_stage all \
  --infer_datasplit test \
  --use_xGPN

Thanks for your time!

EGO4D / episodic-memory

MQ empty annotation #36

Retrieval evaluation starts!