poor results on teamtime

[feixue94]

Dear author,

Thanks for your excellent work. I evaluated the released pretrained_model on teatime. The localization accuracy is only 0.1017.

Do you have any idea what the problem is.

2024-04-25 16:47:42,245 - teatime - INFO - trunc thresh: 0.4 INFO:teatime:trunc thresh: 0.4 2024-04-25 16:47:42,245 - teatime - INFO - iou chosen: 0.0245 INFO:teatime:iou chosen: 0.0245 2024-04-25 16:47:42,248 - teatime - INFO - chosen_lvl: [array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0)] INFO:teatime:chosen_lvl: [array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0)] 2024-04-25 16:47:42,248 - teatime - INFO - Localization accuracy: 0.1017 INFO:teatime:Localization accuracy: 0.1017

[minghanqin]

Hi, Thanks for your attention. All of your chosen level is 0, which means that the model are selecting only Level S results. Have you correctly loaded the models for all three levels?

[feixue94]

thanks for your reply. I figured it out.

[davidpengiupui]

Hi, @feixue94 and @minghanqin ! I also obtained similar bad results after runing eval.sh. Could you please tell me how to fix it? This was how I executed:

python render.py -m output/teatime_1 --feature_level 1 python render.py -m output/teatime_2 --feature_level 2 python render.py -m output/teatime_3 --feature_level 3

cd eval bash eval.sh

[feixue94]

Hi @davidpengiupui , My problem was that the only features of level 1 was extracted.

[davidpengiupui]

Hi @feixue94 ! Thank you for your reply! Do you mean that I need to extract things like teatime_1, teatime_2 and teatime_3 in the argument "feat_dir"? I have done that but still got poor results. Do you change something in the eval.sh regarding this?

[feixue94]

I think you need to to extract multi-level features again.

[davidpengiupui]

I think you need to to extract multi-level features again.

I understand. Thank you!

[yquantao]

I think you need to to extract multi-level features again.

I tried the command below and still got the same error.

python render.py -m output/teatime_1
python render.py -m output/teatime_2
python render.py -m output/teatime_3

What is the right command to extract multi-level features?

[feixue94]

Hi @yquantao please check if you have 3 encoders for 3 different levels of feature

[Zhaooyy]

Hi, @feixue94 and @minghanqin ! I also obtained similar bad results after runing eval.sh. Could you please tell me how to fix it? This was how I executed:

python render.py -m output/teatime_1 --feature_level 1 python render.py -m output/teatime_2 --feature_level 2 python render.py -m output/teatime_3 --feature_level 3

cd eval bash eval.sh

how to extract multi-level features? I have the same problem here, have you fixed it?

[Zhaooyy]

hi, after I runned: python render.py -m output/teatime_1 --feature_level 1 python render.py -m output/teatime_2 --feature_level 2 python render.py -m output/teatime_3 --feature_level 3

sh eval.sh

I got the poor result like yours, how to make sure my "3 encoders for 3 different levels of feature"?

[wxrui182]

I have the same problem. How to fix it?

[garrisonz]

Hi @yquantao please check if you have 3 encoders for 3 different levels of feature

Hi, Can you provide more details? How can I ensure that I have 3 encoders for 3 different levels of feature? Any code I need to change in the project to achieve this result? Look forward for your reply.