J&f evaluation ALL return 1.0?

HetingJian commented 4 years ago

Thanks for seeing this! When i was trying to use the dataset from OSVOS and RVOS given in your repo as '--results_path', the result return from terminal and .csv shows all J&F-mean are 1.0 (i thought its totally the same as the ground-truth when returning this value,well obviosly its not ), so how to fix it and get the right evaluation result? Thanks , i am a beginner and feeling confused.

After my DAVIS path is well-setted my configuration parameters:

--task
semi-supervised
--results_path
results/semi-supervised/osvos

terminal returns

---------- Per sequence results for val ----------
             Sequence  J-Mean  F-Mean
       bike-packing_1     1.0     1.0
       bike-packing_2     1.0     1.0
          blackswan_1     1.0     1.0
          bmx-trees_1     1.0     1.0
          bmx-trees_2     1.0     1.0
         breakdance_1     1.0     1.0
              camel_1     1.0     1.0
     car-roundabout_1     1.0     1.0
         car-shadow_1     1.0     1.0
               cows_1     1.0     1.0
        dance-twirl_1     1.0     1.0
                dog_1     1.0     1.0
          dogs-jump_1     1.0     1.0
          dogs-jump_2     1.0     1.0

dxlong2000 commented 4 years ago

Could you show full your terminal result as well as your code and set-up on how to generate the result?

SaralaJ commented 3 years ago

same error. all returns 1.0. any guidance plz

scaelles commented 3 years ago

Could you provide more information such as the python version as well as the other packages that you have installed?

SaralaJ commented 3 years ago

I downloaded Github File in anaconda prompt - Run the setup.py- downloaded DAVIS 2017 dataset- added the DAVISpath in Evaluation_method.py- then run python evaluation_method.py --task semi-supervised --results_path results/semi-supervised/osvos -----------------and the score I get is only 1.0 for all.
The setup.py has setup( python_requires='>=3.6, <4', install_requires=[ 'Pillow>=4.1.1', 'networkx>=2.0', 'numpy>=1.12.1', 'opencv-python>=4.0.0.21', 'pandas>=0.21.1', 'pathlib2;python_version<"3.5"', 'scikit-image>=0.13.1', 'scikit-learn>=0.18', 'scipy>=1.0.0', 'tqdm>=4.28.1'

xlee1013 commented 3 years ago

Has this problem been solved? I also had the same problem...

xlee1013 commented 3 years ago

same error. all returns 1.0. any guidance plz

I downloaded Github File in anaconda prompt - Run the setup.py- downloaded DAVIS 2017 dataset- added the DAVISpath in Evaluation_method.py- then run python evaluation_method.py --task semi-supervised --results_path results/semi-supervised/osvos -----------------and the score I get is only 1.0 for all. The setup.py has setup( python_requires='>=3.6, <4', install_requires=[ 'Pillow>=4.1.1', 'networkx>=2.0', 'numpy>=1.12.1', 'opencv-python>=4.0.0.21', 'pandas>=0.21.1', 'pathlib2;python_version<"3.5"', 'scikit-image>=0.13.1', 'scikit-learn>=0.18', 'scipy>=1.0.0', 'tqdm>=4.28.1'

Has this problem been solved? I also had the same problem...

SaralaJ commented 3 years ago

Not yet!!! Waiting ... :(

On Thu, 24 Dec 2020, 8:02 pm xlee1013, notifications@github.com wrote:

Has this problem been solved? I also had the same problem...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/davisvideochallenge/davis2017-evaluation/issues/7#issuecomment-750917742, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOLYB5Q5UF3RFQ63HGAO5CTSWNQ2HANCNFSM4OEEX5PQ .

PGSmall commented 3 years ago

I have no problem and this is my modified code (https://github.com/PGSmall/DAVIS-Evaluation) according to Issues #4.

xlee1013 commented 3 years ago

Thanks for seeing this! When i was trying to use the dataset from OSVOS and RVOS given in your repo as '--results_path', the result return from terminal and .csv shows all J&F-mean are 1.0 (i thought its totally the same as the ground-truth when returning this value,well obviosly its not ), so how to fix it and get the right evaluation result? Thanks , i am a beginner and feeling confused.

After my DAVIS path is well-setted my configuration parameters:
--task
semi-supervised
--results_path
results/semi-supervised/osvos
terminal returns
---------- Per sequence results for val ----------
             Sequence  J-Mean  F-Mean
       bike-packing_1     1.0     1.0
       bike-packing_2     1.0     1.0
          blackswan_1     1.0     1.0
          bmx-trees_1     1.0     1.0
          bmx-trees_2     1.0     1.0
         breakdance_1     1.0     1.0
              camel_1     1.0     1.0
     car-roundabout_1     1.0     1.0
         car-shadow_1     1.0     1.0
               cows_1     1.0     1.0
        dance-twirl_1     1.0     1.0
                dog_1     1.0     1.0
          dogs-jump_1     1.0     1.0
          dogs-jump_2     1.0     1.0

The problem is solved, a file needs to be replaced, and a modification is made in line 92(https://github.com/xlee1013/DAVIS2017-evaluation).

SaralaJ commented 3 years ago

Thank you xleee1013, I got the values for your code, with few modifications in it...

BigBoomDream commented 2 years ago

谢谢你看到这个！当我尝试使用您的回购中给出的 OSVOS 和 RVOS 数据集作为“--results_path”时，终端返回的结果和 .csv 显示所有 J&F 均值均为 1.0（我认为它与事实完全一样返回此值时，显然不是），那么如何修复它并获得正确的评估结果？谢谢，我是初学者，感到困惑。

在我的 DAVIS 路径设置好后，我的配置参数：
--task
semi-supervised
--results_path
results/semi-supervised/osvos
终端返回
---------- Per sequence results for val ----------
             Sequence  J-Mean  F-Mean
       bike-packing_1     1.0     1.0
       bike-packing_2     1.0     1.0
          blackswan_1     1.0     1.0
          bmx-trees_1     1.0     1.0
          bmx-trees_2     1.0     1.0
         breakdance_1     1.0     1.0
              camel_1     1.0     1.0
     car-roundabout_1     1.0     1.0
         car-shadow_1     1.0     1.0
               cows_1     1.0     1.0
        dance-twirl_1     1.0     1.0
                dog_1     1.0     1.0
          dogs-jump_1     1.0     1.0
          dogs-jump_2     1.0     1.0

@HetingJian Hi，I also encountered the same problem, how did you solve this problem?

davisvideochallenge / davis2017-evaluation

J&f evaluation ALL return 1.0? #7