votchallenge / toolkit

The official VOT Challenge evaluation and analysis toolkit
http://www.votchallenge.net/
GNU General Public License v3.0
170 stars 46 forks source link

failed to evaluate integration tracker - 'SingleObject' object has no attribute '__name__' #108

Closed barnoy1 closed 10 months ago

barnoy1 commented 1 year ago

vot fails when trying to evaluate the integration python tracker provided in the intergration repo.

upon evaluation using the following command line:

vot evaluate --workspace . NCCPython

vot generates an error implying the tracker is not valid (name attribute missing):

'SingleObject' object has no attribute '__name__'
Traceback (most recent call last):
  File "/home/ronbar/repo/vot/venv/lib/python3.8/site-packages/vot/utilities/cli.py", line 445, in main
    do_evaluate(args)
  File "/home/ronbar/repo/vot/venv/lib/python3.8/site-packages/vot/utilities/cli.py", line 228, in do_evaluate
    run_experiment(experiment, tracker, workspace.dataset, config.force, config.persist)
  File "/home/ronbar/repo/vot/venv/lib/python3.8/site-packages/vot/experiment/__init__.py", line 339, in run_experiment
    transformed.extend(experiment.transform(sequence))
  File "/home/ronbar/repo/vot/venv/lib/python3.8/site-packages/vot/experiment/__init__.py", line 276, in transform
    get_logger().debug("Transforming sequence {} with transformer {}.{}".format(sequence.identifier, transformer.__module__, transformer.__name__))
AttributeError: 'SingleObject' object has no attribute '__name__'

below are the system requirements and detailed steps to reproduce. Thank you in advance.


system info:

Distributor ID: Ubuntu Description: Ubuntu 20.04.5 LTS Release: 20.04 Kernel: 5.15.0-48-generic #54~20.04.1-Ubuntu SMP Thu Sep 1 16:17:26 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

steps to reproduce:

1. bring up

install latest vot-toolkit into project venv

pip install vot-toolkit

2. create workspace

creating new workspace named tracker_ws, and create workspace with basic sequences

vot initialize --workspace tracker_ws tests/basic

3. integration

integrate vot with default tracker as described in support page

3.1 download integration code into target project

git clone git@github.com:votchallenge/integration.git

3.2 configure tracker_ws/trackers.ini file to the default python tracker

[NCCPython]
label = PyNCC
protocol = traxpython
command = ncc_multiobject_manager
# Specify a path to trax python wrapper if it is not visible (separate by ; if using multiple paths)
paths = <FULL_PATH>/integration/python
# Additional environment paths
env_PATH = /usr/local/lib;<FULL_PATH>/integration/python;<FULL_PATH>/trax/build;${PATH}

4. sanity check

testing if tracker is identified.

running vot test inside the workspace reveal that the tracker is present:

cd tracker_ws 
vot test
```bash
List of found trackers: 
 * NCCPython

5. evaluate tracker: FAILED

At this point I tried to run evaluate on this tracker:

vot evaluate --workspace . NCCPython

and got the following error

'SingleObject' object has no attribute '__name__'
Traceback (most recent call last):
  File "/home/ronbar/repo/vot/venv/lib/python3.8/site-packages/vot/utilities/cli.py", line 445, in main
    do_evaluate(args)
  File "/home/ronbar/repo/vot/venv/lib/python3.8/site-packages/vot/utilities/cli.py", line 228, in do_evaluate
    run_experiment(experiment, tracker, workspace.dataset, config.force, config.persist)
  File "/home/ronbar/repo/vot/venv/lib/python3.8/site-packages/vot/experiment/__init__.py", line 339, in run_experiment
    transformed.extend(experiment.transform(sequence))
  File "/home/ronbar/repo/vot/venv/lib/python3.8/site-packages/vot/experiment/__init__.py", line 276, in transform
    get_logger().debug("Transforming sequence {} with transformer {}.{}".format(sequence.identifier, transformer.__module__, transformer.__name__))
AttributeError: 'SingleObject' object has no attribute '__name__'
Tchuanm commented 1 year ago

Hi, you solved this or not? Btw, are you evaluated on VOT2020 or VOTS2023

lukacu commented 1 year ago

Yes, the current main branch has some problems with old challenges that we are trying to address, but time is limited. It is best to try using 0.5.X and trax 3.X . Or wait for August when we will finally have time to fix this issue.

Tchuanm commented 1 year ago

Great, 0.5.x is useful for evaluate vot2020。

barnoy1 commented 1 year ago

Ok I used vot2023 and the error not reproduced. I mean that the missing attribute error is gone.

I'm new to VOT so please forgive me for asking, but what do you mean 0.5.X and 3.X trax?

I have encountered some issues installing trax. Can you please tell me how to install trax 3.x ? I've just tried to complie it from source via cmake

lukacu commented 1 year ago

Both can be installed from pypi by pinning the version of the package. No need to explicitly build anything, just browse the version list and select the most recent one that suits the requirement that I wrote.

lukacu commented 10 months ago

I have now gone through previous challenges and datasets and fixed all errors that I have encountered. Please test version 0.6.5 if the errors are gone.