search

votchallenge / toolkit

The official VOT Challenge evaluation and analysis toolkit
http://www.votchallenge.net/
GNU General Public License v3.0
155 stars 43 forks source link

Some problems when run vot2020 #3

Closed MasterBin-IIAU closed 4 years ago

MasterBin-IIAU commented 4 years ago

Hi, dear vot committe, I found that there might be some problems in the toolkit. 1st problem The sequence file in each video directory is like this: channels.color=color/color But there is only one 'color' directory, so I guess it should be channels.color=color

MasterBin-IIAU commented 4 years ago

2nd problem After fix the 1st problem, I ran 'vot evaluate' then I got the following error:

Traceback (most recent call last): File "/home/masterbin-iiau/anaconda3/envs/vot20/bin/vot", line 8, in sys.exit(main()) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/cli.py", line 322, in main do_evaluate(args, logger) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/cli.py", line 141, in do_evaluate workspace = Workspace(config.workspace) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/workspace.py", line 119, in init self._dataset = VOTDataset(dataset_directory) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/dataset/vot.py", line 104, in init self._sequences = {name.strip() : VOTSequence(os.path.join(path, name.strip()), dataset=self) for name in Progress(names, desc="Loading dataset", unit="sequences") } File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/dataset/vot.py", line 104, in self._sequences = {name.strip() : VOTSequence(os.path.join(path, name.strip()), dataset=self) for name in Progress(names, desc="Loading dataset", unit="sequences") } File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/dataset/vot.py", line 34, in init self._scan(base) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/dataset/vot.py", line 56, in _scan self.groundtruth.append(parse(region)) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/region/io.py", line 14, in parse m, offset_ = create_mask_from_string(string[1:].split(',')) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/region/utils.py", line 155, in create_mask_from_string mask = rle_to_mask(rle, region_w, region_h) IndexError: setitem out of range

After some debugs, I found that this error is caused by some extreme cases in the groundtruth.txt. In the 16th frame of video 'agility', the target is fully occluded, the groundtruth is 'm0,0,0,0,0,1'. But it seems that the function rle_to_mask can not handle this situation properly. (In this case, rle=[0 1], region_w=0, region_h=0)

JudasDie commented 4 years ago

2nd problem After fix the 1st problem, I ran 'vot evaluate' then I got the following error:

Traceback (most recent call last): File "/home/masterbin-iiau/anaconda3/envs/vot20/bin/vot", line 8, in sys.exit(main()) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/cli.py", line 322, in main do_evaluate(args, logger) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/cli.py", line 141, in do_evaluate workspace = Workspace(config.workspace) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/workspace.py", line 119, in init self._dataset = VOTDataset(dataset_directory) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/dataset/vot.py", line 104, in init self._sequences = {name.strip() : VOTSequence(os.path.join(path, name.strip()), dataset=self) for name in Progress(names, desc="Loading dataset", unit="sequences") } File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/dataset/vot.py", line 104, in self._sequences = {name.strip() : VOTSequence(os.path.join(path, name.strip()), dataset=self) for name in Progress(names, desc="Loading dataset", unit="sequences") } File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/dataset/vot.py", line 34, in init self._scan(base) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/dataset/vot.py", line 56, in _scan self.groundtruth.append(parse(region)) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/region/io.py", line 14, in parse m, offset_ = create_mask_from_string(string[1:].split(',')) File "/home/masterbin-iiau/anaconda3/envs/vot20/lib/python3.6/site-packages/vot/region/utils.py", line 155, in create_mask_from_string mask = rle_to_mask(rle, region_w, region_h) IndexError: setitem out of range

After some debugs, I found that this error is caused by some extreme cases in the groundtruth.txt. In the 16th frame of video 'agility', the target is fully occluded, the groundtruth is 'm0,0,0,0,0,1'. But it seems that the function rle_to_mask can not handle this situation properly. (In this case, rle=[0 1], region_w=0, region_h=0)

same problem.

foolwood commented 4 years ago

Please also check the empty mask initialization problem.

After integrating python_ncc to this toolkit, I meet an OverflowError.

The handle provides a selection Rectangle with (x=3.4028234663852886e+38, y=3.4028234663852886e+38, width=-inf, height=-inf).

The ground-truth label (mask) is m0,0,0,0,0,1, which means the toolkit re-set tracker with an empty mask.

lukacu commented 4 years ago

Thanks all for sending errors, it seems that we had some issues with dataset packaging that we are addressing.

lukacu commented 4 years ago

We have corrected the issues with the dataset. You will have to update the toolkit (change in experiment stack) and download the dataset again (metadata issue). In the future please open a separate issue for each problem.