[BUG] Unable to load annotation results: 'NoneType' object has no attribute 'load_credentials'

Commands to reproduce

import fiftyone as fo

dataset_name='fiftyone_cvat'
dataset = fo.load_dataset(dataset_name)
project_name='fiftyone_proj'
anno_key='fiftyone_cvat'
dataset.annotate(anno_key, project_name=project_name, launch_editor=True)

Describe the problem

I create a dataset with 2 image, successfully upload to CVAT and annotated them, but fails to load_annotations using dataset.load_annotations(anno_key) I can successfully export dataset with annotations manually in https://app.cvat.ai/ but using API will occurs an error

Code to reproduce issue

Provide a reproducible test case that is the bare minimum necessary to generate the problem.

Other info/logs

In [7]: dataset.load_annotations(anno_key)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-7-b04bc9154234> in <module>
----> 1 dataset.load_annotations(anno_key)

~\AppData\Roaming\Python\Python39\site-packages\fiftyone\core\collections.py in load_annotations(self, anno_key, dest_field, unexpected, cleanup, **kwargs)
   7596             found, in which case a dict containing the extra labels is returned
   7597         """
-> 7598         return foua.load_annotations(
   7599             self,
   7600             anno_key,

~\AppData\Roaming\Python\Python39\site-packages\fiftyone\utils\annotations.py in load_annotations(samples, anno_key, dest_field, unexpected, cleanup, **kwargs)
   1034     """
   1035     dataset = samples._root_dataset
-> 1036     results = dataset.load_annotation_results(anno_key, **kwargs)
   1037     annotations = results.backend.download_annotations(results)
   1038     label_schema = results.config.label_schema

~\AppData\Roaming\Python\Python39\site-packages\fiftyone\core\collections.py in load_annotation_results(self, anno_key, **kwargs)
   7539             self, anno_key, load_view=False
   7540         )
-> 7541         results.load_credentials(**kwargs)
   7542         return results
   7543 

AttributeError: 'NoneType' object has no attribute 'load_credentials'

import fiftyone.core.annotation as foan
run_doc = foan.AnnotationMethod._get_run_doc(dataset, anno_key)

In [11]: run_doc
Out[11]:
<RunDocument: {
    'key': 'fiftyone_cvat',
    'version': '0.18.0',
    'timestamp': datetime.datetime(2023, 2, 17, 9, 8, 6, 139000),
    'config': {
        'method': 'cvat',
        'cls': 'fiftyone.utils.cvat.CVATBackendConfig',
        'name': 'cvat',
        'label_schema': {'None': {}},
        'media_field': 'filepath',
        'url': 'https://app.cvat.ai',
        'task_size': None,
        'segment_size': None,
        'image_quality': 75,
        'use_cache': True,
        'use_zip_chunks': True,
        'chunk_size': None,
        'task_assignee': None,
        'job_assignees': None,
        'job_reviewers': None,
        'project_name': 'fiftyone_proj',
        'project_id': None,
        'occluded_attr': None,
        'group_id_attr': None,
        'issue_tracker': None,
    },
    'view_stages': [],
    'results': <GridFSProxy: None>,
}>

From the run_doc we can see the results is even though I finish annotation

What areas of FiftyOne does this bug affect?

[x] Core: Core Python library issue

Willingness to contribute

[x] No. I cannot contribute a bug fix at this time

I'm unable to reproduce this running the following:

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart", max_samples=1)
dataset.name = "test"
dataset.persistent = True

dataset.annotate("test_key", label_field="ground_truth", project_name="test")

In another session:

import fiftyone as fo

dataset = fo.load_dataset("test")
results = dataset.load_annotation_results("test_key")
assert results is not None

Can you try running this?

Any additional context as to the sequence of operations you might have performed on the dataset between when you called annotate() and when you called load_annotations()?

Where you working with a source install of FiftyOne at any point over the past few months by chance?

I'm unable to reproduce this running the following:
import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart", max_samples=1)
dataset.name = "test"
dataset.persistent = True

dataset.annotate("test_key", label_field="ground_truth", project_name="test")
In another session:
import fiftyone as fo

dataset = fo.load_dataset("test")
results = dataset.load_annotation_results("test_key")
assert results is not None
Can you try running this?

Any additional context as to the sequence of operations you might have performed on the dataset between when you called annotate() and when you called load_annotations()?

Where you working with a source install of FiftyOne at any point over the past few months by chance?

Hi Brimoor, thank you for your reply, let me provide more context. I create a dataset using:

import fiftyone as fo
from glob import glob

dataset = fo.Dataset('fiftyone_cvat')
images_patt='./images/*'
samples = [fo.Sample(filepath=filepath) for filepath in glob(images_patt, recursive=True)]
dataset.add_samples(samples)

Then

project_name='fiftyone_proj'
anno_key='fiftyone_cvat'
dataset.annotate(anno_key, project_name=project_name, launch_editor=True)

From the fiftyone app I can see my dataset successfully get the images, and then I open CVAT Assign project and task to me I define a label called 'test_label' which is a polygon, and for each image, I do annotation using this polygon label. Then I save the annotations, then select acceptable to make it completed.

Then go back to my ipython terminal, I run:

dataset.load_annotations(anno_key)

Then raise an error, as above, FYI

I'm unable to reproduce this running the following:
import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart", max_samples=1)
dataset.name = "test"
dataset.persistent = True

dataset.annotate("test_key", label_field="ground_truth", project_name="test")
In another session:
import fiftyone as fo

dataset = fo.load_dataset("test")
results = dataset.load_annotation_results("test_key")
assert results is not None
Can you try running this?

Any additional context as to the sequence of operations you might have performed on the dataset between when you called annotate() and when you called load_annotations()?

Where you working with a source install of FiftyOne at any point over the past few months by chance?

Hi brimoor, thank you for your code，I try it out, but for me it raise an error, I am in anaconda env, FYI

In [1]: import fiftyone as fo
   ...: import fiftyone.zoo as foz
   ...:
   ...: dataset = foz.load_zoo_dataset("quickstart", max_samples=1)
   ...: dataset.name = "test"
   ...: dataset.persistent = True
   ...:
   ...: dataset.annotate("test_key", label_field="ground_truth", project_name="test")
Dataset already downloaded
Loading 'quickstart'
 100% |██████████████████████████████████████████████████████████████████████████████████████| 1/1 [28.2ms elapsed, 0s remaining, 35.8 samples/s] 
Dataset 'quickstart-1' created
Found existing field 'ground_truth' with multiple types ['detections', 'instances']. Only the 'detections' will be annotated
Please enter your login credentials.
You can avoid this in the future by setting your `FIFTYONE_CVAT_USERNAME` and `FIFTYONE_CVAT_PASSWORD` environment variables
Username: *******
Password:
Computing metadata...
 100% |██████████████████████████████████████████████████████████████████████████████████████| 1/1 [12.5s elapsed, 0s remaining, 0.1 samples/s] 
Uploading samples to CVAT...
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-1-2d6f516b7965> in <module>
      6 dataset.persistent = True
      7
----> 8 dataset.annotate("test_key", label_field="ground_truth", project_name="test")
utils\cvat.py in upload_samples(self, samples, backend)
   4298                     class_id_map,   4299                     attr_id_map,
-> 4300                 ) = self._create_task_upload_data(
   4301                     config,
   4302                     idx,

~\AppData\Roaming\Python\Python39\site-packages\fiftyone\utils\cvat.py in _create_task_upload_data(self, config, idx, task_name, cvat_schema, project_id, samples_batch, task_ids, job_ids, frame_id_map)
   5065
   5066         # Upload media-> 5067         job_ids[task_id] = self.upload_data(
   5068             task_id,
   5069             samples_batch.values(media_field),

~\AppData\Roaming\Python\Python39\site-packages\fiftyone\utils\cvat.py in upload_data(self, task_id, paths, image_quality, use_cache, use_zip_chunks, chunk_size, job_assignees, job_reviewers)
   4131         while not job_ids:
   4132             job_resp = self.get(self.jobs_url(task_id))
-> 4133             job_ids = [j["id"] for j in job_resp.json()]
   4134
   4135         if job_assignees is not None:

~\AppData\Roaming\Python\Python39\site-packages\fiftyone\utils\cvat.py in <listcomp>(.0)
   4131         while not job_ids:
   4132             job_resp = self.get(self.jobs_url(task_id))
-> 4133             job_ids = [j["id"] for j in job_resp.json()]
   4134
   4135         if job_assignees is not None:

TypeError: string indices must be integers

In [2]:

But it seems still upload to CVAT successfully

Then I open another terminal and run:

In [1]: import fiftyone as fo
   ...:
   ...: dataset = fo.load_dataset("test")
   ...: results = dataset.load_annotation_results("test_key")
   ...: assert results is not None
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-bd74ad2caa92> in <module>
      2 
      3 dataset = fo.load_dataset("test")
----> 4 results = dataset.load_annotation_results("test_key")
      5 assert results is not None

~\AppData\Roaming\Python\Python39\site-packages\fiftyone\core\collections.py in load_annotation_results(self, anno_key, **kwargs)
   7539             self, anno_key, load_view=False
   7540         )
-> 7541         results.load_credentials(**kwargs)
   7542         return results
   7543 

AttributeError: 'NoneType' object has no attribute 'load_credentials'

In [2]:

The error seems to be the same above

Ah okay, now we've found the actual issue. The annotate() call wasn't successful in the first place. It failed with this error:

TypeError: string indices must be integers

This should be resolved by upgrading to fiftyone>=0.19.1, which includes this patch that upgrades the CVAT integration to support CVAT 2.4.

Ah okay, now we've found the actual issue. The annotate() call wasn't actually successful; it failed with this error:
TypeError: string indices must be integers
This should be resolved by upgrading to fiftyone>=0.19.1, which includes this patch that upgrades the CVAT integration to support CVAT 2.4.

Hi brimoor, I try your suggestion upgrading to fiftyone>=0.19.1, then it seems resolved this issues,

(base) C:\code>pip install --upgrade fiftyone
Successfully installed dacite-1.7.0 fiftyone-0.19.1 fiftyone-brain-0.10.0 voxel51-eta-0.8.3

Then run the following:

In [4]: import fiftyone as fo
   ...: import fiftyone.zoo as foz
   ...:
   ...: dataset = foz.load_zoo_dataset("quickstart", max_samples=1)
   ...: dataset.name = "test"
   ...: dataset.persistent = True
   ...:
   ...: dataset.annotate("test_key", label_field="ground_truth", project_name="test")
Dataset already downloaded
Loading 'quickstart'
 100% |████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [19.8ms elapsed, 0s remaining, 52.4 samples/s]
Dataset 'quickstart-1' created
Found existing field 'ground_truth' with multiple types ['detections', 'instances']. Only the 'detections' will be annotated
Please enter your login credentials.
You can avoid this in the future by setting your `FIFTYONE_CVAT_USERNAME` and `FIFTYONE_CVAT_PASSWORD` environment variables
Username: kewenzhu
Password:
Computing metadata...
 100% |████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [3.6s elapsed, 0s remaining, 0.3 samples/s]
Uploading samples to CVAT...
Out[4]: <fiftyone.utils.cvat.CVATAnnotationResults at 0x1dfee573cd0>

In [5]:

Go to another terminal and run:

In [1]: import fiftyone as fo
   ...:
   ...: dataset = fo.load_dataset("test")
   ...: results = dataset.load_annotation_results("test_key")
   ...: assert results is not None

In [2]: results
Out[2]: <fiftyone.utils.cvat.CVATAnnotationResults at 0x19ecfcae5e0>

In [3]:

It successfully get the annotations, and no error any more, Thank you brimooor

Hi brimoor, my issue was resolved after upgrading fiftyone -> v0.19 following your advice, thank you

In [6]: dataset.load_annotations(anno_key)
Please enter your login credentials.
You can avoid this in the future by setting your `FIFTYONE_CVAT_USERNAME` and `FIFTYONE_CVAT_PASSWORD` environment variables
Username: ******
Password: 
Downloading labels from CVAT...
Download complete
Found unexpected labels of type 'polylines' when loading annotations for field 'None'.
Please enter a new or compatible existing field name in which to store these annotations, or empty to skip: field_x
Loading labels for field 'field_x'...
 100% |████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [6.5ms elapsed, 0s remaining, 308.4 samples/s]

voxel51 / fiftyone