cvat-ai / cvat

Annotate better with CVAT, the industry-leading data engine for machine learning. Used and trusted by teams at any scale, for data of any scale.
https://cvat.ai
MIT License
12.24k stars 2.95k forks source link

Export of backup isn't supported for tasks with data from a mounted share #4989

Open thomasm2film opened 1 year ago

thomasm2film commented 1 year ago

Can't post to the "Discussions" section any more? That would probably have been more appropriate for this issue / question:

When trying to Export or backup a project, the following error occurs; Could not backup the project #xx Error: Request failed with status code 500. "Exception: The task cannot be exported because it does not contain any raw data\n"

This error does not occur when exporting a TASK

What could this be?

This error is new since upgrading to 2.2.0 from 2.1 - we also tried the very latest build with same result.

Thanks

bsekachev commented 1 year ago

Hi @thomasm2film

The reason for that may be because in the project you have tasks that were created with the shared storage. image

If so, for sure, it's a bug. Can you submit or refute the idea?

thomasm2film commented 1 year ago

Hi @bsekachev Sorry for the late reply, but this is exactly the case. Used to work beautifully, but since 2.1, the override file for shares has also changed a bit from

version: '3.3'

services:
  cvat_server:
    environment:
      CVAT_SHARE_URL: 'Mounted from /mnt/share host directory'
    volumes:
      - cvat_share:/home/django/share:ro

volumes:
  cvat_share:
    driver_opts:
      type: none
      device: /mnt/share
      o: bind

TO

version: '3.3'

services:
  cvat_server:
    volumes:
      - cvat_share:/home/django/share:ro
  cvat_worker_default:
    volumes:
      - cvat_share:/home/django/share:ro

volumes:
  cvat_share:
    driver_opts:
      type: none
      device: /mnt/share
      o: bind

Let me know if I should provide any more details - thanks!

thomasm2film commented 1 year ago

Any help would be much appreciated - let me know if I can supply further details (and what is needed from us). Thanks

bsekachev commented 1 year ago

@thomasm2film

Let me explain.

Error: Request failed with status code 500. "Exception: The task cannot be exported because it does not contain any raw data\n"

The error occurs because you are trying to backup a project which has tasks created from a shared storage.

This is a bug. I believe CVAT should ignore such tasks and just show a notification about some tasks have not been kept.

CVAT does not back up tasks created from the shared storage if raw data was not copied to CVAT instance from the storage. If you want to copy the data, you can click the corresponding checkbox when creating a task: image In this case you will be able to backup tasks which were created from the shared storage.

This is not a bug. The reason of such solution is that data on the shared storage, in general, can be updated when a user tries to make a backup. So, the raw task data may not exist anymore, they can be changed, etc. So, this is the feature restriction.

If not all of your tasks have been created from the shared storage, in this file cvat/apps/engine/backup.py In class ProjectExporter, try to change this:

    def _write_tasks(self, zip_object):
        for idx, db_task in enumerate(self._db_project.tasks.all().order_by('id')):
            TaskExporter(db_task.id, self._version).export_to(zip_object, self.TASKNAME_TEMPLATE.format(idx))

By this:

    def _write_tasks(self, zip_object):
        for idx, db_task in enumerate(self._db_project.tasks.all().order_by('id')):
            try:
                TaskExporter(db_task.id, self._version).export_to(zip_object, self.TASKNAME_TEMPLATE.format(idx))
            except:
                pass

And rebuild the CVAT container with .dev.yml file. After that you will be able to backup the project.

bsekachev commented 1 year ago

@azhavoro

I am correct about the behaviour?

nmanovic commented 1 year ago

First of all server should not return 500 error in the case. Also probably it is better just try to create backups and generate 400 error in case of any problems.

pktiuk commented 1 year ago

Is this stil an issue?

@zhiltsov-max Why https://github.com/opencv/cvat/pull/5920 was closed?

zhiltsov-max commented 1 year ago

@pktiuk, unfortunately, I had no opportunity to continue working on that PR. Yes, the issue is still actual.