Can't export a task with images having same filenames

[saraAlsiyat1]

I'm trying to export my dataset as YOLO 1.1 but i"m getting the below error. I tried to export/ dump to different formats however all of them give me the below error except Cvat for images 1.1.

Number of jobs per task: 1 Number of frames: 1915

Please this is urgent I need to have my annotated dataset ASAP.

Error: Request failed with status code 500. "AssertionError: Image info differs for item 0_0/ln

Thank you!

[zhiltsov-max]

Hello, could you provide information described in the issue template, especially the server logs? It would be very helpful.

[saraAlsiyat1]

@zhiltsov-max Sure HYG

My actions before raising this issue

• [ ] Read/searched the docs : Yes
• [ ] Searched past issues: Yes

Expected Behaviour

I have created many other tasks and all of them worked just fine with no issues regarding the exporting matter.

Current Behaviour

I have created the task then upload 1915 images then annotated them using a track priority, nothing special. Please note: I did the same with a larger number of images, and everything went successfully.

Possible Solution

I have no idea :( I need your help.

Steps to Reproduce (for bugs)

I'm not sure what you meant here.

Context

I'm doing a research project in the context of COVID 19, so this request is urgent. Redoing the whole annotation thing would cost me a lot of time since I'm working alone in this project.

Your Environment

• Git hash commit (git log -1): I don't have any commit
• Docker version docker version (e.g. Docker 17.0.05): 19.03.12
• Are you using Docker Swarm or Kubernetes? Swarm
• Operating System and version (e.g. Linux, Windows, MacOS): Linux
• Code example or link to GitHub repo or gist to reproduce problem: I only used the GUI.

[zhiltsov-max]

Please, provide the server logs. You can obtain them by typing docker logs cvat in the terminal.

[saraAlsiyat1]

Please, provide the server logs. You can obtain them by typing docker logs cvat in the terminal.

@zhiltsov-max Sure PFA

CvatDockerlogs.txt

[zhiltsov-max]

Does the task have different images with same filenames, possibly with different extensions?

[saraAlsiyat1]

Does the task have different images with same filenames, possibly with different extensions?

Yes some of them have same file names but with different extensions.

[zhiltsov-max]

OK, so, apparently, this is the cause. Currently, this scenario is not supported in any format, except CVAT for * formats.

[saraAlsiyat1]

OK, so, apparently, this is the cause. Currently, this scenario is not supported in any format, except CVAT for * formats.

That means there's no way to export them as any format other than Cvat for images 1.1 right?

Is there any way to rename them or delete the images with the same filenames?

[zhiltsov-max]

Right. Unfortunately, CVAT does not support image addition or removal. In your case you what can do is:

1. Export the task in CVAT format
2. Remove extra items from the file
3. Create a task without images with repeated names
4. Import annotations
5. Export a dataset

[Icxa]

I also face the Error 500 ("Assertion Error") . I am trying to upload the annotation file that I generated in CVAT. I downlaoded the file in CVAT for Video format and after that I would like to re-upload it for another task. I encounter an error there. I am attaching the log file. cvat_logs.txt

[zhiltsov-max]

@Icxa, try to upload annotations using the task overview page


[Icxa]

I tried doing so. It gives me the same error.

[zhiltsov-max]

@Icxa, could you provide task properties, and, possibly, annotations to test?

[Mhs-220]

@zhiltsov-max i have same issue when im trying to export for coco. the error is:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/rq/worker.py", line 812, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python3.5/dist-packages/rq/job.py", line 588, in perform
    self._result = self._execute()
  File "/usr/local/lib/python3.5/dist-packages/rq/job.py", line 594, in _execute
    return self.func(*self.args, **self.kwargs)
  File "/home/django/cvat/apps/dataset_manager/views.py", line 84, in export_task_as_dataset
    return export_task(task_id, dst_format, server_url=server_url, save_images=True)
  File "/home/django/cvat/apps/dataset_manager/views.py", line 62, in export_task
    server_url=server_url, save_images=save_images)
  File "/home/django/cvat/apps/dataset_manager/task.py", line 723, in export_task
    save_images=save_images)
  File "/home/django/cvat/apps/dataset_manager/task.py", line 619, in export
    exporter(dst_file, task_data, **options)
  File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 36, in __call__
    f_or_cls(*args, **kwargs)
  File "/home/django/cvat/apps/dataset_manager/formats/coco.py", line 19, in _export
    extractor = Dataset.from_extractors(extractor) # apply lazy transforms
  File "/home/django/datumaro/datumaro/components/project.py", line 341, in from_extractors
    item = cls._merge_items(existing_item, item, path=path)
  File "/home/django/datumaro/datumaro/components/project.py", line 425, in _merge_items
    image=cls._merge_images(existing_item, current_item),
  File "/home/django/datumaro/datumaro/components/project.py", line 443, in _merge_images
    assert existing_item.image._size == current_item.image._size, "Image info differs for item '%s'" % existing_item.id
AssertionError: Image info differs for item 'working/flickr/pear/000025'

when I tried to export as cvat for images (as you said) it takes some times and give me another error. the error is:

2020-08-25 06:10:23,627 DEBG 'rqworker_default_1' stderr output:
ERROR:cvat.server.task_37:[cvat.apps.dataset_manager.views @ export_task]: exception occurred
Traceback (most recent call last):
  File "/home/django/cvat/apps/dataset_manager/views.py", line 62, in export_task
    server_url=server_url, save_images=save_images)
  File "/home/django/cvat/apps/dataset_manager/task.py", line 723, in export_task
    save_images=save_images)
  File "/home/django/cvat/apps/dataset_manager/task.py", line 619, in export
    exporter(dst_file, task_data, **options)
  File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 36, in __call__
    f_or_cls(*args, **kwargs)
  File "/home/django/cvat/apps/dataset_manager/formats/cvat.py", line 544, in _export_images
    anno_callback=dump_as_cvat_annotation, save_images=save_images)
  File "/home/django/cvat/apps/dataset_manager/formats/cvat.py", line 529, in _export
    frame_data, jpeg_quality=100, create_dir=True)
  File "/home/django/datumaro/datumaro/util/image.py", line 74, in save_image
    cv2.imwrite(path, image, params=params)
cv2.error: OpenCV(4.2.0) /io/opencv/modules/imgcodecs/src/loadsave.cpp:661: error: (-2:Unspecified error) could not find a writer for the specified extension in function 'imwrite_'

Do you have any idea about this situation?

[zhiltsov-max]

@Mhs-220, the first problem is discussed above. The second one looks like there are images, which file names include . in their names, is it possible?

[Mhs-220]

@zhiltsov-max as far as I saw, there were no files that have . in their names (If you didn't mean extension by that). but something I found during investigates, there was a folder contains . in its name, could it be it? and one more thing, the error is happening on all tasks, not only the ones which are in that folder. is it logical?

[zhiltsov-max]

@Mhs-220, yes, a point in any position of image path (except extension) would break it too, it should be fixed. Thanks for reporting! It should happen if there is an image with such a path in a task, no other known causes yet.

[Mhs-220]

@zhiltsov-max thank you so much for helping. so some other things,

1. how could I fix this issue without any data loss? Any Idea?
2. I'm a Backend developer (Django, Flask, Spring, etc.) how can I contribute to this project? I think it doesn't have this kinda bugs.

[zhiltsov-max]

@Mhs-220, if you can change source code, then the simplest thing to try is to change these lines: https://github.com/opencv/cvat/blob/develop/cvat/apps/dataset_manager/formats/cvat.py#L535 . You would need to change some exporting code, rebuild the Docker image and restart services with docker-compose. No data loss is expected.

You can find the contributing guide and tips for local development here.

[Mhs-220]

@zhiltsov-max Ok, after spending some time debugging the error, I found out another thing, one of my files has .gif extension and make that error. after changing some code to remove . from the path, I added some other code to change the extension to png in case of the gif. It seems to work (at least export completed without any error)

[pablodz]

I get this issue when trying to DUMP to Pascal

There is the list of files that I upload to CVAT https://pastebin.com/QXmMH9SZ, and, yes I have some files with dots in the filenames. Also, gif extensions.

I was using the website to do the labeling of 1k images :c How can I solve it without losing everything. Thanks in advance.

[zhiltsov-max]

@ZurMaD, you can use CVAT for * formats, which are not affected by this problem.

[shaojun]

so looking forward to support the file name renaming when upload files at creating task, current behave makes multiple tasks project almost un-workable as different tasks created from users have same file name is quite possible. thanks.

[dummyindex]

4207

Use case: several experts annotate the same set of imaging results with other annotation tools, exported to COCO format. The COCO file contains "images" field holding items with different id but same filenames. CVAT fails to import datasets in this case and complains about the same filenames.

Current solution: 1) save multiple copies of original data (waste of disk space) 2) split one COCO json file into different COCO json files and create one project for each expert?

[zhiltsov-max]

Related (for import): https://github.com/opencv/cvat/issues/7523

Also, discussed a way to resolve it for export there.