visDrone.yaml download script not working

TobiasSchotter commented 2 years ago

Search before asking

[X] I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

I am trying to execute this script from the visDrone.yaml but I get following error:

from utils.general import download, os, Path, yaml

def visdrone2yolo(dir):
    from PIL import Image
    from tqdm import tqdm

    def convert_box(size, box):
        # Convert VisDrone box to YOLO xywh box
        dw = 1. / size[0]
        dh = 1. / size[1]
        return (box[0] + box[2] / 2) * dw, (box[1] + box[3] / 2) * dh, box[2] * dw, box[3] * dh

    (dir / 'labels').mkdir(parents=True, exist_ok=True)  # make labels directory
    pbar = tqdm((dir / 'annotations').glob('*.txt'), desc=f'Converting {dir}')
    for f in pbar:
        img_size = Image.open((dir / 'images' / f.name).with_suffix('.jpg')).size
        lines = []
        with open(f, 'r') as file:  # read annotation.txt
            for row in [x.split(',') for x in file.read().strip().splitlines()]:
                if row[4] == '0':  # VisDrone 'ignored regions' class 0
                    continue
                cls = int(row[5]) - 1
                box = convert_box(img_size, tuple(map(int, row[:4])))
                lines.append(f"{cls} {' '.join(f'{x:.6f}' for x in box)}\n")
                with open(str(f).replace(os.sep + 'annotations' + os.sep, os.sep + 'labels' + os.sep), 'w') as fl:
                    fl.writelines(lines)  # write label.txt

# Download
dir = Path(yaml['dataset'])  # dataset root dir
urls = ['https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-train.zip',
        'https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-val.zip',
        'https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-test-dev.zip',
        'https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-test-challenge.zip']
download(urls, dir=dir, curl=True, threads=4)

# Convert
for d in 'VisDrone2019-DET-train', 'VisDrone2019-DET-val', 'VisDrone2019-DET-test-dev':
    visdrone2yolo(dir / d)  # convert VisDrone annotations to YOLO labels

And if I remove the yaml part I get:

Does this work for anyone?

Additional

No response

glenn-jocher commented 2 years ago

@TobiasSchotter 👋 hi, thanks for letting us know about this possible problem with YOLOv5 🚀. Your issue is not reproducible. VisDrone autodownload and training works correctly, I just tested now myself. If you have internet connection issues that is clearly outside the scope of our support.

We've created a few short guidelines below to help users provide what we need in order to start investigating a possible problem.

How to create a Minimal, Reproducible Example

When asking a question, people will be better able to provide help if you provide code that they can easily understand and use to reproduce the problem. This is referred to by community members as creating a minimum reproducible example. Your code that reproduces the problem should be:

✅ Minimal – Use as little code as possible to produce the problem
✅ Complete – Provide all parts someone else needs to reproduce the problem
✅ Reproducible – Test the code you're about to provide to make sure it reproduces the problem

For Ultralytics to provide assistance your code should also be:

✅ Current – Verify that your code is up-to-date with GitHub master, and if necessary git pull or git clone a new copy to ensure your problem has not already been solved in master.
✅ Unmodified – Your problem must be reproducible using official YOLOv5 code without changes. Ultralytics does not provide support for custom code ⚠️.

If you believe your problem meets all the above criteria, please close this issue and raise a new one using the 🐛 Bug Report template with a minimum reproducible example to help us better understand and diagnose your problem.

Thank you! 😃

TobiasSchotter commented 2 years ago

@glenn-jocher Thank you for your quick reponse but doing the exact same thing as you gives me the following errors:

glenn-jocher commented 2 years ago

@TobiasSchotter thanks for the screenshot. I wonder if this is Windows related. I'll try to test.

glenn-jocher commented 2 years ago

@TobiasSchotter I'm able to reproduce the same error on VOC and VisDrone on Windows in https://github.com/glenn-jocher/yolov5/runs/6613528206

glenn-jocher commented 2 years ago

@TobiasSchotter good news 😃! Your original issue may now be fixed ✅ in PR #7999. This updates our cURL download script to use double quotes which resolves Windows autodownload issues for all datasets.

To receive this update:

Git – git pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
PyTorch Hub – Force-reload model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
Notebooks – View updated notebooks
Docker – sudo docker pull ultralytics/yolov5:latest to update your image

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!

TobiasSchotter commented 2 years ago

@glenn-jocher Thank you for dealing with this on such short notice. Manuelly downloading the data worked fine for me as well.

But just for confirmation I tried it the above way after your fix and it works. Cheers 😃

glenn-jocher commented 2 years ago

@TobiasSchotter great, thanks for the confirmation. I think we'll add more dataset trainings to the CI to verify this functionality automatically in the future.

ultralytics / yolov5