How to train using npy format

[Nido051]

Search before asking

• [X] I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

I would like to perform multi-channel image processing. Is it possible to train by inputting a format such as npy or npz instead of an image format? If someone can do it, I would appreciate it if you could tell me the code. thank you

Additional

No response

[github-actions[bot]]

👋 Hello @Nido051, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.8.0 with all requirements.txt installed including PyTorch>=1.8. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Introducing YOLOv8 🚀

We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀!

Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects.

Check out our YOLOv8 Docs for details and get started with:

pip install ultralytics

[glenn-jocher]

@Nido051 yes, it is possible to train YOLOv5 using the npy or npz format instead of the image format. The code for training with npy or npz files will depend on the structure and content of your data.

To train YOLOv5 with npy or npz files, you will need to modify the data loading code in the datasets module of the YOLOv5 repository. Specifically, you will need to create a custom dataset class that extends the torch.utils.data.Dataset class and implement the necessary methods to load and transform your npy or npz data.

Here is a basic outline of the steps you can follow:

1. Create a custom dataset class that extends torch.utils.data.Dataset and implement the necessary methods such as __len__, __getitem__, and any additional methods required for loading and preprocessing your npy or npz data.

2. In the __getitem__ method of your dataset class, write the code to load the npy or npz files and return the necessary inputs (image data) and targets (bounding box annotations or class labels) required for training.

3. Modify the train.py script in the YOLOv5 repository to use your custom dataset class instead of the default COCO dataset class. You will need to update the datasets import statement and modify the dataset instantiation to use your custom dataset class.

4. Make sure to also update any other relevant parts of the training script, such as data augmentation options and hyperparameters, to suit your needs.

Remember to refer to the documentation and code of the YOLOv5 repository for more detailed information on dataset creation and training customization. Additionally, you can consult the YOLOv5 community for further assistance if needed.

Good luck with your YOLOv5 training using npy or npz files!

[github-actions[bot]]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

[K011-17]

Following this as well. Is it possible to train an existing model without creating a custom dataset class? So far, I have tried the following strategies.

1.In the data.yaml file, update the train and val paths to point to your .npy files instead of image directories.

2.In the datasets.py file, update the LoadImagesAndLabels class. Replace the image loading logic with code to load .npy files using np.load().

3.Modify the load_image() function in the utils\dataloaders..py file to handle .npy files appropriately. You can use np.load() to load the .npy file and convert it to the required tensor format.

I get the following error: The label and path are in the correct format for YOLOv5.

Traceback (most recent call last): File "/home/○○○○○/yolov5-master/train.py", line 647, in main(opt) File "/home/○○○○○/yolov5-master/train.py", line 536, in main train(opt.hyp, opt, device, callbacks) File "/home/○○○○○/yolov5-master/train.py", line 195, in train train_loader, dataset = create_dataloader(train_path, File "/home/○○○○○/yolov5-master/utils/dataloaders.py", line 124, in create_dataloader dataset = LoadImagesAndLabels( File "/home/○○○○○/yolov5-master/utils/dataloaders.py", line 505, in init assert nf > 0 or not augment, f'{prefix}No labels found in {cache_path}, can not start training. {HELP_URL}' AssertionError: train: No labels found in /home/○○○○○/yolov5-master/data/○○○○○/train/labels.cache, can not start training.

[glenn-jocher]

@K011-17 Thanks for sharing your approach. It seems like you're on the right track with the steps you've taken to train YOLOv5 using npy files. However, the error you're encountering suggests that the labels may not be loading correctly.

To address this issue:

1. Verify that your npy files contain the necessary label information in a format compatible with YOLOv5.
2. Check the paths specified in your data.yaml file and ensure they point to the correct npy files containing both images and labels.
3. Confirm that the modifications in datasets.py and dataloaders.py are correctly handling the loading of npy files and their associated labels.

If you're still encountering issues, I recommend double-checking the label format within your npy files and ensuring that the file paths and loading logic are appropriately configured to handle your npy data.

If you continue to face challenges, please feel free to provide additional details about your npy dataset structure, and I am happy to assist further.

[glenn-jocher]

@K011-17 Thanks for providing additional details about your dataset and the issues you're encountering. It seems like the warnings and errors you're seeing are related to the handling of npy files and labels within the YOLOv5 training process.

To address the multi-channel input and the error messages:

1. To train with multi-channel npy data, you will need to update the data loading code to handle 4-channel data appropriately. You may need to modify the data loading logic in the datasets.py and dataloaders.py files to ensure that the 4-channel data is processed correctly.

2. Regarding the warnings about corrupt image/label and the message regarding the labels.cache, it seems like there might be issues with how the npy files are being loaded and processed as images and labels. You may need to modify the loading logic to handle npy files as both images and labels, and ensure that the label cache is being properly generated and utilized.

It's important to note that handling multi-channel input and custom dataset formats may require more extensive modifications to the YOLOv5 codebase. I'd recommend thoroughly reviewing and modifying the relevant data loading and processing logic to accommodate your specific dataset structure and requirements.

Additionally, as you continue troubleshooting, you may find it helpful to consult the YOLOv5 documentation and the YOLOv5 community for further guidance and support in customizing the dataset loading process.

I hope this guidance helps you move forward with training YOLOv5 using your multi-channel npy dataset. If you have further questions or encounter specific code-related issues, feel free to share them here, and I'll be glad to assist you further.

[K011-17]

Thank you for your answer. The yaml file I am using is like below. There is npy file in the images folder.

train : '/home/○○○○○/yolov5-master/data/mydata/train/images/' val: '/home/○○○○○/yolov5-master/data/mydata/val/images/'

nc : ○ names: ['classname1', 'classname2', ... , classname○']

When I checked the npy data using the numpy shape function, it was 4-channel data (480, 640, 4). I would like to train with more channels than this.

In addition to the error message, messages like the one below will be displayed for the total number of images. train: WARNING ⚠️ /home/○○○○○/yolov5-master/data/mydata/train/images/:.npy: ignoring corrupt image/label: cannot identify image file '/home/○○○○○/yolov5master/data/mydata/train/images/.npy'

At the end you will also see a message like this train: WARNING ⚠️ No labels found in /home/○○○○○/yolov5-master/data/mydata/train/labels.cache. train: WARNING ⚠️ Cache directory /home/○○○○○/yolov5-master/data/mydata/train is not writeable: [Errno 2] No such file or directory: '/home/○○○○○/yolov5-master/data/mydata/train/labels.cache.npy' -> 　　　　　　　　　　　　'/home/○○○○○/yolov5-master/data/mydata/train/labels.cache'

I think this part is the cause, but I can't understand why this message is displayed or which function is causing this error. Sorry for the long post. Thank you for your reply.

[glenn-jocher]

@K011-17 Thank you for providing the additional details about your setup and the error messages you are seeing. It looks like you are encountering issues related to the handling of the npy file format as multi-channel data and the processing of labels during training.

To address the challenges you are facing:

1. Handling Multi-Channel Data: To train with npy files containing more than 4 channels (such as the 4-channel data you mentioned), you will need to customize the data loading logic within the YOLOv5 codebase to correctly handle and preprocess the multi-channel input data. You may need to modify the data loading and transformation logic to accommodate your specific data format.

2. Error Messages: The warning messages related to "ignoring corrupt image/label" and the absence of labels in the cache directory indicate that the data loading and processing steps are encountering issues, possibly due to the handling of npy files as images and labels. It is important to ensure that the npy files are being correctly loaded, and the label information is being processed appropriately during training.

Identifying the Source of Errors: To address the warning messages and the cause of the errors, you will need to perform detailed debugging and inspection of the relevant functions within the YOLOv5 codebase. I recommend examining the data loading, processing, and caching functions to understand where the issues are occurring and what modifications are needed.

Customizing Data Loading: Given the complexity of training with multi-channel npy files and custom dataset formats, you may need to delve into the details of the datasets.py, dataloaders.py, and related files to customize the data loading and preprocessing pipeline to accommodate your specific dataset structure and requirements.

I understand that troubleshooting code-related issues can be challenging, but carefully reviewing the relevant sections of the YOLOv5 codebase, consulting the YOLOv5 documentation, and seeking assistance from the YOLOv5 community can provide valuable insights and solutions to address the data loading and training challenges you are encountering.

I hope this guidance helps you in your efforts to train YOLOv5 with your multi-channel npy dataset. If you have further questions or need further support with specific code-related issues, feel free to share them here, and I'll be happy to assist you further.

[K011-17]

I was able to study successfully in npy format. The main changes that were important are listed below.

1. When loading labels with dataloaders.py, it seems that you had to comment out lines 488 to 492 and pass them directly to the cache_labels function.
2. Change cv2.imread to np.load in various places
3. Change the number of channels in yolo.py and plots.py according to the data

I hope that my attempt will be helpful to those who are trying to do something similar. Mr. Glenn, thank you very much for your generous support, including answering my questions many times.

[glenn-jocher]

@K011-17 thank you for sharing your successful experience with training YOLOv5 using the npy format and outlining the crucial changes you made to achieve it. Your insights are valuable and will undoubtedly benefit others attempting a similar approach.

Your contributions to the YOLOv5 community are appreciated, and I'm glad to hear about your successful study in npy format. Your efforts and willingness to share your findings will help others navigate similar challenges and achieve success with custom dataset formats.

I'm here to help, and I truly appreciate your kind words. It's always a pleasure to support and engage with the YOLOv5 community. If you have any more questions or need further assistance in the future, feel free to reach out.

Thank you once again, and best of luck with your continued work with YOLOv5 and custom dataset formats!

[aalling93]

I was able to study successfully in npy format. The main changes that were important are listed below.

1. When loading labels with dataloaders.py, it seems that you had to comment out lines 488 to 492 and pass them directly to the cache_labels function.
2. Change cv2.imread to np.load in various places
3. Change the number of channels in yolo.py and plots.py according to the data

I hope that my attempt will be helpful to those who are trying to do something similar. Mr. Glenn, thank you very much for your generous support, including answering my questions many times.

Hi K011-17, have you forked your version, or can you share it more explicitly. I have tried to use npy files (similar to what you did, but also more steps), but I can not get it to work. Thanks

[glenn-jocher]

Hi @aalling93, thank you for sharing your successful approach with training YOLOv5 using npy format. Your detailed list of important changes will undoubtedly help others attempting a similar workflow.

I understand your request to access the explicit changes and modifications you've made. While I do not have access to the specific repository or changes you've implemented, I encourage you to consider sharing your modifications, either through a public fork of the YOLOv5 repository or by providing detailed documentation of your changes in a separate repository or document. This can benefit others in the community who are looking to work with npy files in YOLOv5.

Your willingness to share your successful experience and modifications is greatly appreciated. It's an excellent contribution to the YOLOv5 community, and I believe it will be beneficial to many developers seeking to work with npy format.

Thank you for your valuable contributions, and I hope your efforts inspire others as they navigate similar challenges.

Best regards.

[K011-17]

Hello! Here are the changes I forgot to make: ・Added ch: option to YOLOv5s.yaml ・Changed the verify_image_label function in dataloaders.py as follows

def verify_image_label(args):
    # Verify one image-label pair
    im_file, lb_file, prefix = args
    nm, nf, ne, nc, msg, segments = 0, 0, 0, 0, '', []  # number (missing, found, empty, corrupt), message, segments
    try:
        # verify images
        im = np.load(im_file)
        #  im.verify()  # PIL verify
        '''shape = exif_size(im)'''    # image size
        shape = im.shape[:2]
        assert (shape[0] > 9) & (shape[1] > 9), f'image size {shape} <10 pixels'
        '''
        assert im.format.lower() in IMG_FORMATS, f'invalid image format {im.format}'
        if im.format.lower() in ('jpg', 'jpeg'):
            with open(im_file, 'rb') as f:
                f.seek(-2, 2)
                if f.read() != b'\xff\xd9':  # corrupt JPEG
                    ImageOps.exif_transpose(Image.open(im_file)).save(im_file, 'JPEG', subsampling=0, quality=100)
                    msg = f'{prefix}WARNING ⚠️ {im_file}: corrupt JPEG restored and saved'
        '''

Since we prioritized operation, we commented out the preprocessing part of dataloaders.py and implemented it. How about this?

[glenn-jocher]

Hello @K011-17,

It looks like you've made some thoughtful modifications to the YOLOv5 codebase to accommodate npy files. Adding a ch: option to the YOLOv5s.yaml allows for specifying the number of channels directly, which is a smart way to handle multi-channel data.

Your changes to the verify_image_label function in dataloaders.py are also in line with the need to process npy files. By loading the image with np.load and commenting out the preprocessing part that is specific to traditional image formats, you've tailored the function to work with npy files.

These modifications should indeed help others who are looking to work with npy files in YOLOv5. It's great to see community members contributing by sharing their solutions.

Remember that by sharing your code changes, you're not only helping others but also potentially receiving feedback that could further improve your approach. If you're open to it, consider creating a pull request to the main YOLOv5 repository with your changes or maintaining a fork that others can reference.

Thank you for your contribution, and I'm glad to see that you've found a working solution for your use case. If you have any more questions or need further assistance, feel free to reach out.

Best regards.

[DeepSkyBlue123]

Hello @K011-17, Do you mind providing your dataloader.py file? Thank you

[K011-17]

Sorry for the late reply! dataloader.py looks like the attached text file. Please copy it to a python file and use it. dataloader.txt However, since the augnmentation is commented out and turned off, please customize it according to your own data. If you have any problems, please feel free to ask!

[glenn-jocher]

Hello @K011-17,

Thank you for your patience! I appreciate your willingness to share your customized dataloader.py file. It's great to see community members helping each other out.

To ensure that your modifications work seamlessly, here are a few additional tips:

1. Customization: As you mentioned, the augmentation is turned off. Depending on your dataset and training requirements, you might want to re-enable and customize the augmentation settings to improve your model's performance.

2. Testing: Make sure to thoroughly test your modified dataloader with a subset of your data to ensure that it correctly handles the npy files and integrates well with the rest of the YOLOv5 pipeline.

3. Documentation: Document any changes you make to the code, especially if you plan to share it with others. This will help others understand your modifications and adapt them to their own needs.

If you encounter any issues or have further questions, feel free to ask. The YOLOv5 community is here to support you!

Best of luck with your training, and thank you for contributing to the community!

[CohenCalvert]

Hello, I have also been trying to setup my data using a .yaml file that reads to an .npz file. (this is my first time ever using Yolo or Yaml files.

i am trying to make my yaml to a MMNIST data set with a train and valadating .npz

however i keep getting: yaml.scanner.ScannerError: mapping values are not allowed here in "dataset.yaml", line 3, column 6

and im not really sure why, since its just a space

`# dataset.yaml

train: "data/train"# Path to the training dataset file (npz format) val: "valid" # Path to the validation dataset file (npz format)

nc: 10 # Number of classes in the dataset

names: 0: "0" 1: "1" 2: "2" 3: "3" 4: "4" 5: "5" 6: "6" 7: "7" 8: "8" 9: "9"`

[pderrenger]

@CohenCalvert the error you're encountering is likely due to incorrect indentation or formatting in your YAML file. Ensure there is a space after the colon in your paths, like this:

train: "data/train"  # Path to the training dataset file (npz format)
val: "valid"  # Path to the validation dataset file (npz format)

Double-check the entire file for consistent indentation and spacing. If the issue persists, try validating your YAML file with an online YAML validator.

[CohenCalvert]

I realized that there was invisible blank characters within my file for some reason. deleting and re-creating my file seemed to fix it

[pderrenger]

Glad to hear you resolved the issue! If you have any more questions or need further assistance, feel free to ask.

[pderrenger]

Glad to hear you resolved the issue by recreating the file! If you have any more questions or need further assistance, feel free to ask.