Support datasets exported by fiftyone on windows platform without requiring modification of .yaml pathnames

liamw9534 commented 3 months ago

Search before asking

[X] I have searched the HUB issues and found no similar feature requests.

Description

It is necessary manually modify the .yaml so that any backslash present in path names (as used under windows) is converted to a forward slash for the dataset zip file to be accepted. It would be nice if such pathname translations were handled automatically by the hub dataset importer.

Use case

Seamless integration of 3rd party tools and OS environments with ultralytics hub.

Additional

No response

github-actions[bot] commented 3 months ago

👋 Hello @liamw9534, thank you for raising an issue about Ultralytics HUB 🚀! Please visit our HUB Docs to learn more:

Quickstart. Start training and deploying YOLO models with HUB in seconds.
Datasets: Preparing and Uploading. Learn how to prepare and upload your datasets to HUB in YOLO format.
Projects: Creating and Managing. Group your models into projects for improved organization.
Models: Training and Exporting. Train YOLOv5 and YOLOv8 models on your custom datasets and export them to various formats for deployment.
Integrations. Explore different integration options for your trained models, such as TensorFlow, ONNX, OpenVINO, CoreML, and PaddlePaddle.
Ultralytics HUB App. Learn about the Ultralytics App for iOS and Android, which allows you to run models directly on your mobile device.
- iOS. Learn about YOLO CoreML models accelerated on Apple's Neural Engine on iPhones and iPads.
- Android. Explore TFLite acceleration on mobile devices.
Inference API. Understand how to use the Inference API for running your trained models in the cloud to generate predictions.

If this is a 🐛 Bug Report, please provide screenshots and steps to reproduce your problem to help us get started working on a fix.

If this is a ❓ Question, please provide as much information as possible, including dataset, model, environment details etc. so that we might provide the most helpful response.

We try to respond to all issues as promptly as possible. Thank you for your patience!

pderrenger commented 3 months ago

Hello,

Thank you for bringing this to our attention and for your detailed description! We understand how important seamless integration is for a smooth workflow.

To address your concern, we recommend ensuring that the issue is reproducible with the latest versions of the Ultralytics HUB and related packages. If the problem persists, we can consider implementing an automatic path conversion feature in the dataset importer to handle backslashes in Windows paths.

In the meantime, as a workaround, you can use a small script to convert backslashes to forward slashes in your .yaml file before importing it. Here's a quick example in Python:

import os

def convert_backslashes_to_forward_slashes(file_path):
    with open(file_path, 'r') as file:
        content = file.read()

    content = content.replace('\\', '/')

    with open(file_path, 'w') as file:
        file.write(content)

# Replace 'your_yaml_file.yaml' with the path to your .yaml file
convert_backslashes_to_forward_slashes('your_yaml_file.yaml')

This script reads your .yaml file, replaces all backslashes with forward slashes, and writes the updated content back to the file.

We appreciate your feedback and will consider it for future updates to improve the user experience. If you have any further questions or run into any other issues, please don't hesitate to let us know.

liamw9534 commented 3 months ago

That's very helpful. I did already put a similar script into my fiftyone export pipeline to avoid it happening in future.

It's worth pointing out that my zipped dataset passed the checks when running ultralytics.hub.check_dataset() even with windows paths in the .yaml. But that's not surprising because the check is running on the same Windows machine.

On Tue, 6 Aug 2024, 18:32 Paula Derrenger, @.***> wrote:

Hello,

Thank you for bringing this to our attention and for your detailed description! We understand how important seamless integration is for a smooth workflow.

To address your concern, we recommend ensuring that the issue is reproducible with the latest versions of the Ultralytics HUB and related packages. If the problem persists, we can consider implementing an automatic path conversion feature in the dataset importer to handle backslashes in Windows paths.

In the meantime, as a workaround, you can use a small script to convert backslashes to forward slashes in your .yaml file before importing it. Here's a quick example in Python:

import os def convert_backslashes_to_forward_slashes(file_path): with open(file_path, 'r') as file: content = file.read()
content = content.replace('\\', '/')

with open(file_path, 'w') as file:
    file.write(content)
Replace 'your_yaml_file.yaml' with the path to your .yaml fileconvert_backslashes_to_forward_slashes('your_yaml_file.yaml')

This script reads your .yaml file, replaces all backslashes with forward slashes, and writes the updated content back to the file.

We appreciate your feedback and will consider it for future updates to improve the user experience. If you have any further questions or run into any other issues, please don't hesitate to let us know.

— Reply to this email directly, view it on GitHub https://github.com/ultralytics/hub/issues/795#issuecomment-2271797814, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABTRST347ALZ5LN7TG47Z7DZQECDNAVCNFSM6AAAAABMCCJVQ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZRG44TOOBRGQ . You are receiving this because you were mentioned.Message ID: @.***>

pderrenger commented 3 months ago

@liamw9534 hi Paula,

Thank you for your proactive approach and for integrating a similar script into your FiftyOne export pipeline. It's great to hear that you're already taking steps to streamline your workflow! 😊

Regarding the ultralytics.hub.check_dataset() function, your observation is indeed correct. The check runs on the local machine, so it might not catch issues related to path formats that could arise when transferring datasets between different operating systems. This is a valuable insight, and we appreciate you sharing it.

To further enhance the robustness of the dataset validation process, we will consider your feedback for future improvements. Ensuring compatibility across different environments is crucial, and your input helps us move in that direction.

If you encounter any other issues or have more suggestions, please feel free to share. We're here to help and continuously improve the Ultralytics HUB experience for everyone.

ultralytics / hub