search

ultralytics / hub

Ultralytics HUB tutorials and support
https://hub.ultralytics.com
GNU Affero General Public License v3.0
138 stars 14 forks source link

Model won't train in Ultralytics Cloud, Disconnected Error #829

Closed ahumphries2 closed 2 months ago

ahumphries2 commented 2 months ago

Search before asking

HUB Component

Training

Bug

Signed up for the Ultralytics Cloud to train my models, it completely trains, starts optimizing weights and then gives me a Disconnected error. Model https://hub.ultralytics.com/models/Bdcsog5ddsVZ4GQM0WR2

image

Environment

Windows 10, MS Edge

Minimal Reproducible Example

No response

Additional

No response

sergiuwaxmann commented 2 months ago

@ahumphries2 Hello! Apologies for the inconvenience. It looks like the best.pt file is missing, reason why the Optimizing Weights step fails. This might happen when the training is resumed. Our team will investigate this issue to prevent it from happening again in the future. I refunded the account balance used for the training and kindly ask you to start the training again from scratch.

ahumphries2 commented 2 months ago

This issue is not resolved. Everytime I try to retrain the model it gets stuck and disconnected. Not sure how I'm disconnected when I'm using the Ultralytics Cloud, and I can't get access to the model until it completes the training and optimization. Please resolve or I need a complete refund of both the Ultralytics Hub Pro account and the Top Off charge. Here is the model in Ultralytics: https://hub.ultralytics.com/models/Bdcsog5ddsVZ4GQM0WR2 image

sergiuwaxmann commented 2 months ago

@ahumphries2 I forwarded your request and it should be processed soon.

Your issue reminds me of this issue: https://github.com/ultralytics/hub/issues/769. Maybe you have a similar problem (in the dataset)?

ahumphries2 commented 2 months ago

I exported the dataset from Roboflow. Are you suggesting in #769 that I change the YAML file before I load into Ultralytics?

sergiuwaxmann commented 2 months ago

@ahumphries2, did you import the Roboflow dataset following the instructions from our documentation on Integrations (https://docs.ultralytics.com/hub/integrations)? If you exported the dataset from Roboflow as a .zip file and uploaded it to Ultralytics HUB, we’ve encountered issues with the format in the past. If this was your approach, please ensure the dataset is in the correct format as outlined in our documentation (https://docs.ultralytics.com/hub/datasets#upload-dataset).

ahumphries2 commented 2 months ago

I have Roboflow API linked to Ultralytics HUB.

I have a Roboflow Segmentation annotated dataset that I want to train as an OBB. How do I do that in Ultralytics Hub?

Andy Humphries Tech Manager – South Branch O (304) 530-1100 x1292 A 587 Robert C. Byrd Industrial Park Rd Moorefield, WV 26836

@.***

From: Sergiu Waxmann @.> Sent: Thursday, September 12, 2024 1:23 PM To: ultralytics/hub @.> Cc: Andrew Humphries @.>; Mention @.> Subject: Re: [ultralytics/hub] Model won't train in Ultralytics Cloud, Disconnected Error (Issue #829)

This message has originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email.

@ahumphries2https://github.com/ahumphries2, did you import the Roboflow dataset following the instructions from our documentation on Integrations (https://docs.ultralytics.com/hub/integrationshttps://docs.ultralytics.com/hub/integrations)? If you exported the dataset from Roboflow as a .zip file and uploaded it to Ultralytics HUB, we’ve encountered issues with the format in the past. If this was your approach, please ensure the dataset is in the correct format as outlined in our documentation (https://docs.ultralytics.com/hub/datasets#upload-datasethttps://docs.ultralytics.com/hub/datasets#upload-dataset).

— Reply to this email directly, view it on GitHubhttps://github.com/ultralytics/hub/issues/829#issuecomment-2346847124, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A2AQMTMRYUVVDEXLEIU7M43ZWHEWHAVCNFSM6AAAAABN6S3XZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBWHA2DOMJSGQ. You are receiving this because you were mentioned.Message ID: @.**@.>>

glenn-jocher commented 2 months ago

Hello Andy,

To train your Roboflow Segmentation dataset as an Oriented Bounding Box (OBB) in Ultralytics HUB, you'll need to ensure your dataset is correctly formatted for OBB tasks. Here's a general approach:

  1. Dataset Preparation: Ensure your dataset annotations are compatible with OBB. This typically involves converting segmentation masks to OBB annotations. You might need to preprocess your dataset to achieve this.

  2. Integration: Since you have the Roboflow API linked, ensure your dataset is imported correctly into Ultralytics HUB. Follow the integration guidelines to avoid any format issues.

  3. Training Setup: Once your dataset is ready, you can proceed to set up your training in Ultralytics HUB. Select the appropriate model that supports OBB and configure your training parameters.

  4. Monitor and Adjust: During training, monitor the results and adjust parameters as needed to optimize performance.

If you encounter any issues, please ensure you're using the latest version of Ultralytics HUB and feel free to reach out with specific questions. 😊

Best of luck with your training!

Ultralytics Team

sergiuwaxmann commented 2 months ago

@ahumphries2 Could the issue be dataset related or should we proceed with the refund?

ahumphries2 commented 2 months ago

Please proceed with the refund, and change my account back to free from pro. Thank you.

Andy Humphries Tech Manager - South Branch

O (304) 530-1100 x1292 A 587 Robert C. Byrd Industrial Park Rd Moorefield, WV 26836

@.***

From: Sergiu Waxmann @.> Sent: Friday, September 13, 2024 5:34 AM To: ultralytics/hub @.> Cc: Andrew Humphries @.>; Mention @.> Subject: Re: [ultralytics/hub] Model won't train in Ultralytics Cloud, Disconnected Error (Issue #829)

This message has originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email.

@ahumphries2https://github.com/ahumphries2 Could the issue be dataset related or should we proceed with the refund?

- Reply to this email directly, view it on GitHubhttps://github.com/ultralytics/hub/issues/829#issuecomment-2348490054, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A2AQMTNB4CY4SVDEBKOIKR3ZWKWPNAVCNFSM6AAAAABN6S3XZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBYGQ4TAMBVGQ. You are receiving this because you were mentioned.Message ID: @.**@.>>

sergiuwaxmann commented 2 months ago

@ahumphries2 Your request was processed. Apologies for the inconvenience.

ahumphries2 commented 2 months ago

Sergiu,

I see the refund for the Topup, but not the charge for the Pro account, which I could not get working.

Andy Humphries Tech Manager – South Branch O (304) 530-1100 x1292 A 587 Robert C. Byrd Industrial Park Rd Moorefield, WV 26836

@.***

From: Sergiu Waxmann @.> Sent: Friday, September 13, 2024 11:56 AM To: ultralytics/hub @.> Cc: Andrew Humphries @.>; Mention @.> Subject: Re: [ultralytics/hub] Model won't train in Ultralytics Cloud, Disconnected Error (Issue #829)

This message has originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email.

@ahumphries2https://github.com/ahumphries2 Your request was processed. Apologies for the inconvenience.

— Reply to this email directly, view it on GitHubhttps://github.com/ultralytics/hub/issues/829#issuecomment-2349276295, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A2AQMTPSEOUJZUFUGERFEWDZWMDH3AVCNFSM6AAAAABN6S3XZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBZGI3TMMRZGU. You are receiving this because you were mentioned.Message ID: @.**@.>>

sergiuwaxmann commented 2 months ago

@ahumphries2 Both refunds were issues. The refund process can take up to 10 working days.

ahumphries2 commented 2 months ago

Thank you

Andy Humphries Tech Manager – South Branch O (304) 530-1100 x1292 A 587 Robert C. Byrd Industrial Park Rd Moorefield, WV 26836

@.***

From: Sergiu Waxmann @.> Sent: Friday, September 13, 2024 12:00 PM To: ultralytics/hub @.> Cc: Andrew Humphries @.>; Mention @.> Subject: Re: [ultralytics/hub] Model won't train in Ultralytics Cloud, Disconnected Error (Issue #829)

This message has originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email.

@ahumphries2https://github.com/ahumphries2 Both refunds were issues. The refund process can take up to 10 working days.

— Reply to this email directly, view it on GitHubhttps://github.com/ultralytics/hub/issues/829#issuecomment-2349283691, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A2AQMTKJ32UXMSEVSOO2WALZWMDWHAVCNFSM6AAAAABN6S3XZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBZGI4DGNRZGE. You are receiving this because you were mentioned.Message ID: @.**@.>>

sergiuwaxmann commented 2 months ago

@ahumphries2 No worries!