search

ultralytics / yolov5

YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite
https://docs.ultralytics.com
GNU Affero General Public License v3.0
50.56k stars 16.31k forks source link

wandb.errors.UsageError: Invalid project name '../runs/train-cls': cannot contain characters '/,\\,#,?,%,:', found '/' #10940

Closed Cong-Wan closed 1 year ago

Cong-Wan commented 1 year ago

Search before asking

YOLOv5 Component

Training

Bug

classify/train: model=/civi/5yolo-7.0/weights/yolov5m-cls.pt, data=/civi/DataSets/VehicleType_Classify, epochs=300, batch_size=128, imgsz=224, nosave=False, cache=None, device=0, workers=8, project=../runs/train-cls, name=230209_5m_VehicleType_Classify_1, exist_ok=False, pretrained=True, optimizer=SGD, lr0=0.001, decay=5e-05, label_smoothing=0.1, cutoff=None, dropout=None, verbose=False, seed=0, local_rank=-1 github: ⚠️ YOLOv5 is out of date by 22 commits. Usegit pullorgit clone https://github.com/ultralytics/yolov5` to update. YOLOv5 🚀 v7.0-63-gcdd804d Python-3.8.0 torch-1.11.0 CUDA:0 (NVIDIA GeForce RTX 3090, 24268MiB)

TensorBoard: Start with 'tensorboard --logdir ../runs/train-cls', view at http://localhost:6006/ wandb: ERROR Invalid project name '../runs/train-cls': cannot contain characters '/,\,#,?,%,:', found '/' Traceback (most recent call last): File "train.py", line 333, in main(opt) File "train.py", line 319, in main train(opt, device) File "train.py", line 72, in train logger = GenericLogger(opt=opt, console_logger=LOGGER) if RANK in {-1, 0} else None File "/civi/5yolo-7.0/utils/loggers/init.py", line 341, in init self.wandb = wandb.init(project=web_project_name(str(opt.project)), File "/civi/anaconda/envs/v5/lib/python3.8/site-packages/wandb/sdk/wandb_init.py", line 1129, in init wi.setup(kwargs) File "/civi/anaconda/envs/v5/lib/python3.8/site-packages/wandb/sdk/wandb_init.py", line 288, in setup settings._apply_init(kwargs) File "/civi/anaconda/envs/v5/lib/python3.8/site-packages/wandb/sdk/wandb_settings.py", line 1535, in _apply_init self.update(init_settings, source=Source.INIT) File "/civi/anaconda/envs/v5/lib/python3.8/site-packages/wandb/sdk/wandb_settings.py", line 1230, in update self.dict[key].update(value, source) File "/civi/anaconda/envs/v5/lib/python3.8/site-packages/wandb/sdk/wandb_settings.py", line 336, in update self._value = self._validate(self._preprocess(value)) File "/civi/anaconda/envs/v5/lib/python3.8/site-packages/wandb/sdk/wandb_settings.py", line 297, in _validate if not v(value): File "/civi/anaconda/envs/v5/lib/python3.8/site-packages/wandb/sdk/wandb_settings.py", line 764, in _validate_project raise UsageError( wandb.errors.UsageError: Invalid project name '../runs/train-cls': cannot contain characters '/,\,#,?,%,:', found '/' ` I have located a similar question in the issue, but the solution to it was not provided in that thread. I am encountering the same error as well. I have utilized the latest code and updated all required packages. Despite running the program on an Ubuntu system, the issue persists and continues to produce the same error. I have also attempted to disable the wandb component, but this has not resolved the problem.

Environment

YOLOv5 🚀 v7.0-63-gcdd804d Python-3.8.0 torch-1.11.0 CUDA:0 (NVIDIA GeForce RTX 3090, 24268MiB)

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

Cong-Wan commented 1 year ago

I got the reason!

open the file

utils/loggers/init.py find this class image

just add os.path.basename before the web_project_name(str(opt.project))

github-actions[bot] commented 1 year ago

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

Access additional Ultralytics ⚡ resources:

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

amadeok commented 1 year ago

it seems it's been updated , now file is C:\Users\user\AppData\Local\Programs\Python\Python310\Lib\site-packages\ultralytics\utils\callbacks\wb.py, should not pass a folder path to the wandb project parameter, this works:

def on_pretrain_routine_start(trainer):
    import os
    """Initiate and start project if module is present."""
    wb.run or wb.init(project=os.path.basename(trainer.args.project) if trainer.args.project else 'YOLOv8', name=trainer.args.name, config=vars(trainer.args))
glenn-jocher commented 1 year ago

@amadeok thank you for sharing your findings!

You are correct that the file wb.py in the ultralytics package has been updated. It seems that passing a folder path to the wandb project parameter is causing the issue.

Your suggested solution of using os.path.basename(trainer.args.project) instead of trainer.args.project should solve the problem. Here's the updated code:

def on_pretrain_routine_start(trainer):
    import os
    """Initiate and start project if module is present."""
    wb.run or wb.init(project=os.path.basename(trainer.args.project) or 'YOLOv8', name=trainer.args.name, config=vars(trainer.args))

Thank you for your contribution to YOLOv5! Let us know if you have any further questions or issues.

amadeok commented 1 year ago

@amadeok thank you for sharing your findings!

You are correct that the file wb.py in the ultralytics package has been updated. It seems that passing a folder path to the wandb project parameter is causing the issue.

Your suggested solution of using os.path.basename(trainer.args.project) instead of trainer.args.project should solve the problem. Here's the updated code:

def on_pretrain_routine_start(trainer):
    import os
    """Initiate and start project if module is present."""
    wb.run or wb.init(project=os.path.basename(trainer.args.project) or 'YOLOv8', name=trainer.args.name, config=vars(trainer.args))

Thank you for your contribution to YOLOv5! Let us know if you have any further questions or issues.

hi, i updated my comment, to take into account when trainer.args.project is None, otherwise another error will occur when the argument project is not specified when running yolo.exe

glenn-jocher commented 1 year ago

@amadeok thank you for updating your comment. Taking into account the case when trainer.args.project is None is important to avoid further errors when the argument project is not specified when running yolo.exe.

Here's the updated code considering the case when trainer.args.project is None:

def on_pretrain_routine_start(trainer):
    import os
    """Initiate and start project if module is present."""
    project = os.path.basename(trainer.args.project) if trainer.args.project else 'YOLOv8'
    wb.run or wb.init(project=project, name=trainer.args.name, config=vars(trainer.args))

Thank you for your contribution and diligence in addressing this issue. Let us know if you have any further questions or if there's anything else we can assist you with.

rajnitish commented 11 months ago

Dear glenn-jocher I have trained my model on Yolov8 on google colab. And stored results in project directory. now hen i am trying to trained with new images using last.pt on jupyter notebook. it is giving me error. from ultralytics import YOLO import os

CURRENT_PATH = os.getcwd() print(CURRENT_PATH) custom_checkpoint_path = os.path.join(CURRENT_PATH,"runs","detect","Military_Aircraft_Detection","weights", "last.pt") model = YOLO(custom_checkpoint_path)

yaml_path = os.path.join(CURRENT_PATH, "Aircraft.yaml") os.path.join(CURRENT_PATH,"runs","detect","Military_Aircraft_Detection"), results = model.train(data=yaml_path,name = "ABC",resume=True)

D:\Capstone\new\Capstone_MAD_Mavericks New https://pypi.org/project/ultralytics/8.0.216 available Update with 'pip install -U ultralytics' Ultralytics YOLOv8.0.188 Python-3.8.13 torch-2.0.1+cpu CPU (Intel Core(TM) i7-8550U 1.80GHz) engine\trainer: task=detect, mode=train, model=D:\Capstone\new\Capstone_MAD_Mavericks\runs\detect\Military_Aircraft_Detection\weights\last.pt, data=D:\Capstone\new\Capstone_MAD_Mavericks\Aircraft.yaml, epochs=75, patience=3, batch=16, imgsz=640, save=True, save_period=-1, cache=False, device=None, workers=8, project=/content/drive/MyDrive/Capstone_MAD_Mavericks/runs/detect, name=Military_Aircraft_Detection, exist_ok=True, pretrained=True, optimizer=auto, verbose=True, seed=0, deterministic=True, single_cls=False, rect=False, cos_lr=False, close_mosaic=10, resume=False, amp=True, fraction=1.0, profile=False, freeze=None, overlap_mask=True, mask_ratio=4, dropout=0.0, val=True, split=val, save_json=False, save_hybrid=False, conf=None, iou=0.7, max_det=300, half=False, dnn=False, plots=True, source=None, show=False, save_txt=False, save_conf=False, save_crop=False, show_labels=True, show_conf=True, vid_stride=1, stream_buffer=False, line_width=None, visualize=False, augment=False, agnostic_nms=False, classes=None, retina_masks=False, boxes=True, format=torchscript, keras=False, optimize=False, int8=False, dynamic=False, simplify=False, opset=None, workspace=4, nms=False, lr0=0.01, lrf=0.01, momentum=0.937, weight_decay=0.0005, warmup_epochs=3.0, warmup_momentum=0.8, warmup_bias_lr=0.0, box=7.5, cls=0.5, dfl=1.5, pose=12.0, kobj=1.0, label_smoothing=0.0, nbs=64, hsv_h=0.015, hsv_s=0.7, hsv_v=0.4, degrees=0.0, translate=0.1, scale=0.5, shear=0.0, perspective=0.0, flipud=0.0, fliplr=0.5, mosaic=1.0, mixup=0.0, copy_paste=0.0, cfg=None, tracker=botsort.yaml, save_dir=\content\drive\MyDrive\Capstone_MAD_Mavericks\runs\detect\Military_Aircraft_Detection TensorBoard: Start with 'tensorboard --logdir \content\drive\MyDrive\Capstone_MAD_Mavericks\runs\detect\Military_Aircraft_Detection', view at http://localhost:6006/

UsageError Traceback (most recent call last) Input In [30], in <cell line: 11>() 9 yaml_path = os.path.join(CURRENT_PATH, "Aircraft.yaml") 10 #, exist_ok= True, project = os.path.join(CURRENT_PATH,"Runs","detect"),save_dir= os.path.join(CURRENT_PATH,"runs","detect","Military_Aircraft_Detection"), ---> 11 results = model.train(data=yaml_path,name = "ABC",resume=True)

File C:\anaconda3\lib\site-packages\ultralytics\engine\model.py:334, in Model.train(self, trainer, **kwargs) 332 self.model = self.trainer.model 333 self.trainer.hub_session = self.session # attach optional HUB session --> 334 self.trainer.train() 335 # Update model and cfg after training 336 if RANK in (-1, 0):

File C:\anaconda3\lib\site-packages\ultralytics\engine\trainer.py:195, in BaseTrainer.train(self) 192 ddp_cleanup(self, str(file)) 194 else: --> 195 self._do_train(world_size)

File C:\anaconda3\lib\site-packages\ultralytics\engine\trainer.py:293, in BaseTrainer._do_train(self, world_size) 291 if world_size > 1: 292 self._setup_ddp(world_size) --> 293 self._setup_train(world_size) 295 self.epoch_time = None 296 self.epoch_time_start = time.time()

File C:\anaconda3\lib\site-packages\ultralytics\engine\trainer.py:215, in BaseTrainer._setup_train(self, world_size) 210 """ 211 Builds dataloaders and optimizer on correct rank process. 212 """ 214 # Model --> 215 self.run_callbacks('on_pretrain_routine_start') 216 ckpt = self.setup_model() 217 self.model = self.model.to(self.device)

File C:\anaconda3\lib\site-packages\ultralytics\engine\trainer.py:160, in BaseTrainer.run_callbacks(self, event) 158 """Run all existing callbacks associated with a particular event.""" 159 for callback in self.callbacks.get(event, []): --> 160 callback(self)

File C:\anaconda3\lib\site-packages\ultralytics\utils\callbacks\wb.py:29, in on_pretrain_routine_start(trainer) 27 def on_pretrain_routine_start(trainer): 28 """Initiate and start project if module is present.""" ---> 29 wb.run or wb.init(project=trainer.args.project or 'YOLOv8', name=trainer.args.name, config=vars(trainer.args))

File C:\anaconda3\lib\site-packages\wandb\sdk\wandb_init.py:1184, in init(job_type, dir, config, project, entity, reinit, tags, group, name, notes, magic, config_exclude_keys, config_include_keys, anonymous, mode, allow_val_change, resume, force, tensorboard, sync_tensorboard, monitor_gym, save_code, id, settings) 1182 if logger is not None: 1183 logger.exception(str(e)) -> 1184 raise e 1185 except KeyboardInterrupt as e: 1186 assert logger

File C:\anaconda3\lib\site-packages\wandb\sdk\wandb_init.py:1161, in init(job_type, dir, config, project, entity, reinit, tags, group, name, notes, magic, config_exclude_keys, config_include_keys, anonymous, mode, allow_val_change, resume, force, tensorboard, sync_tensorboard, monitor_gym, save_code, id, settings) 1159 try: 1160 wi = _WandbInit() -> 1161 wi.setup(kwargs) 1162 assert wi.settings 1163 except_exit = wi.settings._except_exit

File C:\anaconda3\lib\site-packages\wandb\sdk\wandb_init.py:320, in _WandbInit.setup(self, kwargs) 317 # get status of code saving before applying user settings 318 save_code_pre_user_settings = settings.save_code --> 320 settings._apply_init(kwargs) 321 if not settings._offline and not settings._noop: 322 user_settings = self._wl._load_user_settings()

File C:\anaconda3\lib\site-packages\wandb\sdk\wandb_settings.py:1818, in Settings._apply_init(self, init_settings) 1815 init_settings["resume"] = "auto" 1817 # update settings -> 1818 self.update(init_settings, source=Source.INIT) 1820 # handle auto resume logic 1821 if self.resume == "auto":

File C:\anaconda3\lib\site-packages\wandb\sdk\wandb_settings.py:1458, in Settings.update(self, settings, source, **kwargs) 1456 for key in self.__modification_order: 1457 if key in settings: -> 1458 self.dict[key].update(settings.pop(key), source=source) 1460 # update the remaining properties 1461 for key, value in settings.items():

File C:\anaconda3\lib\site-packages\wandb\sdk\wandb_settings.py:579, in Property.update(self, value, source) 561 # - always update value if source == Source.OVERRIDE 562 # - if not previously overridden: 563 # - update value if source is lower than or equal to current source and property is policy 564 # - update value if source is higher than or equal to current source and property is not policy 565 if ( 566 (source == Source.OVERRIDE) 567 or ( (...) 577 ): 578 # self.dict["_value"] = self._validate(self._preprocess(value)) --> 579 self._value = self._validate(self._preprocess(value)) 580 self._source = source

File C:\anaconda3\lib\site-packages\wandb\sdk\wandb_settings.py:549, in Property._validate(self, value) 545 _validator = ( 546 [self._validator] if callable(self._validator) else self._validator 547 ) 548 for v in _validator: --> 549 if not v(value): 550 # failed validation will likely cause a downstream error 551 # when trying to convert to protobuf, so we raise a hard error 552 raise SettingsValidationError( 553 f"Invalid value for property {self.name}: {value}." 554 ) 555 return value

File C:\anaconda3\lib\site-packages\wandb\sdk\wandb_settings.py:961, in Settings._validate_project(value) 959 invalid_chars = {char for char in invalid_chars_list if char in value} 960 if invalid_chars: --> 961 raise UsageError( 962 f"Invalid project name {value!r}: " 963 f"cannot contain characters {','.join(invalid_chars_list)!r}, " 964 f"found {','.join(invalid_chars)!r}" 965 ) 966 return True

UsageError: Invalid project name '/content/drive/MyDrive/Capstone_MAD_Mavericks/runs/detect': cannot contain characters '/,\,#,?,%,:', found '/'

glenn-jocher commented 11 months ago

@rajnitish it looks like the model.train method is encountering an error due to the invalid characters ('/,\,#,?,%,:') found in the project name '/content/drive/MyDrive/Capstone_MAD_Mavericks/runs/detect'.

To resolve this issue, it's important to ensure that the project name used does not contain any of the characters mentioned. You can modify the project name to remove these invalid characters before passing it to the model.train method.

Feel free to make the necessary adjustments to the project name and try running the code again. If you encounter any further issues or have additional questions, please don't hesitate to ask for further assistance.

rajnitish commented 11 months ago

@glenn-jocher I already have last.pt file (best=last, due to patience early stop). Now for new delta images, i want to resume the training.

model = YOLO('best.pt') results = model.train(data=yaml_path ,resume=True)

This is also giving invalid characters ('/,\,#,?,%,:') errors. Are these errors because of the previous training what I did on Google colab. because there I use project path and name with special characters like /content/drive... Google drive uses like this

glenn-jocher commented 11 months ago

@rajnitish The invalid characters ('/,\,#,?,%,:) error could indeed be caused by the special characters used in the project path and name, especially if they were used during the previous training on Google Colab.

To address this issue, I recommend modifying the project path and name to remove the special characters before resuming the training. Ensuring that the project name doesn't contain any of these invalid characters should resolve the error.

Feel free to adjust the project path and name as needed, and then try to resume the training with the modified project details. If you encounter any further issues or have additional questions, please feel free to ask for further assistance.

adindravickarega commented 1 week ago

Hello i also have similar problem with resuming training with YOLOv10..how to change the project name in Google Colab?

pderrenger commented 1 week ago

To change the project name in Google Colab, modify the project parameter in your training script to a valid name without special characters. For example: results = model.train(data=yaml_path, project='NewProjectName', resume=True).

MOstafa20azad commented 1 week ago

@pderrenger In Google Collab, I did what you said and the program ran without error and the result was saved at the end. But I can't find the saved file. I even searched for the root of Google Drive, but it was not found. Do you have any suggestions?

Input: -->project='logs'

output: --> The file saved to: 'logs/2024-10-22_yolov8s_DETR_pretrained'

pderrenger commented 1 week ago

Ensure that the logs directory is correctly mounted to your Google Drive in Colab. You can check the path using !ls logs to verify the files are there. If not, confirm your Google Drive is properly mounted.