inference.py GOT : RuntimeError: Input type (unsigned char) and bias type (c10::Half) should be the same

[zhuzhuojiang]

python inference.py --data ../training/nurse_hat.yaml --model yolo_nas_m --weight runs/train6/ckpt_best.pth --source ./how-to-become-a-critical-care-nurse-australia-ozstudies.jpg --conf 0.66 --save

nurse_hat.yaml: `names:

• no_nurse_hat
• nurse_hat Dir: '/root/autodl-tmp/yolonas/training/NurseHat' images: test: test/images train: train/images val: val/images labels: test: test/labels train: train/labels val: val/labels`

[2023-06-14 13:01:42] INFO - pipelines.py - Fusing some of the model's layers. If this takes too much memory, you can deactivate it by setting fuse_model=False Traceback (most recent call last): File "/root/autodl-tmp/yolonas/YOLO-NAS/inference.py", line 69, in labels, class_names = get_bbox(img) File "/root/autodl-tmp/yolonas/YOLO-NAS/inference.py", line 47, in get_bbox preds = next(model.predict(img_rgb, conf=args['conf'])._images_prediction_lst) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/super_gradients/training/models/detection_models/customizable_detector.py", line 177, in predict return pipeline(images) # type: ignore File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/super_gradients/training/pipelines/pipelines.py", line 96, in call return self.predict_images(inputs, batch_size) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/super_gradients/training/pipelines/pipelines.py", line 111, in predict_images return self._combine_image_prediction_to_images(result_generator, n_images=len(images)) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/super_gradients/training/pipelines/pipelines.py", line 290, in _combine_image_prediction_to_images images_predictions = [next(iter(images_predictions))] File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/super_gradients/training/pipelines/pipelines.py", line 149, in _generate_prediction_result yield from self._generate_prediction_result_single_batch(batch_images) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/super_gradients/training/pipelines/pipelines.py", line 176, in _generate_prediction_result_single_batch model_output = self.model(torch_inputs) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, kwargs) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/super_gradients/training/models/detection_models/customizable_detector.py", line 87, in forward x = self.backbone(x) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/super_gradients/modules/detection_modules.py", line 80, in forward x = getattr(self, layer)(x) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/super_gradients/training/models/detection_models/yolo_nas/yolo_stages.py", line 138, in forward return self.conv(x) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/super_gradients/modules/qarepvgg_block.py", line 179, in forward return self.se(self.nonlinearity(self.rbr_reparam(inputs))) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, **kwargs) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 463, in forward return self._conv_forward(input, self.weight, self.bias) File "/root/miniconda3/envs/yolonas-train/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 459, in _conv_forward return F.conv2d(input, weight, bias, self.stride, RuntimeError: Input type (unsigned char) and bias type (c10::Half) should be the same

[zhuzhuojiang]

use pip install -r requirements.txt AND conda PYTHON=3.10.0

[zhuzhuojiang]

BTW, I changed the line 32 of "train.py" ap.add_argument("--warmup_mode", type=float, default='linear_epoch_step', TO: ap.add_argument("--warmup_mode", type=str, default='linear_epoch_step',

Otherwise it will report type error when use train.py

[naseemap47]

use pip install -r requirements.txt AND conda PYTHON=3.10.0

Hi @zhuzhuojiang , I think this is issue with model or image. Can you share your model and test image. (any sharing method, eg: google drive ) So that I can test and verify, what is the actual issue.

[naseemap47]

BTW, I changed the line 32 of "train.py" ap.add_argument("--warmup_mode", type=float, default='linear_epoch_step', TO: ap.add_argument("--warmup_mode", type=str, default='linear_epoch_step',

Otherwise it will report type error when use train.py

I will look to that. Thank you for finding this issue.

[naseemap47]

BTW, I changed the line 32 of "train.py" ap.add_argument("--warmup_mode", type=float, default='linear_epoch_step', TO: ap.add_argument("--warmup_mode", type=str, default='linear_epoch_step',

Otherwise it will report type error when use train.py

@zhuzhuojiang I fixed this typo error. Thank you for finding. :smiley:

[zhuzhuojiang]

@naseemap47 here the https://drive.google.com/file/d/16tyKd2b_ZB22mKN9-wikzzdMdYyzJDUg/view?usp=sharing
traning by: python train.py --data ~/autodl-tmp/yolonas/training/nurse_hat.yaml --batch 6 --epoch 100 --model yolo_nas_m

./ckpt_best.pth // the best model ./how-to-become-a-critical-care-nurse-australia-ozstudies.jpg // the inference source ./training/nurse_hat.yaml // the yaml file ./traning/NurseHat // the dataset

Thank you for your replied

[naseemap47]

Hi @zhuzhuojiang , I check your model. the error came due to model training didn't happened properly. I trained with your data. Its working good. Please check the training step. Just try one more time to train the model properly. It will work. Please let me know, it working or NOT after training.

[zhuzhuojiang]

Thank you for your time. my model is working well in my another computer. But failed in my training env. Maybe version issue. NOT sure. For example I met another issue in inference.py, line 51 for box, cnf, cs in zip(bboxes, confs, labels): plot_one_box(box[:4], img, label=f'{class_names[cs]} {cnf:.3}', color=colors[cs]) will report error: omegaconf.errors.KeyValidationError: ListConfig indices must be integers or slices, not int64 I need to change code to : for box, cnf, cs in zip(bboxes, confs, labels): plot_one_box(box[:4], img, label=f'{class_names[int(cs)]} {cnf:.3}', color=colors[int(cs)])

[naseemap47]

Hi @zhuzhuojiang , Thank you for testing in different envs. If you facing issue with env, try to do with conda env.

Install dependencies with conda:

conda create -n yolo-nas python=3.9 -y
conda activate yolo-nas
conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch -y
pip install triton==2.0.0
pip install super-gradients

Let me know this is working for you or NOT.

[naseemap47]

For example I met another issue in inference.py, line 51 for box, cnf, cs in zip(bboxes, confs, labels): plot_one_box(box[:4], img, label=f'{class_names[cs]} {cnf:.3}', color=colors[cs]) will report error: omegaconf.errors.KeyValidationError: ListConfig indices must be integers or slices, not int64 I need to change code to : for box, cnf, cs in zip(bboxes, confs, labels): plot_one_box(box[:4], img, label=f'{class_names[int(cs)]} {cnf:.3}', color=colors[int(cs)])

cs is numpy.int64, So there is no need of converting into int (<class 'int'>) I think this issue also came due to your train env. Try with conda as mentioned above comment.

[naseemap47]

Hi @zhuzhuojiang , I think you issue is solved. So I am closing this issue. If you have any issues and questions please let me know. Thank you.