search

ultralytics / yolov3

YOLOv3 in PyTorch > ONNX > CoreML > TFLite
https://docs.ultralytics.com
GNU Affero General Public License v3.0
10.09k stars 3.44k forks source link

KeyError: 'module_list.85.Conv2d.weight' #650

Closed alontrais closed 4 years ago

alontrais commented 4 years ago

Hey I get a new error whan I run the train script:

Downloading https://drive.google.com/uc?export=download&id=158g62Vs14E3aj7oPVPuEnNZMKFNgGyNq as weights/ultralytics49.pt... Done (2.8s)
Traceback (most recent call last):
  File "train.py", line 444, in <module>
    train()  # train normally
  File "train.py", line 111, in train
    chkpt['model'] = {k: v for k, v in chkpt['model'].items() if model.state_dict()[k].numel() == v.numel()}
  File "train.py", line 111, in <dictcomp>
    chkpt['model'] = {k: v for k, v in chkpt['model'].items() if model.state_dict()[k].numel() == v.numel()}
KeyError: 'module_list.85.Conv2d.weight'
daddydrac commented 4 years ago

That’s weird because I have a 2080Ti. I would have thought 11GB is fine.

On Mon, Dec 2, 2019 at 1:32 PM Glenn Jocher notifications@github.com wrote:

Ok, this should do it: 93a70d9 https://github.com/ultralytics/yolov3/commit/93a70d958a1138b082f7b5c29c550b7d383f56f3

If you git pull you can get all the latest updates.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ultralytics/yolov3/issues/650?email_source=notifications&email_token=ABHVQHF3GAWSHGUXEFWWM6DQWVPEZA5CNFSM4JQ3CBSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFUUC6Y#issuecomment-560546171, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHVQHH344V3NWGKOAP6ISTQWVPEZANCNFSM4JQ3CBSA .

daddydrac commented 4 years ago

Thanks, did the git pull, and is working just fine.

daddydrac commented 4 years ago

How long does it take to train normally?

Note, I ran: python3 train.py --data data/custom.data --cfg cfg/yolov3-spp.cfg --weights weights/yolov3-spp.weights

glenn-jocher commented 4 years ago

@joehoeller training speeds are here. https://github.com/ultralytics/yolov3#speed

Roughly a week to train COCO. Smaller datasets faster of course.

glenn-jocher commented 4 years ago

@joehoeller nvidia apex speeds things up a lot. This repo uses it automatically if it installed. https://github.com/NVIDIA/apex

daddydrac commented 4 years ago

Cool, thanks - btw how do we extract metrics like map, false positives etc from detect command like this:

python3 detect.py --source ./coco/images/FLIR_Dataset/training/Data/
--nms-thres 0.7 --conf-thres 0.6 --data data/custom.data --cfg
cfg/yolov3-spp.cfg --weights weights/yolov3-spp.weights
daddydrac commented 4 years ago

@joehoeller nvidia apex speeds things up a lot. This repo uses it automatically if it installed. https://github.com/NVIDIA/apex

Check out my Pytorch/Anaconda/TensorRT container on my github, TensorRT does same thing :)

glenn-jocher commented 4 years ago

@joehoeller to get test metrics run python3 test.py with the same dataset and model you trained on.

$ python3 test.py --weights ultralytics68.pt --img-size 512 --device 0

Namespace(batch_size=16, cfg='cfg/yolov3-spp.cfg', conf_thres=0.001, data='data/coco.data', device='0', img_size=512, iou_thres=0.5, nms_thres=0.5, save_json=False, weights='ultralytics68.pt')
Using CUDA device0 _CudaDeviceProperties(name='GeForce RTX 2080 Ti', total_memory=10989MB)

Downloading https://drive.google.com/uc?export=download&id=1Jm8kqnMdMGUUxGo8zMFZMJ0eaPwLkxSG as ultralytics68.pt... Done (7.6s)
               Class    Images   Targets         P         R   mAP@0.5        F1: 100%|███████████████████████████████████████████████████████████████| 313/313 [08:30<00:00,  1.35it/s]
                 all     5e+03  3.58e+04    0.0823     0.798     0.595     0.145
              person     5e+03  1.09e+04    0.0999     0.903     0.771      0.18
             bicycle     5e+03       316    0.0491     0.782      0.56    0.0925
                 car     5e+03  1.67e+03    0.0552     0.845     0.646     0.104
          motorcycle     5e+03       391      0.11     0.847     0.704     0.194
            airplane     5e+03       131     0.099     0.947     0.878     0.179
                 bus     5e+03       261     0.142     0.874     0.825     0.244
               train     5e+03       212     0.152     0.863     0.806     0.258
               truck     5e+03       352    0.0849     0.682     0.514     0.151
                boat     5e+03       475    0.0498     0.787     0.504    0.0937
       traffic light     5e+03       516    0.0304     0.752     0.516    0.0584
        fire hydrant     5e+03        83     0.144     0.916     0.882     0.248
           stop sign     5e+03        84    0.0833     0.917     0.809     0.153
       parking meter     5e+03        59    0.0607     0.695     0.611     0.112
               bench     5e+03       473    0.0294     0.685     0.363    0.0564
                bird     5e+03       469    0.0521     0.716     0.524    0.0972
                 cat     5e+03       195     0.252     0.908      0.78     0.395
                 dog     5e+03       223     0.192     0.883     0.829     0.315
               horse     5e+03       305     0.121     0.911     0.843     0.214
               sheep     5e+03       321     0.114     0.854     0.724     0.201
                 cow     5e+03       384     0.105     0.849     0.695     0.187
            elephant     5e+03       284     0.184     0.944     0.912     0.308
                bear     5e+03        53     0.358     0.925     0.875     0.516
               zebra     5e+03       277     0.176     0.935     0.858     0.297
             giraffe     5e+03       170     0.171     0.959     0.892      0.29
            backpack     5e+03       384    0.0426     0.708     0.392    0.0803
            umbrella     5e+03       392    0.0672     0.878      0.65     0.125
             handbag     5e+03       483    0.0238     0.629     0.242    0.0458
                 tie     5e+03       297    0.0419     0.805     0.599    0.0797
            suitcase     5e+03       310    0.0823     0.855     0.628      0.15
             frisbee     5e+03       109     0.126     0.872     0.796     0.221
                skis     5e+03       282    0.0473     0.748     0.454     0.089
           snowboard     5e+03        92    0.0579     0.804     0.559     0.108
         sports ball     5e+03       236     0.057     0.733     0.622     0.106
                kite     5e+03       399     0.087     0.852     0.645     0.158
        baseball bat     5e+03       125    0.0496     0.776     0.603    0.0932
      baseball glove     5e+03       139    0.0511     0.734     0.563    0.0956
          skateboard     5e+03       218    0.0655     0.844      0.73     0.122
           surfboard     5e+03       266    0.0709     0.827     0.651     0.131
       tennis racket     5e+03       183    0.0694     0.858     0.759     0.128
              bottle     5e+03       966    0.0484     0.812     0.513    0.0914
          wine glass     5e+03       366    0.0735     0.738     0.543     0.134
                 cup     5e+03       897    0.0637     0.788     0.538     0.118
                fork     5e+03       234    0.0411     0.662     0.487    0.0774
               knife     5e+03       291    0.0334     0.557     0.292    0.0631
               spoon     5e+03       253    0.0281     0.621     0.307    0.0537
                bowl     5e+03       620    0.0624     0.795     0.514     0.116
              banana     5e+03       371     0.052      0.83      0.41    0.0979
               apple     5e+03       158    0.0293     0.741     0.262    0.0564
            sandwich     5e+03       160    0.0913     0.725     0.522     0.162
              orange     5e+03       189    0.0382     0.688      0.32    0.0723
            broccoli     5e+03       332    0.0513      0.88     0.445     0.097
              carrot     5e+03       346    0.0398     0.766     0.362    0.0757
             hot dog     5e+03       164    0.0958     0.646     0.494     0.167
               pizza     5e+03       224    0.0886     0.875     0.699     0.161
               donut     5e+03       237    0.0925     0.827      0.64     0.166
                cake     5e+03       241    0.0658      0.71     0.539      0.12
               chair     5e+03  1.62e+03    0.0432     0.793     0.489    0.0819
               couch     5e+03       236     0.118     0.801     0.584     0.205
        potted plant     5e+03       431    0.0373     0.852     0.505    0.0714
                 bed     5e+03       195     0.149     0.846     0.693     0.253
        dining table     5e+03       634    0.0546      0.82      0.49     0.102
              toilet     5e+03       179     0.161      0.95      0.81     0.275
                  tv     5e+03       257    0.0922     0.903      0.79     0.167
              laptop     5e+03       237     0.127     0.869     0.744     0.222
               mouse     5e+03        95    0.0648     0.863     0.732      0.12
              remote     5e+03       241    0.0436     0.788     0.535    0.0827
            keyboard     5e+03       117    0.0668     0.923     0.755     0.125
          cell phone     5e+03       291    0.0364     0.704     0.436    0.0692
           microwave     5e+03        88     0.154     0.841     0.743     0.261
                oven     5e+03       142    0.0618     0.803     0.576     0.115
             toaster     5e+03        11    0.0565     0.636     0.191     0.104
                sink     5e+03       211    0.0439     0.853     0.544    0.0835
        refrigerator     5e+03       107    0.0791     0.907     0.742     0.145
                book     5e+03  1.08e+03    0.0399     0.667     0.233    0.0753
               clock     5e+03       292    0.0542     0.836     0.733     0.102
                vase     5e+03       353    0.0675     0.799     0.591     0.125
            scissors     5e+03        56    0.0397      0.75     0.461    0.0755
          teddy bear     5e+03       245    0.0995     0.882     0.669     0.179
          hair drier     5e+03        11   0.00508    0.0909    0.0475   0.00962
          toothbrush     5e+03        77    0.0371      0.74     0.418    0.0706
daddydrac commented 4 years ago

training error: assert c.max() <= model.nc, 'Target classes exceed model classes' AssertionError: Target classes exceed model classes

daddydrac commented 4 years ago

UPDATE: I fixed the PR and updated the math, the COCO JSON -> Darknet conversion tool (Dark Chocolate) works now: https://github.com/joehoeller/Dark-Chocolate/issues/2

glenn-jocher commented 8 months ago

Glad to hear that, @daddydrac! The COCO JSON to Darknet conversion tool is a great contribution. Thank you for sharing it with the community! If you have any further questions or need assistance with anything else, feel free to ask.