Training with my own dataset

dthtien commented 5 years ago

Hi @Borda, thanks for the great working. It is highly appreciated if you could support. I want to train the 2 new classes flat rack container and general container and I have some questions.

How many labeled images are enough (minimum)?
Do I have to change the anchors file?

Currently, I have trained 5 images for each image.And got the result like this. output_test This is my training information

#train.txt 
training_set/general_container/images/df3b509a.jpg 1,1,268,198,20
training_set/general_container/images/image9.jpg 15,31,115,108,20
training_set/general_container/images/image10.jpg 20,29,474,288,20
training_set/general_container/images/image28.jpg 42,17,450,347,20
training_set/general_container/images/image29.jpg 34,67,465,311,20
training_set/general_container/images/image32.jpg 48,4,432,351,20
training_set/flat_rack_container/images/1.jpg 15,172,702,447,21
training_set/flat_rack_container/images/10.jpg 6,30,771,499,21
training_set/flat_rack_container/images/10FR40.jpg 104,323,1980,13232,21
training_set/flat_rack_container/images/11.jpg 18,140,464,429,21
training_set/flat_rack_container/images/20-feet-flat-rack-shipping-container-500x500.jpg 25,82,491,423,21
training_set/flat_rack_container/images/20-Flat-Rack.jpg 12,14,434,350,21 438,24,596,300,21

Screen Shot 2019-09-04 at 2 52 30 PM

#classes
general container
flat rack container

Do you know the reason?

Borda commented 5 years ago

How many labeled images are enough (minimum)?

it depends how much your images and classes are different from the pre-trained objects, but I would say that you should have hundreds of new images; the number can be compensated by good image augmentation, meaning generate realistic images and/or produce all variety of cases that can be observed...

Do I have to change the anchors file?

I believe that you can use the original anchors' file unless positions of your object are not very specific, for example only top-left corner or centre of the image...

[IMAGE]

it looks promising that you have detected only the new classes in sample image and not of the old/pre-trained

#classes
general container
flat rack container
Do you know the reason?

your train file has labels for new classes 20 and 21, but your classes file has the new labels on position 0 and 1... I would recommend using labels 0 and 1 instead

dthtien commented 5 years ago

Thank you so much for the information. Currently, I'm using pre-train weight https://pjreddie.com/media/files/darknet53.conv.74. I have fixed following the suggestion But I got this error when running training.py

Borda commented 5 years ago

@dthtien could you pls past the error as text instead of image...

dthtien commented 5 years ago

Oh sorry. The error is here.

2019-09-04 22:02:11.280014: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:502] remapper failed: Invalid argument: Subshape must have computed start >= end since stride is negative, but is 0 and 2 (computed from start 0 and end 9223372036854775807 over shape with rank 2 and stride-1)
1/1 [==============================] - 5s 5s/step - loss: 1747.7900 - val_loss: 1713.0555
Epoch 2/50
1/1 [==============================] - 2s 2s/step - loss: 1613.8818 - val_loss: 1708.9526
Epoch 3/50
Traceback (most recent call last):
  File "scripts/training.py", line 210, in <module>
    _main(**arg_params)
  File "scripts/training.py", line 181, in _main
    callbacks=[tb_logging, checkpoint, reduce_lr, early_stopping])
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/training.py", line 1418, in fit_generator
    initial_epoch=initial_epoch)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/training_generator.py", line 217, in fit_generator
    class_weight=class_weight)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/training.py", line 1211, in train_on_batch
    class_weight=class_weight)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/training.py", line 804, in _standardize_user_data
    check_array_length_consistency(x, y, sample_weights)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/training_utils.py", line 237, in check_array_length_consistency
    'and ' + str(list(set_y)[0]) + ' target samples.')
ValueError: Input arrays should have the same number of samples as target arrays. Found 12 input samples and 16 target samples.

Borda commented 5 years ago

It should be fine now, pls, feel free to and any other issue you find... or reopen this one :)

Borda / keras-yolo3

Training with my own dataset #16