efficient_hrnet

[jaentrouble]

Train with efficient_hrnet

• Goal: raspberry pi 4

[jaentrouble]

1. ehr11211_1

   • lr: lr_step7_2

   • image_size = (256,384)

   • epoch 100 steps 500

   • batch size 32

[jaentrouble]

2. ehr11211_2

   • lr: lr_step7_2

   • changed BN_MOMENTUM to 0.9

   • epoch 100 steps 500

   • batch size 32

------------- Abort epoch 74

• Does sufficiently well after 50 epochs.

• Try smaller model

[jaentrouble]

3. ~mobv3_small~ ehr11211_3

   • lr: lr_step7_2

   • Change to MobileNetV3Small

   • batch size 32

   • epoch 70 steps 500

EDIT: IT WAS NOT MOBV3!!

[jaentrouble]

4. ~mobv3_small_block~ ehr11211_block

   • lr: lr_step7_2

   • Train with mini data to detect a block only

     • 0.pck for train, 1.pck for val (~300 for train, ~180 for val)

     • Need more data anyway, so see how it can do with this little data

   • epoch 50 steps 500

EDIT: IT WAS NOT MOBV3!!

[jaentrouble]

5. mobv3_small_block_2

   • lr: lr_step7_2

   • More data

     • 4.pck for val (2448 for train, 157 for val)

   • epoch 50 steps 500

   • No augment other than flipping for now (Will add next time, if it seems essential)

[jaentrouble]

6. mobv3_small_block_3

   • lr: lr_step7_2

   • add shift/rotate augmenting

   • add parallel generator

   • batch size 128

   • epoch 100 steps 500

[jaentrouble]

7. mobv3_small_head

   • lr: lr_step7_2

   • Same as mobv3_small_block_3, but instead of block, track head

   • To see if block shape was the problem of not detecting well, or the model's size

   • epoch 100 steps 500

[jaentrouble]

8. mobv3_small_head_2

   • lr: lr_step7_2

   • mix data from before (which also has head)

   • Add color augmentation

   • epoch 100, steps 500

[jaentrouble]

9. mobv3_small_07_head

   • lr: lr_step7_2

   • same data config as 8

   • phi = -2

     • resolution : (288,224)

     • backbone: mobv3_small_07

   • batch size 196

   • epoch 100, steps 327

[jaentrouble]

Quant-aware model

• Changed all models to functional API
• It can now do quant-aware training

[jaentrouble]

10. mobv3_small_07_head_q
    • lr: lr_step7_2
    • added data (arduino)
    • other things same as 9
    • batch size 512
    • epoch 100, steps 125

Result: Bad. Try different learning rate

[jaentrouble]

11. mobv3_small_07_head_q2
    • lr: lr_step8_3
    • Try lower learning rate
    • epoch 50, steps 125
    • batch size 512
    • Lowered rotate limit to 10 degrees
    • other things unchanged

Abort at epoch 36 Result: Worse than 10. Try higher learning rate. Maybe too small model?

[jaentrouble]

12. mobv3_small_07_head_q3
    • lr: lr_step9
    • Try higher learning rate
    • epoch 50, steps 125
    • batch size 512
    • Other things unchanged

[jaentrouble]

Notes

• In cases like this, where all input data are very similar, increasing batch may not help reducing training time.
• This is because the loss function takes 'mean' value of all elements in batches, which means increasing batch size may help determining 'correct' gradient, but will not help with per-step gradient size.
• Therefore, if batch size is big enough, the number of steps needed to converge may not change.
• Evidence: the sudden drop of maximum-point distance happened almost 3 times more epochs in test 12 than test 9.

[jaentrouble]

13. mobv3_small_07_head_q3q
    • Quantization-aware fine tuning
    • lr: lr_step7_2
    • epoch 20, steps 200
    • batch size 256
    • Load from mobv3_small_07_head_q3/50

Error: Dynamic seems to fail. Try again with full integer(with sample)

[jaentrouble]

Reshape has problems. Workaround: Use concrete function instead.

14. mobv3_small_07_head_q3q3 (q2 is aborted)

    • Quantization-aware fine tuning with full integer (sample data)
    • qLoad from mobv3_small_07_head_q3q/20 (inplemented qload - load after quant aware)

15. mobv3_small_07_head_q3q4

    • Quantization-aware fine tuning, quantization without sample data
    • quantization with sample data returns error

[jaentrouble]

---------------------- 4 Chamber ----------------------

[jaentrouble]

1. mobv3_small_07_head_4cham_0
   • New data (4chamber test + 3led) -> about 8000 frames
   • ~load mobv3_small_07_head_q3/50~ The savefile is 'before' fixing the error! Retrain from the scratch
   • lr: 1e-5 (low_lr)
   • batch 196
   • epoch 100
   • steps 500

Result

• Best val result : epoch 80

[jaentrouble]

2. mobv3_small_07_head_4cham_0q0

• quantize-aware fine tuning
• epoch 20 step 500
• batch 192
• lr: 1e-5 (low_lr)