search

FIRST-Tech-Challenge / fmltc

FIRST Machine Learning Toolchain
Other
38 stars 14 forks source link

Steps must be multiple of 100 #291

Closed AlexKaiser26 closed 1 year ago

AlexKaiser26 commented 1 year ago

If you try to put in 1413 or any number that is not a multiple of 100 it will continue to train the model until 1413, then on the dashboard it will say 1500 steps and will result in 0 learning with 0 confidence on any image.

texasdiaz commented 1 year ago

That is partially correct. Steps should be in a multiple of 100 (the documentation recommends steps be in increments of 100 because that's how they're saved, otherwise you completely lose any incremental steps). If you train for 1413 steps, it will train for 1413 steps, but you will only be able to access the last save point, which occurred at 1400 (every 100 steps). I can't remember off the top of my head, but we might now continue training to the next save point if you haven't exceeded your training time yet.

However, "0 learning with 0 confidence on any image" has little to do with the Steps used for training (if you use too few it'll not train properly, tho) and everything to do with the content used to train the model. Please start an FTC Community Forum thread if you want me to help you with training your model.

AlexKaiser26 commented 1 year ago

Whenever I try to view images or graphs it is all at 1500 steps which shows 0 learning on the graphs and 0 confidence on the images. If it was made to force it to be a multiple of 100 then this wouldn’t be a issue

My model was training fine until it finished and was at 1500 steps.

Once I get on my computer I will upload some screenshots.

texasdiaz commented 1 year ago

What's your team number?

AlexKaiser26 commented 1 year ago

10464

texasdiaz commented 1 year ago

Wow, you're right, there's something very weird going on there. It's as if your last save point in that model just got utterly trashed (it's missing data even). It looks like you've got a pretty decent model with your other 1500 step model, so I'm glad to see it didn't stop you from proceeding. We'll look into it, and if you find yourself needing the 24 minutes back let me know.

Though, I must admit, your Teddy Bear / Zebra / Chair models are all standard categories recognized by the stock Model Zoo already. Have you tried using the stock pre-trained Model Zoo model to detect those objects? More on where to find the TensorFlow 2.x stock Model Zoo models here: https://ftc-community.firstinspires.org/t/tensorflow-model-zoo-models/159/2

-Danny

AlexKaiser26 commented 1 year ago

I tried the stock zoo model before I trained it which is why I used those images, it wasn't working and started detecting the poles and actual people as zebra's so I'm not sure what the issue was there. Thanks for your help though.