search

jbohnslav / deepethogram

Other
99 stars 32 forks source link

InAccurate Predictions? #34

Closed toujames closed 3 years ago

toujames commented 3 years ago

Hi,

I have general questions about the workflow mentioned in the Getting started documents. We're seeing how accurate the model infers the behaviors compared to a manual coder/labeler.

We're using ICC to compare the difference between manual coder and the model. Some behaviors we're getting good ICC and other's not so great.

If we keep correcting the incorrect behaviors the model predicts, would this not be the same as re training the dataset again?

Our method is 1) labeling a video (with the correct behaviors) 2) training using flow generator/feature extractor /sequence 3) Comparing the inference with manual coder, (this is where we are at)

Once we're at step 3 and the model isn't so accurate, can we not just go to 2 instead of importing predictions as labels?

Thank you. James

toujames commented 3 years ago

Thanks. will 'import predictions as labels' overwrite the current labels I already have?

jbohnslav commented 3 years ago

It will ask you whether you want to overwrite the label for the current video (not recommended!) or only import the predictions for the frames you haven't labeled. I recommend using it on new videos-- never overwriting previously labeled videos, obviously.

jbohnslav commented 3 years ago

Just updated the codebase for beta; please upgrade DeepEthogram. If you're still having model performance issues, open a new ticket.