Currently trying to train some embeddings of just the Mdivi and LLO frames, to see how nice of a spread I can get. Using resnet18 plus a couple aggregation methods - flattening CNN frames reps to pass into an MLP, and passing CNN frame reps through an LSTM. I just want to see if this is a good model to represent a few frames, and then I will move onto a different learning strategy. The theme seems to be that the embeddings are not far at all from final decision boundary, val near random, suggesting train is able to classify off of some brittle feature of the train videos.
TODO - I think with the switch to pytorch lightning some random process may not be seeded correctly.
TODO - using out of the box resnet-18 but should go back and use the one I did for the initial baselines, it had leakyrelu instead of relu.
TODO - Still pretty blind on which types of data give which results (ie are frames taken from the end of the video easier to classify?) would be nicer to make better visualization workflow. similar to #2, I want my written thesis to explain the data pretty nicely
Currently trying to train some embeddings of just the Mdivi and LLO frames, to see how nice of a spread I can get. Using resnet18 plus a couple aggregation methods - flattening CNN frames reps to pass into an MLP, and passing CNN frame reps through an LSTM. I just want to see if this is a good model to represent a few frames, and then I will move onto a different learning strategy. The theme seems to be that the embeddings are not far at all from final decision boundary, val near random, suggesting train is able to classify off of some brittle feature of the train videos.
TODO - I think with the switch to pytorch lightning some random process may not be seeded correctly. TODO - using out of the box resnet-18 but should go back and use the one I did for the initial baselines, it had leakyrelu instead of relu. TODO - Still pretty blind on which types of data give which results (ie are frames taken from the end of the video easier to classify?) would be nicer to make better visualization workflow. similar to #2, I want my written thesis to explain the data pretty nicely