search

ijyliu / computer-vision-project

Using classical and neural image embeddings and finetuned end-to-end networks to achieve top-tier performance on a vehicle type classification task. Containerized and deployed model as a web app
https://cv-web-app-3m4f2rmfzq-uc.a.run.app/
0 stars 0 forks source link

Feature Importance for Best Model (If Time Allows) #70

Open ijyliu opened 2 months ago

ijyliu commented 2 months ago

provides insights into what features are important (based off of autogluon, I think all are!)

we need to know what the best model is first!

ijyliu commented 2 months ago

permutation_importance.xlsx

here's permutation improtance for logistic regression on all features. @CalBlitz do you wanna run your plot script on this

ijyliu commented 2 months ago

im pretty solid in us saying vgg is most important and thus deserving to be run on its own, given it's low dimension compared to vit

CalBlitz commented 2 months ago

That's what I've been saying. Great we have the feature importance thing. Not having ResNet feels like such a bummer.

CalBlitz commented 2 months ago

I have ResNet ready if you wanna use only that with other classifiers.

CalBlitz commented 2 months ago

image image

ijyliu commented 2 months ago

We still have overfit. Can you make a version of the data with VGG + 2 simple features

On Sun, Apr 14, 2024, 3:41 PM Gaurav Sharan Srivastava < @.***> wrote:

I have ResNet ready if you wanna use only that with other classifiers.

— Reply to this email directly, view it on GitHub https://github.com/ijyliu/computer-vision-project/issues/70#issuecomment-2054207637, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQCGE4I37WH2BLA6R7VJULDY5MA2JAVCNFSM6AAAAABF5XEG66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJUGIYDONRTG4 . You are receiving this because you were assigned.Message ID: @.***>

ijyliu commented 2 months ago

The two most important simple features

On Sun, Apr 14, 2024, 3:56 PM Isaac Liu @.***> wrote:

We still have overfit. Can you make a version of the data with VGG + 2 simple features

On Sun, Apr 14, 2024, 3:41 PM Gaurav Sharan Srivastava < @.***> wrote:

I have ResNet ready if you wanna use only that with other classifiers.

— Reply to this email directly, view it on GitHub https://github.com/ijyliu/computer-vision-project/issues/70#issuecomment-2054207637, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQCGE4I37WH2BLA6R7VJULDY5MA2JAVCNFSM6AAAAABF5XEG66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJUGIYDONRTG4 . You are receiving this because you were assigned.Message ID: @.***>

ijyliu commented 2 months ago

Er wait hang on let me check what's allowed

On Sun, Apr 14, 2024, 3:56 PM Isaac Liu @.***> wrote:

The two most important simple features

On Sun, Apr 14, 2024, 3:56 PM Isaac Liu @.***> wrote:

We still have overfit. Can you make a version of the data with VGG + 2 simple features

On Sun, Apr 14, 2024, 3:41 PM Gaurav Sharan Srivastava < @.***> wrote:

I have ResNet ready if you wanna use only that with other classifiers.

— Reply to this email directly, view it on GitHub https://github.com/ijyliu/computer-vision-project/issues/70#issuecomment-2054207637, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQCGE4I37WH2BLA6R7VJULDY5MA2JAVCNFSM6AAAAABF5XEG66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJUGIYDONRTG4 . You are receiving this because you were assigned.Message ID: @.***>

ijyliu commented 2 months ago

yeah vgg + two most important simple features.

On Sun, Apr 14, 2024 at 3:58 PM Isaac Liu @.***> wrote:

Er wait hang on let me check what's allowed

On Sun, Apr 14, 2024, 3:56 PM Isaac Liu @.***> wrote:

The two most important simple features

On Sun, Apr 14, 2024, 3:56 PM Isaac Liu @.***> wrote:

We still have overfit. Can you make a version of the data with VGG + 2 simple features

On Sun, Apr 14, 2024, 3:41 PM Gaurav Sharan Srivastava < @.***> wrote:

I have ResNet ready if you wanna use only that with other classifiers.

— Reply to this email directly, view it on GitHub https://github.com/ijyliu/computer-vision-project/issues/70#issuecomment-2054207637, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQCGE4I37WH2BLA6R7VJULDY5MA2JAVCNFSM6AAAAABF5XEG66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJUGIYDONRTG4 . You are receiving this because you were assigned.Message ID: @.***>

ijyliu commented 2 months ago

i would say do pca but actually it seems like the pca seems to damage test accuracy pretty consistently

On Sun, Apr 14, 2024 at 4:03 PM Isaac Liu @.***> wrote:

yeah vgg + two most important simple features.

On Sun, Apr 14, 2024 at 3:58 PM Isaac Liu @.***> wrote:

Er wait hang on let me check what's allowed

On Sun, Apr 14, 2024, 3:56 PM Isaac Liu @.***> wrote:

The two most important simple features

On Sun, Apr 14, 2024, 3:56 PM Isaac Liu @.***> wrote:

We still have overfit. Can you make a version of the data with VGG + 2 simple features

On Sun, Apr 14, 2024, 3:41 PM Gaurav Sharan Srivastava < @.***> wrote:

I have ResNet ready if you wanna use only that with other classifiers.

— Reply to this email directly, view it on GitHub https://github.com/ijyliu/computer-vision-project/issues/70#issuecomment-2054207637, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQCGE4I37WH2BLA6R7VJULDY5MA2JAVCNFSM6AAAAABF5XEG66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJUGIYDONRTG4 . You are receiving this because you were assigned.Message ID: @.***>

ijyliu commented 2 months ago

Unless the two most important includes HOG as one of them. Lol. Basically not worth it then

So I guess make the importance plot on that logistic reg data i sent and if hog is least important we can keep trying

On Sun, Apr 14, 2024, 4:04 PM Isaac Liu @.***> wrote:

i would say do pca but actually it seems like the pca seems to damage test accuracy pretty consistently

On Sun, Apr 14, 2024 at 4:03 PM Isaac Liu @.***> wrote:

yeah vgg + two most important simple features.

On Sun, Apr 14, 2024 at 3:58 PM Isaac Liu @.***> wrote:

Er wait hang on let me check what's allowed

On Sun, Apr 14, 2024, 3:56 PM Isaac Liu @.***> wrote:

The two most important simple features

On Sun, Apr 14, 2024, 3:56 PM Isaac Liu @.***> wrote:

We still have overfit. Can you make a version of the data with VGG + 2 simple features

On Sun, Apr 14, 2024, 3:41 PM Gaurav Sharan Srivastava < @.***> wrote:

I have ResNet ready if you wanna use only that with other classifiers.

— Reply to this email directly, view it on GitHub https://github.com/ijyliu/computer-vision-project/issues/70#issuecomment-2054207637, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQCGE4I37WH2BLA6R7VJULDY5MA2JAVCNFSM6AAAAABF5XEG66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJUGIYDONRTG4 . You are receiving this because you were assigned.Message ID: @.***>

CalBlitz commented 2 months ago

See the graphs above