search

maikherbig / AIDeveloper

GUI-based software for training, evaluating and applying deep neural nets for image classification
BSD 2-Clause "Simplified" License
108 stars 20 forks source link

Implement NFNets #47

Open NoMammoth opened 3 years ago

NoMammoth commented 3 years ago

Is it possible to add NFNets to the model zoo? I've watched your video on youtube but I'm afraid i'm unable to do it by myself without learning how to code first...

For your information I'm trying to determine if it's feasible (and what accuracies can be expected) to recognize 14 different metals so that these don't have to be sorted manually... Current train times are 72 hours before the accuracy flattens out. If NFNets are as fast as they claim this would reduce train time significantly.

Kind regards

maikherbig commented 3 years ago

Hi, I have stumbled across this publication last week and I really want to try it! Currently, AIDeveloper runs on Python 3.5.6 but the NFNet implementations I found use Python 3.6.9. Unfortunately, you cant run Python 3.6.9 code on Python 3.5.6 without doing quite some changes. By the way: AIDeveloper was motivated because code did break all the time, especially after updating some package. I needed something stable I can rely on. Fixing code all the time is not fun.

While NFNets seem to be very performant, I would also like to stress that all these recent/cool/modern models are heavily over-engineered. You mostly don't need them to solve real-world problems like classifying metal. I had most success by starting with very simple models. I guess trying the models "LeNet5_do" and "Nitta_et_al_6l" in AIDeveloper, would not require too much time. Maybe they already give you what you need?! If you are not happy with the resulting accuracy, you could also try some of the pre-trained models (MobileNet, ResNet,...). In my experience, the process of training a few simple models already allows you to get some insight. Most of the time I find some error e.g. in the database.

To increase the accuracy, the following methods work for me

And the following method does not work (most of the time)

Sorry that I cant solve your request right away. I want to implement NFNets, but that takes some time. I hope that I could still help you.

NoMammoth commented 3 years ago

Hi,

Thank you for the quick reply.

Currently I'm already using a rather large model (resnext101 - pretrained) and the achieved accuracy for 8 different metals is around 94%. Currently I only have 500 pictures of each metal type (350 in training set, 75 test set and 75 validation).

However I'd like to reach accuracy above 98%. I'm 100% certain there are no bugs in the current dataset. (every metal that is not clear can be tested by means of handheld XRF device). The metals that have been shredded are far from clean and sometimes have a coating to make things even harder. Even for trained people it is not easy ( and these people can pick it up to have an idea of the weight and turn it around,... ) A similar dataset would be interesting indeed but I've haven't been able to find one so far. A larger dataset is what I'm thinking of next.

Thank you for the quick reply and looking forward to the NFNets, hopefully somebody writes an implementation in python 3.5.6 soon!

By the way, thank you for making this software. It has been a great tool to use.

Kind regards.

maikherbig commented 3 years ago

During my studies (physics), I had a practical course where I took images of metal surfaces. Below, I paste one image. Is your task similar/ do your images look similar? Messing3