Set of DNN pre-trained models for a range of image classification domains (fashion, furnitures, ...)

[beniz]

This is the discussion / report ticket for the set of models available here: http://www.deepdetect.com/applications/model/ They include classification or clothes, furnitures, gender, planes, cars, etc... See full list and instructions on the page above.

[kmatzen]

Do you have documentation as to which photos from each synset appeared in your training and test sets?

[beniz]

@kmatzen I don't have documentation about that.

But it should be pretty easy to get the list of synsets from the model class list. I might be able to provide a list of the exact synsets in most cases if this would prove something important for some of the model users.

It's a bit more tricky for the files I believe. The list of files is likely to be stored in the training db, and from there, there may be a way to get back to the photo without providing the file itself.

It may not be of great direct help at this stage, but I've fixed a script to grab what publicly online from the full Imagenet (~85%): https://github.com/beniz/imagenet_downloader

I've actually trained from that, which may make the matching to the full Imagenet tarball more tricky even. FTR, I've had access to the full Imagenet dump sometimes in the past but they have recently resetted the credentials as it seems.

[roblkw226]

In the information about the website, you've listed that you split the dataset into training and testing. I understand you cannot share the images due to licensing. But presuming that we're looking at the same imagenet dataset, how did you decide the split? If for example, you simply chose the first 4500 to be training, and a remaining 500 to be testing, then we could do the same.

Otherwise we run the risk of using the training data in our tests, which would obviously be problematic. Is there some log somewhere of what files you used?

[beniz]

@roblkw226 good point, unfortunately the splits are random from a shuffled dataset (and no seed). I guess you would like to get an appreciation of the accuracy of one or more models, is that correct ? We could think of a few ways to help you do that.

[roblkw226]

Yes, that's right. We'd like to understand the accuracy, before and after some fine-tuning adjustments we have in mind. We'd appreciate any help in that area.

[beniz]

OK, there are several ways to get or approximate the accuracy:

• Finetune the model of interest on your own dataset. Finetuning is the primary reason behind sharing the training net, the solver file and the model.json file that contains the training calls. Having a training set of your own will allow you to control the value of any adjustment you decide to take.
• Another method I've used in the past when the test set is not available: call on Google image for the categories of interest and build your test set from there. Intersection with Imagenet and any of the training images is unlikely (sometimes it may be safer to not use the exact full length Imagenet-like class names though). Typically such a tool can easily be built based on https://github.com/asciimoo/searx
• We could run a little experiment in which I would share the testing set for one of the models and see how useful it can be in practice. If your interest mostly lies in the datasets and less in the pre-trained models and network description files, I guess we could again think of some ways around that.

If you need more lively interaction, ping me on gitter, https://gitter.im/beniz/deepdetect