Identify good picture collection to download

rom1504 / image_embeddings

Using efficientnet to provide embeddings for retrieval

https://rom1504.github.io/image_embeddings/

MIT License

151 stars 32 forks source link

Identify good picture collection to download #1

Closed rom1504 closed 4 years ago

rom1504 commented 4 years ago

At least a million of them for good measure. Would be good if they had labels (can be simple text label, no need for quality), to show something nice in the ui.

rom1504 commented 4 years ago

ideas :

download picture from google image
extract pictures from youtube videos
use image net (but that's too easy)
find some interesting picture dataset in kaggle
use a crawler to download images from website :
- an housing website
- animal pictures such as https://pixabay.com/images/search/animal/
- wikipedia pictures

problem with 1M pictures is it also means hosting it somewhere. Might be good to keep the url so it can be used for visualization ?

rom1504 commented 4 years ago

might be reasonable to start with a simple example with 100 pictures, use that as basic example and expand to the more complete example afterwards

rom1504 commented 4 years ago

https://lionbridge.ai/datasets/top-10-image-classification-datasets-for-machine-learning/ https://www.tensorflow.org/datasets/catalog/sun397

rom1504 commented 4 years ago

plan :

use https://www.tensorflow.org/datasets/catalog/sun397
download it
select 100 decent images of it
run inference
commit the 100 images and 100 embeddings
build basic python knn and js knn
then run fast inference on all 100k images
try python knn on them
put screenshots in readme
write blog post

rom1504 commented 4 years ago

using tf_flowers instead as sun397 is too big for a simple example

rom1504 commented 4 years ago

bootstrapped in https://github.com/rom1504/image-embeddings/blob/master/ImageEmb.ipynb Next steps :

[x] put that in clean .py files
[x] create setup.py / requirements.txt
[x] create clean README
[x] consider putting on pypi
[x] screenshot in readme
[ ] js knn
[ ] blogpost

Other things to consider :

[ ] provide other ways to get pictures (add a simple resizer that work for generic picture folder)

rom1504 commented 4 years ago

Remaining :

[x] inference in .py (in api and cli)
[x] knn in .py (in api and cli)
[x] get screenshots for readme and blogpost
[ ] js knn

rom1504 commented 4 years ago

done opening issues for the rest (js knn and blogpost)