search

ImageMonkey / imagemonkey-core

ImageMonkey is an attempt to create a free, public open source image dataset.
https://imagemonkey.io
47 stars 10 forks source link

(info - interesting use case) #220

Open dobkeratops opened 5 years ago

dobkeratops commented 5 years ago

https://www.turingtribe.com/story/nvidia-deep-learning-model-gaugan-turns-doodles-into-stunning-photorealistic-landscapes-ohKw2EkeQvvwgZrEt?fbclid=IwAR1Rf8VndFSNNux-UnMs8iKLxNxtn3MexVFHYAkBNF6_7Lpv8pPYqB7E1jY

This is the kind of application that interests me.. hence my interest in expanding the vocabulary for surface types, incidental objects etc. I'm sure NN's can revolutionise creating textured assets for games.

I haven't read through the detail yet but it seems to involve a mix of GANN's and labelling (-> to allow artist to select what to fill areas with)

Of course results like they have are data and processing intensive ("trained using a million images" ..) . (they did say 'a million open source images' .. but did they have segmentation available or just per image descriptions? and images from where?).

But they're going for photorealistic results starting with simple areas.

There might be a useable middle ground , e.g. enhancing cartoony/stylized images.. enhancing low-poly art/pixel art/voxel-art. Or filling in the gaps for photogrammetry ("what does the other side of this building look like").

My hope is that you could get to a similar level if you have many domain specific labellers overlapping in the same dataset.. we have to start somewhere .

Nvidia of course have their pricey DGX multi-GPU machines to train their models aswell..

bbernhard commented 5 years ago

Thanks a lot for sharing, that looks very interesting!

Unfortunately, I couldn't find neither the dataset nor the sourcecode online, so I guess they want to keep it private for monetizing reasons. Would have been really interesting to see how they pulled that off.

This is the kind of application that interests me.. hence my interest in expanding the vocabulary for surface types, incidental objects etc. I'm sure NN's can revolutionise creating textured assets for games.

btw: I want to create another brainstorming/project planning thread in the next few days, so if there is a particular direction you wanna go with the dataset, please let me know. :) It's been a while since the last project planning/brainstorming thread and I think it might make sense to do something similar for 2019. I think the last goal ("reaching the 20k mark") went really well (mainly due to the huge number of images you've scraped & uploaded ;)), so I hope we find a similar motivating goal for 2019 :)

dobkeratops commented 5 years ago

awesome , let me think a bit more before rambling too much here. Plenty of suggestions and the iPad/pencil keeps using this fresh