hysts / anime-face-detector

Anime Face Detector using mmdet and mmpose
MIT License
416 stars 24 forks source link

Re-thinking anime(Illustration/draw/manga) character face detection #3

Closed koke2c95 closed 2 years ago

koke2c95 commented 3 years ago

awesome work!

especially face clustering very neat

this work reminds me of

How can Illustration be aligned and what can I do with these 2d landmark?

Scaling and rotating images and crop: FFHQ aligned code and webtoon result

Artstation-Artistic-face-HQ which counts as Illustration Use FFHQ aligned

and new FFHQ aligned https://arxiv.org/abs/2109.09378

but anime Illustration is not the same as real FFHQ, where perspective-related (pose) means destroying the centre, and local parts exaggeration destroying the global

[DO.1] directly k-mean dictionary (run a dataset) proximity aligned

Mention this analysis

[DO.2] because there are not many features can use, add continuous 2D spatial feature (pred), more point and even beyond

this need hack model (might proposed)

[DO.3] Should be used directly as a filter to assist with edge extraction (maximum reserve features)

guide VAE, SGF generation, or anime cross image Synthesis

if the purpose is not to train the generation model, probably use is to extend the dataset. if training to generate models, will greatly effect generated eye+chin centre aligned visual lines don't keeping real image features just polylines

Or need more key points in clustering, det box pts (easy [DO.4]), and beyond to the whole image

and thank for your reading this

hysts commented 3 years ago

Interesting! It seems there are a lot of related works and areas. I knew some of them, but I didn't know the others. Thank you for letting me know. I'll check them out when I have time. By the way, I trained these models, as I wanted to try training StyleGAN with aligned anime face images, but couldn't find a good landmark model.