omni-us / research-GANwriting

Source code for ECCV20 "GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images"
MIT License
66 stars 22 forks source link

Usecase question #12

Open SB2020-eye opened 2 years ago

SB2020-eye commented 2 years ago

Hi. I don't have a deep understanding of all this, so please bear with me.

I am working hobby project with the ornate handwriting of a medieval manuscript. The manuscript is in Latin. There are no letter "j"s (i is used), no "k"s (didn't exist), no "v"s (u is used), no "w"s (didn't exist), and very few "y"s.

Would research-GANwriting be capable of doing either of the following tasks?

Produce the letters that don't exist based on the letters that do.

Produce multiple, unique instances of letters that are few in number. (The letters that exist in abundance slightly vary from one to the next, because this is handwriting. So an "e", for example, looks slightly different every time. I'm asking if MC-GAN could create more "y"s, for example, with each one slightly varying from the others, yet plausibly the product of the original scribe.)

Thank you!

hendraet commented 2 years ago

I tried to use GANwriting for a similar task and tried to synthesise numbers. From experience, I can tell you that it won't work if you don't have any numbers in your dataset and it will not work properly if you have very few numbers or a heavily imbalanced dataset. The synthesised numbers will be heavily distorted if the network is even able to synthesise anything. I guess, the same is true in your case. In order to use GANwriting, you should have at least some samples of every character in your alphabet and then try to balance your whole training dataset by using additional augmented samples. But be aware that even this augmentation will likely not lead to the desired results.