Reusing the same synthesizer

X-rayLaser / pytorch-handwriting-synthesis-toolkit

Handwriting generation and handwriting synthesis as described in Alex Graves's paper https://arxiv.org/abs/1308.0850. Pytorch implementation.

MIT License

66 stars 11 forks source link

Reusing the same synthesizer #8

Open daphna-kaplan opened 1 year ago

daphna-kaplan commented 1 year ago

I am looking for a way to save the created synthesizer and then re-use the same handwriting a few times. Is it possible? it seems to make sense that it is possible but I am not familiarized enough to know how.

X-rayLaser commented 1 year ago

Yes. I think it is possible, but I need to look into code and the paper to be certain.

I vaguely recall that there is a technique that allows to specify the style of handwriting (called priming, if I remember correctly). In a nutshell, it sort of configures neural net to mimic a particular handwriting style captured in the presented sample. The technique requires a sample of handwriting (that has the style one would wish to mimic) as well as its corresponding prompt/transcript.

Actually, the script txt2script (which generates a multi-line handwriting page) makes use of priming, but in a slightly different way. It generates the first line in a random style, then it synthesizes the following lines in the style of the first one.

X-rayLaser commented 1 year ago

From the usage point of view, one way to support this feature is the following:

add a boolean flag called "--export_style" to the synthesize.py command; when the flag is set, for each normal synthesized image the script will additionally create a style file
add an optional argument called "--style" to the synthesize.py command that specifies path to the exported style; when style is provided, the script will synthesize the handwriting using that style

The user could generate a bunch of handwritings (with corresponding style files), pick the one they like the most and reuse it's style for generating other handwriting. Of course, this is not the only possible workflow and I am open to suggestions.

msecchi3 commented 11 months ago

Hi everyone, very interesting and valuable repository. I'm trying to follow the recommended instructions to produce a single type of output but I see that I can't create the folder to save the style outputs. Can you give me some suggestions in order to choose the desired output style? Thanks in advance.

msecchi3 commented 11 months ago

to whom it may concern, I solved my issue by taking inspiration from the text_to_script function and adapting it to my needs. Thank you!