train english network

[ShaihW]

Hi, First, thanks for this great repository

I am trying to train such a diffusion network for english font I trained style encoder that works fine I plugged it into the diffusion model, removed the strokes (my cfg file does not contain "stroke_path: " line), supplied english chars in gen_char.txt and total_eng.txt (replaces of total_chn.txt file) and of course supplied my english dataset

now the problem is that the result is bad. I also suspect that it looks a bit Chinese

I will appreciate any help with this

[Hxyz-123]

Hello, can you provide the cfg file and several generated images?

[ShaihW]

Thank you for the answer Attached 2 examples of the same epoch (but different style image was fed to the network) I concatenated the 10 results to one figure. of 2 rows and 5 columns

the training cfg is:

data_dir: '/home/shai/Font-diff/eng_ch_a/'
chara_nums: 62
diffusion_steps: 1000
noise_schedule: 'linear'
image_size: 80
num_channels: 128
num_res_blocks: 3
lr: 0.0002
batch_size: 8
log_interval: 250
save_interval: 30000
train_step: 420000
attention_resolutions: '40, 20, 10'
sty_encoder_path: './pretrained_models/cont_model_78.ckpt' #### trained with DG-Font
model_save_dir: './treng_trained_models'
classifier_free: False
total_train_step: 800000
resume_checkpoint: ""

test cfg:

dropout: 0.1
chara_nums: 62
diffusion_steps: 1000
noise_schedule: 'linear'
image_size: 80
num_channels: 128
num_res_blocks: 3
batch_size: 5
num_samples: 10
attention_resolutions: '40, 20, 10'
use_ddim: False
timestep_respacing: ddim25

model_path: '/home/shai/Font-diff/treng_trained_models/model540000.pt'
sty_img_path: '/home/shai/Font-diff/eng_chars/id_9936/00001.png'
total_txt_file: './total_en.txt'
gen_txt_file: './en_char.txt'
img_save_path: './result_img_EN_5'
classifier_free: True
cont_scale: 3.0
sk_scale: 3.0

[Hxyz-123]

You can set the 'classifier_free' to 'False' in test cfg and try it again.

[mmmmmjaai]

may i have question? how did you get the file "cont_model_78.ckpt" Is get from the code, if i wanna to get the sty_encoder,how should i do ,thanks a lot

[ShaihW]

may i have question? how did you get the file "cont_model_78.ckpt" Is get from the code, if i wanna to get the sty_encoder,how should i do ,thanks a lot

Trained it using DG-Font

[jonhilgart22]

Any luck with this?

[shiyongde]

same problem

[shiyongde]

fixed for "set the 'classifier_free' to 'False' in test cfg"

[bibinkunjumon]

may i have question? how did you get the file "cont_model_78.ckpt" Is get from the code, if i wanna to get the sty_encoder,how should i do ,thanks a lot

Trained it using DG-Font

That not clarifying the problem. I also need to apply this for English Font.Whats the method? 2 problems i am stuck with in your code : sty_encoder_path: './pretrained_models/chinese_styenc.ckpt' # path to pre-trained style encoder

Where to get an ENglish style encoder ?

stroke_path: './chinese_stroke.txt' # encoded strokes

How to make english stroke set like this file with 1s and 0s

[Outlanderll]

Thank you for the answer Attached 2 examples of the same epoch (but different style image was fed to the network) I concatenated the 10 results to one figure. of 2 rows and 5 columns

the training cfg is:

data_dir: '/home/shai/Font-diff/eng_ch_a/'
chara_nums: 62
diffusion_steps: 1000
noise_schedule: 'linear'
image_size: 80
num_channels: 128
num_res_blocks: 3
lr: 0.0002
batch_size: 8
log_interval: 250
save_interval: 30000
train_step: 420000
attention_resolutions: '40, 20, 10'
sty_encoder_path: './pretrained_models/cont_model_78.ckpt' #### trained with DG-Font
model_save_dir: './treng_trained_models'
classifier_free: False
total_train_step: 800000
resume_checkpoint: ""

test cfg:

dropout: 0.1
chara_nums: 62
diffusion_steps: 1000
noise_schedule: 'linear'
image_size: 80
num_channels: 128
num_res_blocks: 3
batch_size: 5
num_samples: 10
attention_resolutions: '40, 20, 10'
use_ddim: False
timestep_respacing: ddim25

model_path: '/home/shai/Font-diff/treng_trained_models/model540000.pt'
sty_img_path: '/home/shai/Font-diff/eng_chars/id_9936/00001.png'
total_txt_file: './total_en.txt'
gen_txt_file: './en_char.txt'
img_save_path: './result_img_EN_5'
classifier_free: True
cont_scale: 3.0
sk_scale: 3.0

Hello，I would like to ask why I get an error like the following when I use a cpkt trained by DG-Font？I would appreciate a reply from you! Traceback (most recent call last): File "/home/llll/PycharmProjects/Fontdiffproject/Font-diff/train.py", line 122, in main() File "/home/llll/PycharmProjects/Fontdiffproject/Font-diff/train.py", line 56, in main model.sty_encoder.load_state_dict(tmp_dict) File "/home/llll/anaconda3/envs/fontdiffuser/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1671, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for StyleEncoder: Missing key(s) in state_dict: "features.0.weight", "features.0.bias", "features.1.weight", "features.1.bias", "features.1.running_mean", "features.1.running_var", "features.4.weight", "features.4.bias", "features.5.weight", "features.5.bias", "features.5.running_mean", "features.5.running_var", "features.8.weight", "features.8.bias", "features.9.weight", "features.9.bias", "features.9.running_mean", "features.9.running_var", "features.11.weight", "features.11.bias", "features.12.weight", "features.12.bias", "features.12.running_mean", "features.12.running_var", "features.15.weight", "features.15.bias", "features.16.weight", "features.16.bias", "features.16.running_mean", "features.16.running_var", "features.18.weight", "features.18.bias", "features.19.weight", "features.19.bias", "features.19.running_mean", "features.19.running_var", "features.22.weight", "features.22.bias", "features.23.weight", "features.23.bias", "features.23.running_mean", "features.23.running_var", "features.25.weight", "features.25.bias", "features.26.weight", "features.26.bias", "features.26.running_mean", "features.26.running_var", "cont.weight", "cont.bias".