XiangLi1999 / Diffusion-LM

Diffusion-LM
Apache License 2.0
1.02k stars 133 forks source link

about {path-to-diffusion-lm} #34

Open lcy5058 opened 1 year ago

lcy5058 commented 1 year ago

When finally generating controllable text, python scripts/infill.py --model_path {path-to-diffusion-lm} command, which path does "path-to-diffusion-lm" refer to, the first step model path, or the last but one model generated from the classifier? "improved diffusion/diffusion_models/diff_e2e-tgt_block_rand16_transformer_lr0.0001_0.0_2000_sqrt_Lsimple_h128_s2_d0.1_sd102_xstart_e2e" or "Classifier_models/e2e-tgt-tree_e=6_b=10_m=bert base uncased_wikitext-103-raw-v1_101_wp_None"

JamesL404 commented 1 year ago

Hello, May I ask what path did you use in the decode Diffusion-LM part?

lcy5058 commented 1 year ago

Hello, May I ask what path did you use in the decode Diffusion-LM part?

improved diffusion/diffusion_models/diff_e2e-tgt_block_rand16_transformer_lr0.0001_0.0_2000_sqrt_Lsimple_h128_s2_d0.1_sd102_xstart_e2e

JamesL404 commented 1 year ago

Hello, May I ask what path did you use in the decode Diffusion-LM part?

improved diffusion/diffusion_models/diff_e2e-tgt_block_rand16_transformer_lr0.0001_0.0_2000_sqrt_Lsimple_h128_s2_d0.1_sd102_xstart_e2e

I am using the same path but the printed output list is empty. Have you met the same problem?

lcy5058 commented 1 year ago

Hello, May I ask what path did you use in the decode Diffusion-LM part?

improved diffusion/diffusion_models/diff_e2e-tgt_block_rand16_transformer_lr0.0001_0.0_2000_sqrt_Lsimple_h128_s2_d0.1_sd102_xstart_e2e

I am using the same path but the printed output list is empty. Have you met the same problem? The path is "diffusion_models/diff_e2e-tgt_block_rand16_transformer_lr0.0001_0.0_2000_sqrt_Lsimple_h128_s2_d0.1_sd102_xstart_e2e"

XiangLi1999 commented 1 year ago

there is nothing special about batch_decode. It's a helper function that calls the right command. If it doesn't work, my recommendation is to run the command directly:

python scripts/text_sample.py --model_path diffusion_models/diff_roc_pad_rand128_transformer_lr0.0001_0.0_2000_sqrt_Lsimple_h128_s2_d0.1_sd101_xstart_e2e/ema_0.9999_100000.pt --batch_size 50 --num_samples 50 --top_p 1.0 --out_dir genout

if you want to figure out why it doesn't work, I guess (since you return empty list), there might be something about this try/except https://github.com/XiangLi1999/Diffusion-LM/blob/759889d58ef38e2eed41a8c34db8032e072826f4/improved-diffusion/scripts/batch_decode.py#L16

XiangLi1999 commented 1 year ago

to answer another question about which model path to use for controllable generation. you use the Diffusion-LM model path (the first one). To pass in the control classifier path, you might want to modify in the code (sorry for the bad coding practice, I will clean this up in the next version of the code release), e.g., you can search from the string "from_pretrained" to narrow the search space. In the case of Tree, it should be line 316. https://github.com/XiangLi1999/Diffusion-LM/blob/759889d58ef38e2eed41a8c34db8032e072826f4/improved-diffusion/scripts/infill.py#L316

AI7Xiao commented 5 months ago

Hello! Thank you very much for sharing. I would like to ask which file are you using in the path: "diffusion_models/diff_e2e-tgt_block_rand16_transformer_lr0.0001_0.0_2000_sqrt_Lsimple_h128_s2_d0.1_sd102_xstart_e2e" that you mentioned? There are five files here: log, progress, random_emb.torch, training_args.json, vocab.json.

I only enter the path "diffusion_models/diff_e2e-tgt_block_rand16_transformer_lr0.0001_0.0_2000_sqrt_Lsimple_h128_s2_d0.1_sd102_xstart_e2e" , the following error will occur: IsADirectoryError: [Errno 21] Is a directory: 'diffusion_models/diff_e2e-tgt_block_rand16_transformer_lr0.0001_0.0_2000_sqrt_Lsimple_h128_s2_d0.1_sd102_xstart_e2e'

I would appreciate it if you could take some time to answer my questions. Thanks.

maitekakaxi commented 4 months ago

I had the same problem as above. I would appreciate it if you could take some time to answer my questions. Thanks.