Exposing SIREN's theta_initial and theta parameters.

afiaka87 commented 3 years ago

Update: The initial image generated for delta 1,2,3,4,5,...41 https://user-images.githubusercontent.com/3994972/106395379-9f77c380-63c7-11eb-96ba-ccda3da22334.mp4

As dicussed in another issue, deep-daze is sometimes overly sensitive to its initial input. In searching for a lead on how to resolve this issue, I decided to expose the SIREN hyperparameter denoted theta (I think) in the original notebook. So far I've found some interesting results, but I'm still working out what of it is useful. It's probably just best if I show you some examples of different theta on the same text input. For reference, I'm using @dginev seed and text, as he mentioned not being able to get a "cheery" output out of the rather cheery phrase "seed of hope". You can find that discussion here: https://github.com/lucidrains/deep-daze/issues/9#issuecomment-763249044

--seed=872073 text: "seed of hope" iterations: 1000 epochs: 1 image_width: 256 num_layers: 32 batch_size: 64 gradient_accumulate_every: 1

This is the code I'm using to run through different theta parameters for the same input. Careful, this code will run 180 full 1000 iteration runs.

#!/bin/bash

for ((theta=1;theta<180;theta++)); do
  echo "processing theta: " + $theta;
  imagine "seed of hope"\
     --seed=872072 \
     --num_layers=32 \
     --image_width=256 --save_progress=True --save_every=10 --epochs=1 \
     --batch_size=32 --gradient_accumulate_every=1 \
     --iterations=1000
     --overwrite=True \
     --theta_initial=$theta \
     --theta_hidden=$theta \
     --save_date_time=True;
  wait; # Important, otherwise the loop will continue before finishing and you'll run out of memory.
done

Here's the code if you'd like to try yourself. @lucidrains Would love your input on this. Not sure it's ready to be merged quite yet but let me know if you're interested in that. https://github.com/afiaka87/deep-daze/tree/theta_output_dir_params

Early results:

This is what I was able to run on my RTX 2070 this morning. Each image represents a change in the theta value by 1. The final image is theta=56. As you can see, it definitely gives different results. But I'm still not sure what to make of it.

1_seed_of_hope 2_seed_of_hope 3_seed_of_hope 4_seed_of_hope 5_seed_of_hope 6_seed_of_hope 7_seed_of_hope 8_seed_of_hope 9_seed_of_hope 10_seed_of_hope 11_seed_of_hope 12_seed_of_hope 13_seed_of_hope 14_seed_of_hope 15_seed_of_hope 16_seed_of_hope 17_seed_of_hope 18_seed_of_hope 19_seed_of_hope 20_seed_of_hope 21_seed_of_hope 22_seed_of_hope 23_seed_of_hope 24_seed_of_hope 25_seed_of_hope 26_seed_of_hope 27_seed_of_hope 28_seed_of_hope 29_seed_of_hope 30_seed_of_hope 31_seed_of_hope 32_seed_of_hope 33_seed_of_hope 34_seed_of_hope 35_seed_of_hope 36_seed_of_hope 37_seed_of_hope 38_seed_of_hope 39_seed_of_hope 40_seed_of_hope 41_seed_of_hope 42_seed_of_hope 43_seed_of_hope 44_seed_of_hope 45_seed_of_hope 47_seed_of_hope 48_seed_of_hope 49_seed_of_hope 51_seed_of_hope 54_seed_of_hope 55_seed_of_hope 56_seed_of_hope

dginev commented 3 years ago

It may end up useful... I can now use the new setting allowing me to start off from a pre-existing image, so can see the generation starting off with very bright and "cheery" imagery when seeded accordingly. But, somewhat mysteriously, as the generation progresses there is a noticeable slide back into darker hues, even for super-positive keywords (sunny, summer, angel, etc). The technical causes are beyond me, but one wonders if this parameter may not be a part of the answer?

lucidrains commented 3 years ago

@afiaka87 @dginev released in 0.4.2! https://github.com/lucidrains/deep-daze/commit/0e879593cdecada727a40ddf7a4f55b58651f35c

afiaka87 commented 3 years ago

Much obliged.

lucidrains / deep-daze

Exposing SIREN's theta_initial and theta parameters. #36

Early results: