search

isl-org / StableViewSynthesis

MIT License
211 stars 34 forks source link

Leave-one-out rendering #8

Open grgkopanas opened 3 years ago

grgkopanas commented 3 years ago

Hi,

I am trying to do a leave-one-out rendering for the input cameras, I have been doing it this way: In this line https://github.com/intel-isl/StableViewSynthesis/blob/main/experiments/dataset.py#L263 I am adding a nbs.remove(idx) before passing it to the next function.

The results are rather extremely blurry, am I doing something wrong?

All the best, George Kopanas

griegler commented 3 years ago

You should be able to just use/adapt this method. It creates an Dataset object with mode=train, which is exactly the leave-one-out behaviour that you want.

griegler commented 3 years ago

In more detail, add an entry here with some name that calls a modified (such that it loads your models) get_train_set_tat method. Then you can just use the provided code for evaluation and use your evaluation set name.

grgkopanas commented 3 years ago

Hi,

Thanks for the reply. I managed to use get_train_set_tat but I am quickly running out of GPU memory in train mode in a 16GB GPU. With get_eval_set_tat I can render just fine.

Is there any quick and dirty way to use less GPU with get_train_set_tat ?

Cheers, George

griegler commented 3 years ago

What do you use for n_nbs? You could try to reduce the number.

grgkopanas commented 3 years ago

I had that at default which seems to be 3. Seems already pretty small to me right?

griegler commented 3 years ago

Yes, that should be more than feasible. Batch size is set to 1?

grgkopanas commented 3 years ago

Batch size is 1, both for eval and train. I am attaching my git diff in case you notice something. Unfortunately I don't have a GPU with bigger memory available.

svs_diff.txt and my command line arguments are: --net resunet3.16_penone.dirs.avg.seq+9+1+unet+5+2+16.single+mlpdir+mean+3+64+16 --cmd eval --iter last --eval-dsets tat-scene-hugo --eval-scale 0.45

Thanks again, George

akashsharma02 commented 2 years ago

In more detail, add an entry here with some name that calls a modified (such that it loads your models) get_train_set_tat method. Then you can just use the provided code for evaluation and use your evaluation set name.

Hi @griegler Could you point to the code which removes the target image from the neighbors when the Dataset object is created with mode=train?

I was able to adapt my dataset and run evaluation, however I was wondering if there is actually leave one out rendering occuring? If the target image exists in the dataset, wouldn't the unprojected points on the mesh, always take points from the very same target image, or could you please clarify, if I understand something incorrectly.

Perhaps, I need to define a subseq and explicitly add indices that are to be skipped in the source images, is that correct?