Open hamzapehlivan opened 2 years ago
Oh, you need to specify '--resolution=512'
Thanks for the reply, it now worked!
However, I was not able to replicate the results of the paper. In the paper, the followings were reported for Places2 dataset.
P-IDS: 13.3 ±0.1, U-IDS: 27.4 ±0.1, FID: 7.9 ±0.0
What I got: P-IDS 11.6, U-IDS 25.9 , FID 8.0788
Running it several times also led to the same slightly inaccurate results.
I have the same issue with FFHQ Dataset, too.
One more point is that I got very similar results to paper when I specified the masked ratio between 0 and 0.2
Any idea why this happens?
Quite strange. What is your result for FFHQ?
For the metric ids10k-h0
, I am getting:
P-IDS: 30.0, U-IDS 44.5, FID: 0.5483
For the metric
ids10k
, I am getting: P-IDS:14.8, U-IDS: 28.2, FID: 3.8678
What is your numpy and tf version?
My TF version is "1.15.0", and my Numpy version is "1.17.3".
I am using the provided Dockerfile. However, metric evaluation was not working, therefore, I changed it a bit:
FROM tensorflow/tensorflow:1.15.0-gpu-py3 RUN pip install scipy==1.3.3 \ requests==2.22.0 \ Pillow==6.2.1 \ tqdm \ imageio \ scikit-learn
If it helps, here is my log.txt:
dnnlib: Running run_metrics.run() on localhost... Evaluating metrics "ids10k" for "models/co-mod-gan-ffhq-9-025000.pkl"... Setting up TensorFlow plugin "fused_bias_act.cu": Preprocessing... Loading... Done. Setting up TensorFlow plugin "upfirdn_2d.cu": Preprocessing... Loading... Done. truncation=None 100%|##########| 1250/1250 [08:05<00:00, 2.57it/s] co-mod-gan-ffhq-9-025000 time 9m 42s ids10k-FID 3.8398 ids10k-U 0.2777 ids10k-P 0.1442 dnnlib: Finished run_metrics.run() in 9m 56s.
I still have no idea what causes this error... One hypothesis is that the randomness of the mask generator's behavior has changed, which may slightly bias the difficulty of the task
Hi,
Thanks for sharing this great work!
I have a question about running
run_metrics.py
with Places2 dataset.As it is indicated in Datasets section, I downloaded the validation set of Places2, and converted into TFRecords with
--shuffle --compressed
flags.However, when I try to run `run_metrics.py' (I set metrics=idk36k5), I got the following:
When I debug the code, I realized that
tfr_shapes=[[74989]]
intraining/dataset.py
line 82. Here is the "features" dictionary without bytes_list:I am using the provided Docker image.