Running DiffProxy on Linux

[liangs111]

Hi, thanks for publishing the code of this work. I am trying to run this code on Ubuntu 20.04. I installed the stylegan3 environment and set the sat_dir to my automation toolkit installation location. Then, I try to synthesize the dataset by running python synthesis.py --data_path=./data/sbs --generator_name=arc_pavement --n_samples=307200 (the commend in readme misses a dash for the n_samples) and the following error message is what I got.

In Node 509, found unregistered params: {'pattern_width_random', 'global_pattern_width_random', 'pattern_scale', 'pattern_amount', 'pattern_width', 'color_random', 'pattern_height_random', 'pattern_height_decrease', 'pattern_height', 'arcs_amount'} In Node 510, found registered params:set() In Node 510, found unregistered params: {'pattern_width_random', 'global_pattern_width_random', 'pattern_scale', 'pattern_amount', 'pattern_width', 'color_random', 'pattern_height_random', 'pattern_height_decrease', 'pattern_height', 'arcs_amount'} In Node 511, found registered params:set() In Node 511, found unregistered params: {'pattern_width_random', 'global_pattern_width_random', 'pattern_scale', 'pattern_amount', 'pattern_width', 'color_random', 'pattern_height_random', 'pattern_height_decrease', 'pattern_height', 'arcs_amount'} Begin Sampling Traceback (most recent call last): File "/home/liangs/anaconda3/lib/python3.9/shutil.py", line 823, in move os.rename(src, real_dst) FileNotFoundError: [Errno 2] No such file or directory: './data/sbs/arc_pavement/arc_pavement/tmp/baseColor.png' -> './data/sbs/arc_pavement/arc_pavement/00000000.png'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/liangs/research/DiffProxy/synthesis.py", line 48, in synthesize_data() File "/home/liangs/research/DiffProxy/synthesis.py", line 37, in synthesize_data sampler.sample(output_dir, n_samples=n_samples, vis_every=vis_every) File "/home/liangs/research/DiffProxy/sbs_generators.py", line 368, in sample image_names_list = self.save_sample(output_graph_filename, output_dir, i) File "/home/liangs/research/DiffProxy/sbs_generators.py", line 479, in save_sample shutil.move(image_filename, image_name) File "/home/liangs/anaconda3/lib/python3.9/shutil.py", line 843, in move copy_function(src, real_dst) File "/home/liangs/anaconda3/lib/python3.9/shutil.py", line 444, in copy2 copyfile(src, dst, follow_symlinks=follow_symlinks) File "/home/liangs/anaconda3/lib/python3.9/shutil.py", line 264, in copyfile with open(src, 'rb') as fsrc: FileNotFoundError: [Errno 2] No such file or directory: './data/sbs/arc_pavement/arc_pavement/tmp/baseColor.png'

It seems like the subprocess.run of sbsrender didn't produce the SVBRDF outputs. Can you help me figure out the cause of the above error? Thank you!

[yiwei-hu]

The code was tested on a Windows machine, but it should also work on Linux. I would suggest you to try a newer version of substance automation toolkit. It was tested on 12.1.0 version.

[jiayaozhang]

When I run this train.py and got this below error

python stylegan/train.py --cfg stylegan2 --outdir=./training-runs --data=./data/train/arc_pavement_300k.zip --generator_name=arc_pavement --gpus=1 --batch=8 --gamma=10 --map-depth=4 --glr=0.0025 --dlr=0.001 --cbase=16384 --snap=50 --cond=true --aug=noaug --kimg=10000 --metrics=none --mirror=false --norm_type=norm --no_gan=True --cond_d=True

Setting up training phases...
Distributing across 1 GPUs...
Traceback (most recent call last):
  File "/home/janice/Documents/DiffProxy/stylegan/train.py", line 309, in <module>
    main() # pylint: disable=no-value-for-parameter
  File "/home/janice/anaconda3/envs/MeshDiffusion/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/janice/anaconda3/envs/MeshDiffusion/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/janice/anaconda3/envs/MeshDiffusion/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/janice/anaconda3/envs/MeshDiffusion/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/janice/Documents/DiffProxy/stylegan/train.py", line 304, in main
    launch_training(c=c, desc=desc, outdir=opts.outdir, dry_run=opts.dry_run, no_gan=opts.no_gan)
  File "/home/janice/Documents/DiffProxy/stylegan/train.py", line 100, in launch_training
    subprocess_fn(rank=0, c=c, temp_dir=temp_dir, no_gan=no_gan)
  File "/home/janice/Documents/DiffProxy/stylegan/train.py", line 48, in subprocess_fn
    training_loop_nogan.training_loop(rank=rank, **c)
  File "/home/janice/Documents/DiffProxy/stylegan/training/training_loop_nogan.py", line 165, in training_loop
    loss = dnnlib.util.construct_class_by_name(device=device, G=G, D=None, augment_pipe=augment_pipe, **loss_kwargs) # subclass of training.loss.Loss
  File "/home/janice/Documents/DiffProxy/stylegan/dnnlib/util.py", line 303, in construct_class_by_name
    return call_func_by_name(*args, func_name=class_name, **kwargs)
  File "/home/janice/Documents/DiffProxy/stylegan/dnnlib/util.py", line 296, in call_func_by_name
    func_obj = get_obj_by_name(func_name)
  File "/home/janice/Documents/DiffProxy/stylegan/dnnlib/util.py", line 289, in get_obj_by_name
    module, obj_name = get_module_from_obj_name(name)
  File "/home/janice/Documents/DiffProxy/stylegan/dnnlib/util.py", line 260, in get_module_from_obj_name
    importlib.import_module(module_name) # may raise ImportError
  File "/home/janice/anaconda3/envs/MeshDiffusion/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'loss'

Can you help with this? And hope this work could be spread and help all the artists to improve their work efficiency.

[yiwei-hu]

Hi @jiayaozhang , it seems an import error. I am not very sure how this happens. It might be platform/system dependent. I would recommend you try to explicitly add the path of that module into your system path. For example,

import sys
sys.path.append('your_path_to_the_[./stylegan/training]_folder')

If you add such piece of code before you import this module i.e., loss = dnnlib.util.construct_class_by_name(...), I guess this error should be addressed.

[jiayaozhang]

Thanks, after below scripts I am able to run on windows and linux both successfully🎉

1. conda create -n DiffProxy python=3.7
2. pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
3. conda install -c anaconda scikit-image scikit-learn
4. pip install ninja click requests psutil
5. copy and paste sbs_generator.py and util.py into C:\Users\Admin\miniconda3\envs\diffproxy\Lib (Windows) and \home\anaconda\env\diffproxy\lib\python3.7\ (Linux)
6. Add these lines in C:\Users\Admin\miniconda3\envs\Diffproxy\Lib\importlib\__init__.py

   import sys
   sys.path.append('D:\Research\DiffProxy')
   sys.path.append('D:\Research\DiffProxy\stylegan\training')