Segmentation Fault

[Hitesh-Nagothu]

Getting segmentation fault while trying to run the 'python demo_imitator.py --gpu_ids 1'.

Config: Pytorch 1.2/ CUDA 10.0

[Hitesh-Nagothu]

Providing below the trace on: python demo_imitator.py --gpu_ids 0

Config on HPC:

Device Name Tesla V100-PCIE-16GB Get Device Count 2 Current Device 0 Is CUDA available(torch.cuda.is_available()) :True -----------------------------------Trace------------------------------------ ------------ Options ------------- T_pose: False batch_size: 4 bg_ks: 13 bg_model: ./outputs/checkpoints/deepfillv2/net_epoch_50_id_G.pth bg_replace: False body_seg: False cam_strategy: smooth checkpoints_dir: ./outputs/checkpoints/ cond_nc: 3 data_dir: /p300/datasets/iPER dataset_mode: iPER debug: False do_saturate_mask: False face_model: assets/pretrains/sphere20a_20171020.pth front_warp: False ft_ks: 3 gen_name: impersonator gpu_ids: 0 has_detector: False hmr_model: assets/pretrains/hmr_tf2pt.pth image_size: 256 images_folder: images_HD ip: is_train: False load_epoch: 0 load_path: ./outputs/checkpoints/lwb_imper_fashion_place/net_epoch_30_id_G.pth map_name: uv_seg model: impersonator n_threads_test: 2 name: running norm_type: instance only_vis: False output_dir: ./outputs/results/ part_info: assets/pretrains/smpl_part_info.json port: 31100 post_tune: False pri_path: ./assets/samples/A_priors/imgs repeat_num: 6 save_res: False serial_batches: False smpl_model: assets/pretrains/smpl_model.pkl smpls_folder: smpls src_path: swap_part: body test_ids_file: val.txt tex_size: 3 tgt_path: time_step: 10 train_ids_file: train.txt uv_mapping: assets/pretrains/mapper.txt view_params: R=0,90,0/t=0,0,0 -------------- End ---------------- ./outputs/checkpoints/running 0%| | 0/3 [00:00<?, ?it/s]Network impersonator was created Loading net: ./outputs/checkpoints/lwb_imper_fashion_place/net_epoch_30_id_G.pth Network deepfillv2 was created Loading net: ./outputs/checkpoints/deepfillv2/net_epoch_50_id_G.pth

                    Personalization: meta imitation...

Segmentation fault

[piaozhx]

have you ever install neural-renderer successfully? Segmentation fault always happens by this reason

[Hitesh-Nagothu]

Yes I have installed neural-renderer successfully

[Hitesh-Nagothu]

Please reference the below trace, I was able to successfully install and import neural_renderer. (Running demo_imitator.py)

_Config on HPC:

Device Name Tesla V100-PCIE-16GB Get Device Count 2 Current Device 0 Is CUDA available(torch.cuda.isavailable()) :True

(lwgan) [hiteshna@xyz impersonator-master]$ python -c "import neural_renderer; print(neural_renderer.version); " 1.1.3 (lwgan) [hiteshna@xyz impersonator-master]$ python demo_imitator.py --gpu_ids 0 ------------ Options ------------- T_pose: False batch_size: 4 bg_ks: 13 bg_model: ./outputs/checkpoints/deepfillv2/net_epoch_50_id_G.pth bg_replace: False body_seg: False cam_strategy: smooth checkpoints_dir: ./outputs/checkpoints/ cond_nc: 3 data_dir: /p300/datasets/iPER dataset_mode: iPER debug: False do_saturate_mask: False face_model: assets/pretrains/sphere20a_20171020.pth front_warp: False ft_ks: 3 gen_name: impersonator gpu_ids: 0 has_detector: False hmr_model: assets/pretrains/hmr_tf2pt.pth image_size: 256 images_folder: images_HD ip: is_train: False load_epoch: 0 load_path: ./outputs/checkpoints/lwb_imper_fashion_place/net_epoch_30_id_G.pth map_name: uv_seg model: impersonator n_threads_test: 2 name: running norm_type: instance only_vis: False output_dir: ./outputs/results/ part_info: assets/pretrains/smpl_part_info.json port: 31100 post_tune: False pri_path: ./assets/samples/A_priors/imgs repeat_num: 6 save_res: False serial_batches: False smpl_model: assets/pretrains/smpl_model.pkl smpls_folder: smpls src_path: swap_part: body test_ids_file: val.txt tex_size: 3 tgt_path: time_step: 10 train_ids_file: train.txt uv_mapping: assets/pretrains/mapper.txt view_params: R=0,90,0/t=0,0,0 -------------- End ---------------- ./outputs/checkpoints/running 0%| | 0/3 [00:00<?, ?it/s]Network impersonator was created Loading net: ./outputs/checkpoints/lwb_imper_fashion_place/net_epoch_30_id_G.pth Network deepfillv2 was created Loading net: ./outputs/checkpoints/deepfillv2/net_epoch_50_id_G.pth

                    Personalization: meta imitation...

Segmentation fault

[piaozhx]

Maybe it is caused by different cuda version, (e.g. neural_render is complied by cuda9.0 but pytorch is complied by cuda10.0), please checkout your local cuda version(cat /usr/local/cuda/version.txt or nvcc --version) and pytorch version(pip show torch)

[Hitesh-Nagothu]

Checked the CUDA version active and the torch cuda. They indeed are both the same and on 10.0 CUDA. Yet, segfault. Do you require any specifics about the environment config to locate the issue? Thanks for responding.

(lwgan) [hiteshna@cpn impersonator-master]$ python -c "import torch; print(torch.version.cuda)" 10.0.130 (lwgan) [hiteshna@cpn impersonator-master]$ module load cuda/10.0 CUDA 10.0 with cuDNN 7.6.2 has been loaded.

[Hitesh-Nagothu]

Resolved

[shujonnaha]

@Hitesh-Nagothu Can you please describe how you fixed it? I am getting the same error. Thanks in advance.

[Hitesh-Nagothu]

@shujonnaha I first checked if the neural_renderer is properly installed. If yes, then checked for the torch Cuda compatibility based on the respective Cuda version. And it might be silly to mention, it was resolved when during the initial execution I did not explicitly assign a gpu_id (as in demo code). I was using a remote HPC and not a root user.

[shujonnaha]

Thanks! Can you please explain a bit more about how you checked if neural_renderer is properly installed? Also, when you say respective cuda version, do you refer to the cuda version used to compile neural_renderer?