RuntimeError: cusolver error: CUSOLVER_STATUS_INTERNAL_ERROR, when calling `cusolverDnCreate(handle)`

[zhang-hao-yuan]

python demo.py --image-folder samples --exp-cfg data/conf.yaml --output-folder OUTPUT_FOLDER --show False --save-params True --save-vis True --save-mesh True
Processing with R-CNN: 0%| | 0/1 [00:00<?, ?it/s]/home/na/anaconda3/envs/expose/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /pytorch/c10/core/TensorImpl.h:1156.) return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode) Processing with R-CNN: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1.05it/s] 2024-09-19 12:28:48.391 | INFO | main:main:241 - Saving results to: OUTPUT_FOLDER 2024-09-19 12:28:48.393 | WARNING | expose.models.attention.predictor:init:91 - Apply hand network on body: True 2024-09-19 12:28:48.393 | WARNING | expose.models.attention.predictor:init:93 - Apply hand network on hands: True 2024-09-19 12:28:48.393 | WARNING | expose.models.attention.predictor:init:95 - Predict hands: True 2024-09-19 12:28:48.393 | WARNING | expose.models.attention.predictor:init:102 - Predict head: True 2024-09-19 12:28:48.393 | INFO | expose.models.attention.predictor:init:109 - Condition hand on body: True 2024-09-19 12:28:48.393 | INFO | expose.models.attention.predictor:init:112 - Condition hand wrist pose on body: True 2024-09-19 12:28:48.393 | INFO | expose.models.attention.predictor:init:117 - Condition hand finger pose on body: True 2024-09-19 12:28:48.393 | INFO | expose.models.attention.predictor:init:121 - Condition hand shape on body shape: False 2024-09-19 12:28:48.393 | INFO | expose.models.attention.predictor:init:169 - Condition head on body: True 2024-09-19 12:28:48.393 | INFO | expose.models.attention.predictor:init:170 - Condition expression on body: True 2024-09-19 12:28:48.393 | INFO | expose.models.attention.predictor:init:172 - Condition shape on body: False 2024-09-19 12:28:48.393 | INFO | expose.models.attention.predictor:init:173 - Condition neck pose on body: False 2024-09-19 12:28:48.393 | INFO | expose.models.attention.predictor:init:175 - Condition jaw pose on body: True 2024-09-19 12:28:48.849 | INFO | expose.models.attention.predictor:init:252 - Body model: SMPLXLayer( Gender: NEUTRAL Number of joints: 55 Betas: 10 Number of PCA components: 6 Flat hand mean: False Number of Expression Coefficients: 10 (vertex_joint_selector): VertexJointSelector() ) 2024-09-19 12:28:49.034 | INFO | expose.models.backbone.hrnet:init_weights:487 - => init weights from normal distribution 2024-09-19 12:28:49.293 | WARNING | expose.models.backbone.hrnet:load_weights:519 - => please download pre-trained models first! 2024-09-19 12:28:49.293 | WARNING | expose.models.backbone.hrnet:load_weights:520 - data/network_weights/hrnet/imagenet/hrnet_w48-8ef0771d.pth does not exist! 2024-09-19 12:28:49.305 | INFO | expose.models.common.networks:init:267 - Building iterative regressor with 3 stages 2024-09-19 12:28:49.305 | INFO | expose.models.common.networks:init:282 - Detach mean: False 2024-09-19 12:28:49.363 | INFO | expose.models.backbone.resnet:resnet18:113 - Loading pretrained ResNet-18 2024-09-19 12:28:49.386 | INFO | expose.models.common.networks:init:267 - Building iterative regressor with 3 stages 2024-09-19 12:28:49.387 | INFO | expose.models.common.networks:init:282 - Detach mean: False 2024-09-19 12:28:49.387 | INFO | expose.models.attention.head_predictor:init:81 - Building head predictor with 3 stages 2024-09-19 12:28:49.438 | INFO | expose.models.backbone.resnet:resnet18:113 - Loading pretrained ResNet-18 2024-09-19 12:28:49.462 | INFO | expose.models.common.networks:init:267 - Building iterative regressor with 3 stages 2024-09-19 12:28:49.462 | INFO | expose.models.common.networks:init:282 - Detach mean: False 2024-09-19 12:28:49.462 | INFO | expose.models.attention.predictor:init:490 - 2D Head crop keyps loss: KeypointLoss(Norm type: L1) 2024-09-19 12:28:49.462 | INFO | expose.models.attention.predictor:init:502 - 2D Left hand crop keyps loss: KeypointLoss(Norm type: L1) 2024-09-19 12:28:49.462 | INFO | expose.models.attention.predictor:init:515 - 2D Left hand crop keyps loss: KeypointLoss(Norm type: L1) 2024-09-19 12:28:49.462 | INFO | expose.models.common.smplx_loss_modules:init:48 - Stages to penalize: [-1] 2024-09-19 12:28:49.463 | INFO | expose.models.common.smplx_loss_modules:init:400 - Stages to regularize: [-1] 2024-09-19 12:28:49.558 | INFO | expose.utils.checkpointer:init:44 - Creating directory data/checkpoints 2024-09-19 12:28:49.559 | INFO | expose.utils.checkpointer:load_checkpoint:90 - Load pretrained: False 2024-09-19 12:28:49.559 | WARNING | expose.utils.checkpointer:load_checkpoint:93 - Loading checkpoint from data/checkpoints/model.ckpt! 2024-09-19 12:28:49.841 | WARNING | expose.utils.checkpointer:load_checkpoint:121 - The following keys were not found: ['smplx.head_idxs', 'smplx.body_model.left_hand_components', 'smplx.body_model.right_hand_components', 'smplx.body_model.left_hand_mean', 'smplx.body_model.right_hand_mean', 'smplx.body_model.pose_mean', 'smplx.body_model.dynamic_lmk_bary_coords'] 2024-09-19 12:28:49.841 | WARNING | expose.utils.checkpointer:load_checkpoint:124 - The following keys were not expected: ['smplx.body_model.source_idxs', 'smplx.body_model.target_idxs', 'smplx.body_model.extra_joint_regressor', 'smplx.body_model.dynamic_lmk_b_coords', 'smplx.hand_predictor.hand_offset', 'smplx.hand_predictor.hand_model.extra_joints_idxs', 'smplx.hand_predictor.hand_model.faces_tensor', 'smplx.hand_predictor.hand_model.v_template', 'smplx.hand_predictor.hand_model.shapedirs', 'smplx.hand_predictor.hand_model.J_regressor', 'smplx.hand_predictor.hand_model.posedirs', 'smplx.hand_predictor.hand_model.parents', 'smplx.hand_predictor.hand_model.lbs_weights', 'smplx.hand_predictor.pca_decoder.pca_basis', 'smplx.hand_predictor.pca_decoder.inv_pca_basis', 'smplx.hand_predictor.pca_decoder.mean', 'smplx.head_predictor.head_offset', 'smplx.head_predictor.head_vertices_ids', 'smplx.head_predictor.head_model.faces_tensor', 'smplx.head_predictor.head_model.v_template', 'smplx.head_predictor.head_model.shapedirs', 'smplx.head_predictor.head_model.expr_dirs', 'smplx.head_predictor.head_model.J_regressor', 'smplx.head_predictor.head_model.posedirs', 'smplx.head_predictor.head_model.parents', 'smplx.head_predictor.head_model.lbs_weights', 'smplx.head_predictor.head_model.lmk_faces_idx', 'smplx.head_predictor.head_model.lmk_bary_coords', 'smplx.head_predictor.head_model.dynamic_lmk_faces_idx', 'smplx.head_predictor.head_model.dynamic_lmk_b_coords', 'smplx.head_predictor.head_model.neck_kin_chain', 'smplx.body_loss.edge_loss.gt_connections', 'smplx.body_loss.edge_loss.est_connections', 'smplx.hand_loss.edge_loss.gt_connections', 'smplx.hand_loss.edge_loss.est_connections', 'smplx.head_loss.edge_loss.gt_connections', 'smplx.head_loss.edge_loss.est_connections'] 0%| | 0/1 [00:00<?, ?it/s] Traceback (most recent call last): File "demo.py", line 554, in main( File "/home/na/anaconda3/envs/expose/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context return func(*args, kwargs) File "demo.py", line 288, in main model_output = model(body_imgs, body_targets, full_imgs=full_imgs, File "/home/na/anaconda3/envs/expose/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, *kwargs) File "/home/na/Desktop/smplx-test/expose-master/expose/models/smplx_net.py", line 71, in forward output = self.smplx(images, targets=targets, File "/home/na/anaconda3/envs/expose/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(input, kwargs) File "/home/na/Desktop/smplx-test/expose-master/expose/models/attention/predictor.py", line 1094, in forward left_hand_points_to_crop = self.points_to_crops( File "/home/na/anaconda3/envs/expose/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/home/na/Desktop/smplx-test/expose-master/expose/models/common/bbox_sampler.py", line 58, in forward inv_crop_transforms = torch.inverse(crop_transforms) RuntimeError: cusolver error: CUSOLVER_STATUS_INTERNAL_ERROR, when calling cusolverDnCreate(handle)