adapt_detnet_dual.py not working very well (+@)

[giesth39]

Hi

I am interested in the S2DHand challenge, so i was trying to run the adapt_detnet_dual.py file. there is an brief index of the situation i'm facing with

1. i changed the data name(since from google drive i got test-iccv2023)
2. getting ValueError: cannot reshape array of size 1 into shape (42,)
3. 'test_ego_data.json' file shows null in 'joint_valid' and 'keypoints' +@ also getting error in calc_metrics.py, too.

since the assemblyhands dataset github says to organize the dataset like;

• ${DATA_DIR}

  • images
    • ego_images_rectified
      • split: {test, train, val}
  • videos
    • exo_videos_rectified
      • nusar-2021_*
  • annotations
    • skeleton.txt
    • split: {demo, test, train, val}
      • assemblyhands_${split}_ego_calib_v1-1.json
      • assemblyhands_${split}_ego_data_v1-1.json
      • assemblyhands_${split}_joint_3d_v1-1.json

  and the S2DHand only requires images and annotation, i've changed the googledrive test-iccv2023 folder like;

  -test-iccv2023

  • assemblyhands_test_ego_calib_iccv2023_v1-1.json
  • assemblyhands_test_abs_depth_iccv2023-v1-3.json
  • assmeblyhands_test_ego_data_iccv2023-v1-3.json
  • assemblyhands_test_exo_calib_v1-1.json
  • templates -assemblyhands_test_joint3d_iccv23-v1-3_pred_template.json to test-iccv2023
  • assemblyhands_test_ego_calib_v1-1.json
  • assemblyhands_test_abs_depth_v1-1.json
  • assmeblyhands_test_ego_data_v1-1.json
  • assemblyhands_test_exo_calib_v1-1.json -assemblyhands_test_joint3d_v1-1.json

and then tried the code

python3 adapt_detnet_dual.py -trs ah -tes ah --evaluate -eid 37 --gpus 0 --pic -1 --checkpoint pretrain --setup 0 --pair 1,3

and i got

also, if i run python3 calc_metrics.py --checkpoint in_dataset_adapt/evaluation/ah --setup 0 --pair 1,2

I will get ZeroDivisionError: division by zero from the line164 in calc_metrics.py print(f'total:{num}, mean Mono-M:{mono / num:.2f}, mean Dual-M:{merge / num:.2f}')

Can i get some help about this?

+) should i make a 'pretrain' directory under 'S2DHands' to run cp ${DATA_DIR}/S2DHand-pretrain/* pretrain ??

[MickeyLLG]

Hi, thanks for being interested in our work. Your situation is because the testing annotations of AssemblyHands is not released. The "test-iccv2023" files are for the workshop challenge last year and thus have no ground-truth information. To run our code, I recommend to use validation set instead. Specifically, you can revise the data_split arguments to data_split='val' in adapt_detnet_dual.py (Lines 144 and 219). The validation set has ground-truth annotations so the code should be runnable. Of course, you may also contact the author of AssemblyHands dataset for the access of test set annotations.

+) Yes, please mkdir pretrain before copying. The pretrain directory has been updated to this repository. I have updated the README according to your issues. Feel free to add comments if there is any question.