AlphaPose_to_OpenPose - adjustment

[EmanuelWicki]

Hey David,

I just converted the AlphaPose Json file to openpose json file. Maybe alpha pose has changed its json structure and therefore it didnt work with the current .py file. I modified the file as described below to make it work:

ORIGINAL FILE --------------------------------------------------------------------------------------------------------------------------- "# Open AlphaPose json file with open(input_alphapose_json_file, 'r') as alpha_json_f: alpha_js = json.load(alpha_json_f) json_dict = {'version':1.3, 'people':[]} coords = [] frame_next = int(alpha_js[0].get('image_id').split('.')[0]) for i, a in enumerate(alpha_js): frame_prev = int(a.get('image_id').split('.')[0]) coords = a.get('keypoints') if frame_next != frame_prev or i==0:

Save openpose json file with all people contained in the previous frame

            if i != 0:
                json_file = os.path.join(output_openpose_json_folder, os.path.splitext(os.path.basename(str(frame_prev-1).zfill(5)))[0]+'.json')
                with open(json_file, 'w') as js_f:
                    js_f.write(json.dumps(json_dict))
            # Reset json_dict
            json_dict['people'] = [{'person_id':[-1], 
                'pose_keypoints_2d': [], 
                'face_keypoints_2d': [], 
                'hand_left_keypoints_2d':[], 
                'hand_right_keypoints_2d':[], 
                'pose_keypoints_3d':[], 
                'face_keypoints_3d':[], 
                'hand_left_keypoints_3d':[], 
                'hand_right_keypoints_3d':[]}]
        else:
            # Add new person to json_dict
            json_dict['people'] += [{'person_id':[-1], 
                'pose_keypoints_2d': [], 
                'face_keypoints_2d': [], 
                'hand_left_keypoints_2d':[], 
                'hand_right_keypoints_2d':[], 
                'pose_keypoints_3d':[], 
                'face_keypoints_3d':[], 
                'hand_left_keypoints_3d':[], 
                'hand_right_keypoints_3d':[]}]
        # Add coordinates to json_dict
        json_dict['people'][-1]['pose_keypoints_2d'] = coords

        frame_next = int(a.get('image_id').split('.')[0])

    # Save last frame
    json_file = os.path.join(output_openpose_json_folder, os.path.splitext(os.path.basename(str(frame_prev).zfill(5)))[0]+'.json')
    with open(json_file, 'w') as js_f:
        js_f.write(json.dumps(json_dict))"

       *MODIFIED FILE*---------------------------------------------------------------------------------------------------------------
       "    # Open AlphaPose json file
with open(input_alphapose_json_file, 'r') as alpha_json_f:
    alpha_js = json.load(alpha_json_f)
    json_dict = {'version': 1.3, 'people': []}

    # Process each file in the JSON
    for image_id, data in alpha_js.items():
        frame_index = image_id.split('.')[0]  # Assuming '0.jpg' should lead to frame index '0'
        people = data['bodies']

        # Initialize a new list of people for the current frame
        json_dict['people'] = []

        for person in people:
            joints = person['joints']
            # Add a new person entry
            person_entry = {
                'person_id': [-1], 
                'pose_keypoints_2d': joints, 
                'face_keypoints_2d': [], 
                'hand_left_keypoints_2d':[], 
                'hand_right_keypoints_2d':[], 
                'pose_keypoints_3d':[], 
                'face_keypoints_3d':[], 
                'hand_left_keypoints_3d':[], 
                'hand_right_keypoints_3d':[]
            }
            json_dict['people'].append(person_entry)

        # Save the JSON for the current frame
        json_file = os.path.join(output_openpose_json_folder, f"{frame_index}.json")
        with open(json_file, 'w') as js_f:
            js_f.write(json.dumps(json_dict))"

[EmanuelWicki]

But now I get this error "Single-person analysis selected.

IndexError Traceback (most recent call last) Cell In[2], line 1 ----> 1 Pose2Sim.personAssociation()

File C:\ProgramData\anaconda3\Lib\site-packages\Pose2Sim\Pose2Sim.py:348, in personAssociation(config) 345 logging.info("---------------------------------------------------------------------") 346 logging.info(f"\nProject directory: {project_dir}") --> 348 track_2d_all(config_dict) 350 end = time.time() 351 elapsed = end-start

File C:\ProgramData\anaconda3\Lib\site-packages\Pose2Sim\personAssociation.py:436, in track_2d_all(config) 434 except: 435 raise NameError('Model not found in skeletons.py nor in Config.toml') --> 436 tracked_keypointid = [node.id for , _, node in RenderTree(model) if node.name==tracked_keypoint][0] 438 # 2d-pose files selection 439 pose_listdirs_names = next(os.walk(pose_dir))[1]

IndexError: list index out of range"

But I have selected the right keypoint (Neck) for tracking, checked in the skeleton if it exists for my model (HALPE_136) and it does. I also selected alphapose and HALPE_136. Any idea?

[davidpagnon]

I did not quite understand what you changed in the code, could you format it by putting it between three quotes like that?

Did you set pose_model = 'HALPE_136' in skeletons.py? Are you sure you respected the case (all capitals)? Can you do a print(pose_model) just before the try/except?

[EmanuelWicki]

what do you mean exactly with set pose_model in skeletons.py? where should I set this? I only select it in config.toml

[davidpagnon]

Sorry I meant in Config.toml!

[EmanuelWicki]

Yes I did. The only thing thats odd is that i dont have 126 keypoint in the json file... maybe this could be the problem

[davidpagnon]

Okay, can you still do what I asked for earlier?

[EmanuelWicki]

"""#!/usr/bin/env python

-- coding: utf-8 --

''' ########################################################

Convert AlphaPose json file to OpenPose json files

########################################################

Converts AlphaPose single json file to OpenPose frame-by-frame files.

Usage: 
python -m AlphaPose_to_OpenPose -i input_alphapose_json_file -o output_openpose_json_folder
OR python -m AlphaPose_to_OpenPose -i input_alphapose_json_file
OR from Pose2Sim.Utilities import AlphaPose_to_OpenPose; AlphaPose_to_OpenPose.AlphaPose_to_OpenPose_func(r'input_alphapose_json_file', r'output_openpose_json_folder')

'''

INIT

import json import os import argparse

AUTHORSHIP INFORMATION

author = "David Pagnon" copyright = "Copyright 2023, Pose2Sim" credits = ["David Pagnon"] license = "BSD 3-Clause License" version = '0.6' maintainer = "David Pagnon" email = "contact@david-pagnon.com" status = "Development"

FUNCTIONS

def AlphaPose_to_OpenPose_func(*args): ''' Converts AlphaPose single json file to OpenPose frame-by-frame files.

Usage: 
python -m AlphaPose_to_OpenPose -i input_alphapose_json_file -o output_openpose_json_folder
OR python -m AlphaPose_to_OpenPose -i input_alphapose_json_file
OR from Pose2Sim.Utilities import AlphaPose_to_OpenPose; AlphaPose_to_OpenPose.AlphaPose_to_OpenPose_func(r'input_alphapose_json_file', r'output_openpose_json_folder')
'''

try:
    input_alphapose_json_file = os.path.realpath(args[0]['input_alphapose_json_file']) # invoked with argparse
    if args[0]['output_openpose_json_folder'] == None:
        output_openpose_json_folder = os.path.splitext(input_alphapose_json_file)[0]
    else:
        output_openpose_json_folder = os.path.realpath(args[0]['output_openpose_json_folder'])
except:
    input_alphapose_json_file = os.path.realpath(args[0]) # invoked as a function
    try:
        output_openpose_json_folder = os.path.realpath(args[1])
    except:
        output_openpose_json_folder = os.path.splitext(input_alphapose_json_file)[0]

if not os.path.exists(output_openpose_json_folder):    
    os.mkdir(output_openpose_json_folder)

# Open AlphaPose json file
with open(input_alphapose_json_file, 'r') as alpha_json_f:
    alpha_js = json.load(alpha_json_f)
    json_dict = {'version': 1.3, 'people': []}

    # Process each file in the JSON
    for image_id, data in alpha_js.items():
        frame_index = image_id.split('.')[0]  # Assuming '0.jpg' should lead to frame index '0'
        people = data['bodies']

        # Initialize a new list of people for the current frame
        json_dict['people'] = []

        for person in people:
            joints = person['joints']
            # Add a new person entry
            person_entry = {
                'person_id': [-1], 
                'pose_keypoints_2d': joints, 
                'face_keypoints_2d': [], 
                'hand_left_keypoints_2d':[], 
                'hand_right_keypoints_2d':[], 
                'pose_keypoints_3d':[], 
                'face_keypoints_3d':[], 
                'hand_left_keypoints_3d':[], 
                'hand_right_keypoints_3d':[]
            }
            json_dict['people'].append(person_entry)

        # Save the JSON for the current frame
        json_file = os.path.join(output_openpose_json_folder, f"{frame_index}.json")
        with open(json_file, 'w') as js_f:
            js_f.write(json.dumps(json_dict))

if name == 'main': parser = argparse.ArgumentParser() parser.add_argument('-i', '--input_alphapose_json_file', required = True, help='input AlphaPose single json file') parser.add_argument('-o', '--output_openpose_json_folder', required = False, help='output folder for frame-by-frame OpenPose json files') args = vars(parser.parse_args())

AlphaPose_to_OpenPose_func(args)"""

[EmanuelWicki]

This was my error "" File "C:\ProgramData\anaconda3\envs\alphapose\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "C:\ProgramData\anaconda3\envs\alphapose\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "\Pose2Sim\Utilities\AlphaPose_to_OpenPose.py", line 116, in AlphaPose_to_OpenPose_func(args) File "\Pose2Sim\Utilities\AlphaPose_to_OpenPose.py", line 68, in AlphaPose_to_OpenPose_func frame_next = int(alpha_js[0].get('image_id').split('.')[0]) KeyError: 0" "

[EmanuelWicki]

and if i put a print statement for pose_model before try/except, nothing happens

[davidpagnon]

If you could put your code between three backticks like so, it would be easier to read :)

But overall, can you just tell me what you changed in the code so that I don't have to compare the whole file? Also, I don't get it, how is it that you come across a different error now?

[EmanuelWicki]

The error I posted was the error i got with the original alphapse to openpose file. With the new code it workes. Orignial line:

frame_next = int(alpha_js[0].get('image_id').split('.')[0]) 

Modified line:

for image_id, data in alpha_js.items():
    frame_index = image_id.split('.')[0]

[EmanuelWicki]

The right model is selected. But if i try to print the detected_keypoint right after it was selected (last line of the code) and I get detected keypoint: None. What could be the reason for that?

def track_2d_all(config):
    '''
    For each frame,
    - Find all possible combinations of detected persons
    - Triangulate 'tracked_keypoint' for all combinations
    - Reproject the point on all cameras
    - Take combination with smallest reprojection error
    - Write json file with only one detected person
    Print recap message

    INPUTS: 
    - a calibration file (.toml extension)
    - json files from each camera folders with several detected persons
    - a Config.toml file
    - a skeleton model

    OUTPUTS: 
    - json files for each camera with only one person of interest    
    '''

    # Read config
    project_dir = config.get('project').get('project_dir')
    session_dir = os.path.realpath(os.path.join(project_dir, '..', '..'))
    multi_person = config.get('project').get('multi_person')
    pose_model = config.get('pose').get('pose_model')
    print('pose model: ', pose_model)
    tracked_keypoint = config.get('personAssociation').get('tracked_keypoint')
    print('tracked keypoint: ', tracked_keypoint)

[EmanuelWicki]

it seems as if it couldnt load the config.toml information of the personAssociation part

[davidpagnon]

I think you are using an older version of the code (and potentially of the Config file as well).

You should have tracked_keypoint = config.get('personAssociation').get('single_person').get('tracked_keypoint')

[EmanuelWicki]

Dear David, Thank you very much for your time! I really appreciate this. You were right i did have a old version. Now it can read in the values from my personAssociation.py file. But regarding the alphapose model i still have some problems:

1. If I use e.g. HALPE_26 Model to analyze a video with the following command:

   python scripts/demo_inference.py --detector yolo --cfg "path_to_halpe_26_config.yaml" --checkpoint "path_to_halpe_26_checkpoint.pth" --video "path_to_video" --outdir "path_to_output_directory"  --pose_track --save_video --format cmu

   and then use the script to convert it into openpose format:

   python -m AlphaPose_to_OpenPose -i <path>\alphapose-results.json -o <path>

   with the original AlphaPose_to_OpenPose file i get the following error:

   Traceback (most recent call last):
   File "C:\ProgramData\anaconda3\envs\Pose2Sim\lib\runpy.py", line 194, in _run_module_as_main
   return _run_code(code, main_globals, None,
   File "C:\ProgramData\anaconda3\envs\Pose2Sim\lib\runpy.py", line 87, in _run_code
   exec(code, run_globals)
   File "<Path>\Pose2Sim\Utilities\AlphaPose_to_OpenPose.py", line 116, in <module>
   AlphaPose_to_OpenPose_func(args)
   File "<Path>\Pose2Sim\Utilities\AlphaPose_to_OpenPose.py", line 68, in AlphaPose_to_OpenPose_func
   frame_next = int(alpha_js[0].get('image_id').split('.')[0])
   KeyError: 0

   If I do it with the modified code mentioned above in this conversation, it seems to work and the format seems valid, short preview of the json file (with 18 Keypoints because of --format cmu, if i leave this away, i get 26 keypoints ):

   {"version": 1.3, "people": [{"person_id": [-1], "pose_keypoints_2d": [19.92374610900879, 608.7724609375, 0.3861883580684662, ... , 0.19044001400470734, 579.1724853515625, 0.1993613839149475], "face_keypoints_2d": [], "hand_left_keypoints_2d": [], "hand_right_keypoints_2d": [], "pose_keypoints_3d": [], "face_keypoints_3d": [], "hand_left_keypoints_3d": [], "hand_right_keypoints_3d": []}]}

   If i place these json files in the pose folder of a batch Session - Single Participant - Static trial and run personAssociation i get the following error:

   
   Pose model:  HALPE_26
   0%|                                                                                          | 0/173 [00:00<?, ?it/s]coords:  []
   0%|                                                                                          | 0/173 [00:00<?, ?it/s]
   ---------------------------------------------------------------------------
   IndexError                                Traceback (most recent call last)
   Cell In[2], line 1
   ----> 1 Pose2Sim.personAssociation()

File C:\ProgramData\anaconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\Pose2Sim.py:352, in personAssociation(config) 349 logging.info("---------------------------------------------------------------------") 350 logging.info(f"\nProject directory: {project_dir}") --> 352 track_2d_all(config_dict) 354 end = time.time() 355 elapsed = end-start

File C:\ProgramData\anaconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\personAssociation.py:698, in track_2d_all(config) 695 personsIDs_comb = persons_combinations(json_files_f) 697 # choose persons of interest and exclude cameras with bad pose estimation --> 698 error_proposals, proposals, Q_kpt = best_persons_and_cameras_combination(config, json_files_f, personsIDs_comb, P_all, tracked_keypoint_id, calib_params) 700 error_min_tot.append(np.mean(error_proposals)) 701 cameras_off_count = np.count_nonzero([np.isnan(comb) for comb in proposals]) / len(proposals)

File C:\ProgramData\anaconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\personAssociation.py:215, in best_persons_and_cameras_combination(config, json_files_framef, personsIDs_combinations, projection_matrices, tracked_keypoint_id, calib_params) 213 error_comb_all, comb_all, Q_comb_all = [], [], [] 214 for comb in combinations_with_cams_off: --> 215 error_comb, comb, Q_comb = triangulate_comb(comb, coords, projection_matrices, calib_params, config) 216 error_comb_all.append(error_comb) 217 comb_all.append(comb)

File C:\ProgramData\anaconda3\envs\Pose2Sim\lib\site-packages\Pose2Sim\personAssociation.py:113, in triangulate_comb(comb, coords, P_all, calib_params, config) 110 likelihood_threshold = config.get('personAssociation').get('likelihood_threshold_association') 112 # Replace likelihood by 0. if under likelihood_threshold --> 113 coords[:,2][coords[:,2] < likelihood_threshold] = 0. 114 comb[coords[:,2] == 0.] = np.nan 116 # Filter coords and projection_matrices containing nans

IndexError: index 2 is out of bounds for axis 1 with size 0

Sorry for this long message. I hope you can help me find a solution been stuck here...

[EmanuelWicki]

This issue has been resolved. Sorry for the confusion. You explained it very detailed and good in your readme and I missed it. For Halpe_26 - no need for --format cmu and that way your code to convert to openpose works perfectly fine and also the rest is smooth. Thank you very much for your help!