Open changjo opened 2 years ago
Hi, thanks for bringing this issue to our attention! If possible can you share the snippet of code used so we can recreate the third image?
In our experiments we did synchronize the top-view and gnss data. There was quite a lot of error but I don't think it was as bad as the third image. I guess I can share the video, but I can't share the code just yet since it's still a mess. Also, we did this way back in January before we started annotating the videos so we infered blobs and pitch keypoints using older methods from OpenCV and not YOLOv5 etc. Apologies.
I think synchronization is a good topic for the user guide so I will work on cleaning the code / making a more detailed tutorial after finishing #34 since that seems like a bigger problem!
Hi, thanks for the detailed explanations.
I haven't tried to project GNSS coordinates onto the top-view
but wide-view
. Your video shows they are synchronized well. I think I need to project them to the top-view
. I also would like to know if the first row data of the first file of GNSS data matches the first frame of the wide-view
and top-view
video. I am attaching the snippet of the code I use. Thanks.
GNSS_FOUR_CORNERS_KEYS = ['(0,0)', '(0,68)', '(105,68)', '(105,0)']
WV_FOUR_CORNERS_KEYS = ['(0.0, 0.0)', '(0.0, 68.0)', '(105.0, 68.0)', '(105.0, 0.0)']
def get_homography(keypoints, four_corners_keys, unit='cm', inverse=False):
original_coords = []
pitch_coords = []
for key in four_corners_keys:
original_coords.append(keypoints[key])
pitch_coords.append(list(map(lambda x: float(x) * 100 if unit == 'cm' else float(x), key.strip('()').split(','))))
print(key, keypoints[key])
original_coords = np.array(original_coords).reshape(-1, 1, 2)
pitch_coords = np.array(pitch_coords).reshape(-1, 1, 2)
if inverse:
H, _ = cv2.findHomography(pitch_coords, original_coords)
else:
H, _ = cv2.findHomography(original_coords, pitch_coords)
return H
# Convert GNSS cooridnates to local pitch coodinates
gnss_keypoints = load_json(GNSS_KEYPOINTS_FILENAME)
gnss_file_list = sorted(glob(os.path.join(DATA_ROOT, 'GNSS/*.csv')))
gnss_H = get_homography(gnss_keypoints, GNSS_FOUR_CORNERS_KEYS)
player_lon_lat_coords = np.array(df[['Lon', 'Lat']]).reshape(-1, 1, 2)
player_pitch_coords = cv2.perspectiveTransform(player_lon_lat_coords, gnss_H).reshape(-1, 2)
gnss_data_df['x'] = player_pitch_coords[:, 0]
gnss_data_df['y'] = player_pitch_coords[:, 1]
# Convert wide-view cooridnates to local pitch coodinates
wv_keypoints = load_json(WIDE_VIEW_KEYPOINTS_FILENAME)
wv_file_list = sorted(glob(os.path.join(DATA_ROOT, 'wide-view/*.csv')))
wv_H = get_homography(wv_keypoints, WV_FOUR_CORNERS_KEYS)
wv_H_inv = get_homography(wv_keypoints, WV_FOUR_CORNERS_KEYS, inverse=True)
df['image_x'] = df['bb_left'] + df['bb_width'] / 2
df['image_y'] = df['bb_top'] + df['bb_height']
player_image_coords = np.array(df[['image_x', 'image_y']]).reshape(-1, 2)
player_pitch_coords = cv2.perspectiveTransform(player_image_coord.reshape(-1, 1, 2), wv_H).reshape(-1, 2)
wv_data_df['x'] = player_pitch_coords[:, 0]
wv_data_df['y'] = player_pitch_coords[:, 1]
frame_num = 1
wv_coords_on_pitch = np.array(wv_data_df.loc[frame_num][['x', 'y']])
wv_coords_on_image = cv2.perspectiveTransform(wv_coords_on_pitch.reshape(-1, 1, 2), wv_H_inv).reshape(-1, 2)
first_time = '2022-01-01 13:40:18.100'
gnss_coords_on_pitch = np.array(gnss_data_df.loc[first_time][['x', 'y']])
gnss_coords_on_image = cv2.perspectiveTransform(gnss_coords_on_pitch.reshape(-1, 1, 2), wv_H_inv).reshape(-1, 2)
I projected the first row of the GNSS coordinates onto the first frame of the top-view
. You may check it. Thanks.
Hi Atom. I have extracted the
top-view
andwide-view
image frames and checked the first image but they are not synchronous. I have computed the homographies forwide-view
andGNSS
coordinates and converted them to the local pitch coordinates. To check them, I also computed the inverse homography matrix from thewide-view
coordinates and applied it to the first frame of thewide-view
local pitch coordinates and the first row (13:40:18.100000
) of theGNSS
local pitch coordinates and displayed them but they also are not matched.Red dots are from
wide-view
and blue dots are fromGNSS
.