search

sign-language-processing / datasets

TFDS data loaders for sign language datasets.
https://sign-language-processing.github.io/#existing-datasets
82 stars 27 forks source link

What are .poseheader files? #52

Open kaustesseract opened 9 months ago

kaustesseract commented 9 months ago

Hi,

I was wondering if you can give a more detailed description of what are .poseheader files? What is the significance of it? How can we create for our own custom dataset?

AmitMY commented 9 months ago

Hi, When storing a dataset in a disk-mapped fashion with TFDS, we store poses as three objects:

This way to save the data makes it very fast to read, but does not give the user additional information we have on the poses, for example, the names of the points (which can be used for more readable code), or the connection between points (can be used to visualize the data).

For that end, we expose the .poseheader file, which is the PoseHeader from https://github.com/sign-language-processing/pose

Assuming we have datum which has datum['pose'] in it, we can now construct a Pose object, by doing (pseudocode):

with open('holistic.poseheader', 'rb') as f:
  header = PoseHeader.read(BufferReader(f.read()))

body = NumPyPoseBody(fps=datum['pose']['fps'], data=datum['pose']['data'], confidence=datum['pose']['confidence'])
pose = Pose(header=header, body=body)

# Additional code here, for example, visualization...

Including this file is not mandatory in the curation of a dataset.

Anyway, what dataset did you have in mind?