Python - The output file cannot display the complete data

[Xiangshougudu]

Hi, I have encountered a problem while trying to use ezc3d with python. I am trying to store 3D data with 324000 sample points, but I have noticed that the generated .c3d file only displays 65536 sample points. I attempted to randomly generate some data, and the issue persists. I suspect it is due to overlooking a setting for a specific Parameter. Can you help me？

This is my code:

import numpy as np import ezc3d

c3d = ezc3d.c3d() c3d['parameters']['POINT']['UNITS']['value'] = ['m'] c3d['parameters']['POINT']['RATE']['value'] = [30] c3d['parameters']['POINT']['LABELS']['value'] = ('point1', 'point2', 'point3', 'point4', 'point5') c3d['data']['points'] = np.random.rand(4, 5, 324000) c3d['data']['points'][1, :, :] = 2 c3d['data']['points'][2, :, :] = 3

c3d.add_parameter("POINT", "newParam", [1, 2, 3]) c3d.add_parameter("NewGroup", "newParam", ["MyParam1", "MyParam2"])

c3d.write("path_to_c3d.c3d")

[pariterre]

Hello there!

The 65536 number comes from the fact that the number of sample points is stored in an 8-bit integer in a C3D file. Therefore the maximum number of samples is 0xFFFF (or 65535). That said, I previously made a patch to overcome this limit by setting the number of point to -1 (effectively not storing the number of point) and just figuring the number of points on the fly while reading the data. I probably forgot to apply the same patch to writing data.

I'll have a look!

[pariterre]

Dear @Xiangshougudu I could not reproduce the error you are reporting here. When writing a file with arbitrary number of frames, it does create the same amount of frames (with a small caviat that I am currently fixing #305 ).

Are you sure you are running the last version of ezc3d (1.5.6)

Here is the code I used:

import numpy as np
import ezc3d

# Create dummy data
c3d = ezc3d.c3d()
c3d['parameters']['POINT']['UNITS']['value'] = ['m']
c3d['parameters']['POINT']['RATE']['value'] = [30]
c3d['parameters']['POINT']['LABELS']['value'] = ('point1', 'point2', 'point3', 'point4', 'point5')
c3d['data']['points'] = np.random.rand(4, 5, 320000)
c3d['data']['points'][1, :, :] = 2
c3d['data']['points'][2, :, :] = 3
c3d['data']['points'][3, :, :] = 1

c3d.add_parameter("POINT", "newParam", [1, 2, 3])
c3d.add_parameter("NewGroup", "newParam", ["MyParam1", "MyParam2"])

c3d.write("path_to_c3d.c3d")

# Read the dummy data in a new file and create a second file from that one
c3d_post = ezc3d.c3d("path_to_c3d.c3d")
c3d_post.write("path_to_c3d_post.c3d")

# Read the second file and compare the data
c3d_post_post = ezc3d.c3d("path_to_c3d_post.c3d")

# These should be the same
print(f"Number of frames (from header original   ): {c3d['header']['points']['last_frame']}")  # This evaluates to 0 as header is not updated when creating a new c3d object
print(f"Number of frames (from header first copy ): {c3d_post['header']['points']['last_frame']}")
print(f"Number of frames (from header second copy): {c3d_post_post['header']['points']['last_frame']}")

print(f"Number of frames (from data original   ): {c3d['data']['points'].shape[2]}")
print(f"Number of frames (from data first copy ): {c3d_post['data']['points'].shape[2]}")
print(f"Number of frames (from data second copy): {c3d_post_post['data']['points'].shape[2]}")

print(f"Original   - first copy  = {np.sum(c3d['data']['points'] - c3d_post['data']['points'])}")  # This difference comes from the conversion of the data from double to float
print(f"Original   - second copy = {np.sum(c3d['data']['points'] - c3d_post_post['data']['points'])}")  # This difference comes from the conversion of the data from double to float
print(f"First copy - second copy = {np.sum(c3d_post['data']['points'] - c3d_post_post['data']['points'])}")  # No difference

FWI: I noticed from your code that such amount of frames was ridiculously slow to create so I fixed the speed of creating a large file

[Xiangshougudu]

Dear @pariterre , I'm sorry for not fully describing my problem. In fact, the version of Python on my computer is 3.8.13, and I'm unable to install the lastest ezc3d(1.5.6) in the environment. It can only run ezc3d(1.5.0). I upgraded the Python version from 3.8.13 to 3.9.18 and updated to the latest version of ezc3d (1.5.6) in the environment. Unfortunately, the issue still persists.

Like you, I was able to see that the file contained 320,000 points in the code. However, when I try to read the "path_to_c3d.c3d" file using Mokka-0.6.2 Windows 64-bit, I only see 65,535 points. This is a screenshot of the software.

[pariterre]

Hum.. I suspect this is a Mokka issue then as the file actually contains the points. I would not be surprised that the scrollbar is limited to 65,535, it would actually make sense... Do you know if Mokka is actually able to display more than 65,000 points?

[Xiangshougudu]

Yes, I am certain that Mokka can display more than 65535 points. Additionally, I suspect there might be an issue during the writing of the c3d file. This is because, for data of the same shape(432000 points), the .c3d file I wrote occupies 54002KB of space, while data exported by other software occupies 108003KB of space. Here is a screenshot:

[pariterre]

Since the data is present in the C3D file, I suspect that the 100Mo file is simply because the other one used INTEGER to store the data while in ezc3d we are using FLOAT which takes less space... Can you provide me with both files? Since this issue is mokka related, I must have a file which works on mokka to compare what is different from the one created from ezc3d :)

[pariterre]

This is a kind reminder :)