the-siesta-group / edfio

Read and write EDF/EDF+ files.
Apache License 2.0
25 stars 5 forks source link

Make unit tests pass on Windows computers #12

Closed marcoross closed 8 months ago

hofaflo commented 8 months ago

For future reference: On windows, the default dtype produced by np.arange with integer arguments is np.int64 on linux, but np.int32 on windows (ref). In the code snippet below, integer_part_length = -1 for an integer value, which leads to factor = 10**8. Calculating value * factor can then lead to an overflow even for a relatively small value.

length = 8
integer_part_length = str(value).find(".")
factor = 10 ** (length - 1 - integer_part_length)
round_func(value * factor) / factor