ISO Format Issue

[KaraMelih]

The neutrino messages seem to fail if they have more or less than 6 decimal points.

[habig]

So, two things here.

1) we want 9 digits in principle since experiment clocks are "good" (for varying definitions of good) to that level. 2) we might like the format checker to pad with zeros or round before sending it off.

https://en.wikipedia.org/wiki/ISO_8601

Says "_There is no limit on the number of decimal places for the decimal fraction. However, the number of decimal places needs to be agreed to by the communicating parties. For example, in Microsoft SQL Server, the precision of a decimal fraction is 3 for a DATETIME, i.e., "yyyy-mm-ddThh:mm:ss[.mmm]"_"

So: we just need to figure out how to tell our instance of the ISO format checker that we want "nine, please". And to pad out or round as appropriate.

[habig]

https://nickb.dev/blog/iso8601-and-nanosecond-precision-across-languages/ discusses this for python, and suggests using a numpy object to work around

[KaraMelih]

I am looking into this, just to keep track, this is the current behavior we have;

try:
    fromisoformat = datetime.fromisoformat
except AttributeError as e:
    from dateutil.parser import isoparse as fromisoformat

def clean_time_input(time):
    if time is None:
        time = datetime.utcnow()

    if isinstance(time, str):
        time = fromisoformat(time)

    return time.isoformat()

neutrino_time_test1 = "2023-06-12 18:30"
neutrino_time_test2 = "2023-06-12 18:30:10"
neutrino_time_test3 = "2023-06-12 18:30:10.123"
neutrino_time_test4 = "2023-06-12 18:30:10.123456"
neutrino_time_test5 = "2023-06-12 18:30:10.123456789"

neutrino_time_test6 = "2023-06-12T18:30:10"
neutrino_time_test7 = "2023-06-12T18:30:10.123"
neutrino_time_test8 = "2023-06-12T18:30:10.123456"
neutrino_time_test9 = "2023-06-12T18:30:10.123456789"

[exec(f"print('testing: {i}\t', clean_time_input(neutrino_time_test{i}))") for i in range(1,10)]

and we want all of them to translate into;

testing: 1   2023-06-12T18:30:00.000000
testing: 2   2023-06-12T18:30:10.000000
testing: 3   2023-06-12T18:30:10.123000
testing: 4   2023-06-12T18:30:10.123456
testing: 5   2023-06-12T18:30:10.123456

testing: 6   2023-06-12T18:30:10.000000
testing: 7   2023-06-12T18:30:10.123000
testing: 8   2023-06-12T18:30:10.123456
testing: 9   2023-06-12T18:30:10.123456

Currently, this function linearizes them as follows;

testing: 1   2023-06-12T18:30:00
testing: 2   2023-06-12T18:30:10
testing: 3   2023-06-12T18:30:10.123000
testing: 4   2023-06-12T18:30:10.123456
testing: 5 "Invalid isoformat string: '2023-06-12 18:30:10.123456789'"

testing: 6   2023-06-12T18:30:10
testing: 7   2023-06-12T18:30:10.123000
testing: 8   2023-06-12T18:30:10.123456
testing: 9 "Invalid isoformat string: '2023-06-12T18:30:10.123456789'"

[KaraMelih]

Okay, I think I found the way to do this using numpy.datetime_as_string()
All of the 9 example cases I wrote above return the same precision with this function. I'll work on implementing it everywhere

[exec(f"print('testing: {i}\t', np.datetime_as_string(np.datetime64(neutrino_time_test{i}), unit='ns'))") for i in range(1,10)]

[habig]

If this has been merged, should we close this issue?

[KaraMelih]

Yes, I couldn't find the exact PR to link here. But in the main branch, this seems to be already implemented.

https://github.com/SNEWS2/SNEWS_Publishing_Tools/blob/ab176e1c8ede372d8acc79340d3580faa7c999b2/snews_pt/messages.py#L102-L105

Closing the issue.