Closed kkappler closed 1 month ago
@kujaku11 when we merge this in, we may want to invert the try/except from
try:
df.start = pd.to_datetime(df.start.str.decode("utf-8"), format="mixed")
df.end = pd.to_datetime(df.end.str.decode("utf-8"), format="mixed")
except ValueError:
df.start = pd.to_datetime(df.start.str.decode("utf-8"))
df.end = pd.to_datetime(df.end.str.decode("utf-8"))
to
try:
df.start = pd.to_datetime(df.start.str.decode("utf-8"))
df.end = pd.to_datetime(df.end.str.decode("utf-8"))
except UNKNOWN_EXCEPTION:
df.start = pd.to_datetime(df.start.str.decode("utf-8"), format="mixed")
df.end = pd.to_datetime(df.end.str.decode("utf-8"), format="mixed")
but I do not know what the Exception was that you encountered, nor do I have an example dataset with the error.
Since we ran fine for years the "mixed" option, then probably most data will not need it, so it might be slightly faster to invert. That said, these timestamps are in the channel_summary table, which is never very big so maybe it doesn't matter.
A recent change was made regarding time formats, adding the keyword argument
format="mixed"
tochannel_table.py
.While this may have fixed the specific instance of data, aurora is failing a lot of tests with messages like:
Another tool that uses mth5 to build datasets is hitting the same issue.
I will try wrapping this in a try/except so it uses both the old and the new command on
patches
branch and see if that fixes it.