GEUS-Glaciology-and-Climate / pypromice

Process AWS data from L0 (raw logger) through Lx (end user)
https://pypromice.readthedocs.io
GNU General Public License v2.0
12 stars 4 forks source link

Weird time stamp at THU_U makes the processing crash #168

Closed BaptisteVandecrux closed 10 months ago

BaptisteVandecrux commented 10 months ago

In the logger file THU_U_2021_raw_SlimTableMem.txt line12949 there is the timestamp "2021-06-01 18:00:00.0128" which fails to be read by pd.to_datetime.

Consequently the time column for that file is read as type "object", which causes an error downstream during L0toL1:

Traceback (most recent call last):

  File "C:\Users\bav\AppData\Local\Temp\ipykernel_99968\1977842627.py", line 1, in <module>
    pAWS_tx.getL1()

  File "C:\Users\bav\Anaconda3\lib\site-packages\pypromice\process\aws.py", line 93, in getL1
    toL1(L0, self.vars)

  File "C:\Users\bav\Anaconda3\lib\site-packages\pypromice\process\L0toL1.py", line 46, in toL1
    ds = addTimeShift(ds, vars_df)

  File "C:\Users\bav\Anaconda3\lib\site-packages\pypromice\process\L0toL1.py", line 152, in addTimeShift
    df['doy'] = df.index.dayofyear

AttributeError: 'Index' object has no attribute 'dayofyear'

Solution is to force a pd.to_datetime after https://github.com/GEUS-Glaciology-and-Climate/pypromice/blob/86b4362c20f44ae2e0ecadcc11a146ac0a022cb2/src/pypromice/process/aws.py#L338-L343

if that to_datetime fails because of a wrong fromat, we try again with a format='mixed'