search

aidanheerdegen / publish_cosima_data

0 stars 1 forks source link

Minor date issues #10

Closed AndyHoggANU closed 5 years ago

AndyHoggANU commented 5 years ago

Splitvar is now working well with most data. There is a minor residual issue with the time slices that are used. We are expecting file to be split by year. In the ice files, it gives us names like:

hi-m_access-om2-025_219801_219812.nc 
hi-m_access-om2-025_219812_219912.nc
...
hi-m_access-om2-025_225612_225712.nc
hi-m_access-om2-025_225712_225801.nc

This seems to be slightly inconsistent, but we can probably live with filename inconsistencies.

When I dig into those actual dates, I find that there are 11 time entires in the first file, 12 in all others, except for the last file which has 1. That is:


hi-m_access-om2-025_219801_219812.nc:  * time         (time) datetime64[ns] 2198-02-01 2198-03-01 ... 2198-12-01
hi-m_access-om2-025_219812_219912.nc:   * time         (time) datetime64[ns] 2199-01-01 2199-02-01 ... 2199-12-01
...
hi-m_access-om2-025_225612_225712.nc:  * time         (time) datetime64[ns] 2257-01-01 2257-02-01 ... 2257-12-01
hi-m_access-om2-025_225712_225801.nc:   * time         (time) datetime64[ns] 2258-01-01```

I infer that what has happened here, is that dates in CICE are posted at the end of the month, and splitvar interprets this as being in the following month. I think this could all be fixed by offsetting forward by half a month? Or even a few days? Or is there are more transparent way to do this?
aidanheerdegen commented 5 years ago

I tried some tests and couldn't reproduce. Can you try again, and push your current version so I'm sure we're running with the same files.

aidanheerdegen commented 5 years ago

@AndyHoggANU try adding --usebounds option to the ice processing and see if that fixes the issue (Spoiler: it should)

I've started the job that will update conda, but it is waiting on another job to complete. When you can see splitvar 0.2.2 in conda list you should be good to go.

AndyHoggANU commented 5 years ago

Giving it a burl now...

aidanheerdegen commented 5 years ago

This code change fixes the inconsistency in date formatting for the ice data

https://github.com/coecms/splitvar/commit/a5afc1b37c7bf623cae80d4c48545ab0d68ecfec