Abstract date handling to another package?

[rafaqz]

Your DateTime types and handling seem to be generally useful for attaching to any spatial datasets with a temporal dimension.

Abstracting them to another package would mean those types could be used to specify date time standards in any geo array type, without needing the binary and conda deps associated with this package.

I'll do the work if it makes sense to you to do.

[Alexander-Barth]

It easier for me to keep them together as there are less moving parts. Conda deps is indeed a big dependency and it would be great to remove it, but I am not aware of a better way. Also NetCDF.jl currently requires this package. I think the number using users interested in the dates following the CF convention, but not interested in loading NetCDF files is too low to make this change.

[rafaqz]

Sure. I was trying to plan out a spatial interface where metadata like calendar and projection that are common between spatial data can stay attached if it needs to be modified, moved to memory, has some dimensions extracted etc. So we can just plot, transform etc anything using the same tools.

I'm thinking that to allow consistency across spatial packages without conda dependencies everywhere, we need some kind of Base package(s) that hold the types, but I could be wrong. Anyway, I'm going to make the base packages for all my own spatial types to depend on, feel free to use them if you like.

[Alexander-Barth]

Yes, I think I will be quite likely a user of these package. For your package, it might be useful to have a look a xarray (http://xarray.pydata.org/en/stable/), but you know it probably.

[rafaqz]

Thanks I actually don't! I haven't use spatial tools in python, but that's pretty much what I'm thinking of. Except more an interface without so much implementation - we just need to agree on shared types and method names and use existing packages to do the rest.

[Alexander-Barth]

What I like about xarray (and from what I have seen in the documentation, without being a user of xarray, so take this with a grain of salt)

• handle n-dimensional arrays from the start (too often only lon/lat are considered and all other dimensions are an after-tough)
• coordinate values are just arrays: this is much more flexible than just supporting a fixed list of projections. For projections, like the Mercator projection, I would probably define a "virtual array" type, where every lon/lat value is computed once it is indexed. Optimized routines can be handled with multiple-dispatch (I am not sure if this is clear and I do not think that xarrays does this actually).
• generalization of data frames for multi-dimensional arrays (so not specific to geospatial data, but still very useful for such datasets)

Actually, I noticed that xarray also supports non-standard dates using the standalone-package cftime (formally part of the python's netCDF4 library).

http://xarray.pydata.org/en/stable/weather-climate.html#cftimeindex https://github.com/Unidata/cftime

So maybe it is not a bad idea after all, to make a separate package... and maybe needed anyway in the long-run.

[rafaqz]

I just noticed that too! Might be a good idea ;)

I'm totally on board with doing N dimensions properly - I need 4 a lot of the time, even 5. We just need a standard way of indicating what the dimensions are and what order, then any spatial dataset could be converted to another format fairly easily.

I wrote my thoughts on this here: https://github.com/rafaqz/GeoArrayBase.jl/blob/master/src/GeoArrayBase.jl#L51-L88

Your array idea sound good too, especially the lazy computation. I was hoping to just define a coordinates() function and leave those details up to the implementation.

There are some more discussion of some of these things here, if you haven't already seen it: https://github.com/mkborregaard/VerySimpleRasters.jl/issues/1

Edit: thinking about cftime, I think a more general version could be more useful in julia - GeoTime or something - that could hold any geospatial tiime standards, with CF as the main components.

[rafaqz]

Abstracted to a GeoTime package: https://github.com/rafaqz/GeoTime.jl

It just needs a few docs.

I kept all your commits and license copyright, and I can give you push rights, or move the package somewhere better.

[Alexander-Barth]

I was doing the same thing, about the same time: https://github.com/Alexander-Barth/CFTime.jl How shall we proceed, do you want to take the package, or shall we work on it jointly (maybe under the JuliaGeo organization)? I have some users for a dependent package still relying on Julia 0.6 and we are working together on a project. It is OK for you to keep Julia 0.6 support for a while?

[rafaqz]

Oh that's funny. Maybe use your version then, I deleted the 0.6 stuff.

What do you think about making the name more general, so that it can hold other datetime standards as well if they come up? the CFTime name might discourage that.

Also yes, JuliaGeo is probably a good place for this.

[Alexander-Barth]

What kind of other calendars are you thinking about? To me list of calendars in the CF standard looks quite extensive. In any case, if somebody want to add Martian DateTime, that would be really cool, despite the package called CFTime :-)

[Alexander-Barth]

@yeesian @visr Do you agree that it is a good idea to move a package about handling dates from the climate and forecast convention (mostly used with NetCDF files) to the JuliaGeo organization? Currently the code is available here: https://github.com/Alexander-Barth/CFTime.jl

[visr]

Nice work on CFTime.jl, @Alexander-Barth. Indeed I'm also an xarray / cftime user. Makes sense to me to host CFTime.jl in JuliaGeo, feel free to move it there. Of course I hope you'll be able to maintain it there as well. CF time is clear to me, what exactly a geo time would be, I don't know. First thing I thought was geologic time :)

[Alexander-Barth]

Martijn, thank you for inviting me to JuliaGeo! I have transferred CFTime.jl to JuliaGeo and yes, this will maintain it in JuliaGeo. https://github.com/JuliaGeo/CFTime.jl