First column in downloaded timeseries CSV file should in time units

[tmcphillips]

Currently the first column of a downloaded CSV file is titled ‘index’, and the values correspond to the zero-based index of each time point in the series rather than to the time (e.g. year) at that time point. For data sets that are known to be in time units of years, the first column should be named 'year', and the values should reflect the year corresponding to each point in the time series. For the full PaleoCAR run the values in the year column should run from 1 to 2000 (currently the first column is titled 'index' and the values range from 0 to 1999).

If the requests to the timeseries service are posed in terms of years (e.g. by including 'timeResolution=year&timeZero=1' as part of the query line for the PaleoCAR data set), the CSV field in the response from the service automatically will satisfy the above requirements.

[Zodiase]

@jterstriep Should this be part of the ES document? I don't see the need for the client to change either timeResolution or timeZero.

[tmcphillips]

@jterstriep Is there a plan to address soon in Elasticsearch documents for the datasets? I am very worried that once the configuration is changed on the ES side, we will find bugs that need to be fixed either in the timeseries service or in the client to make this work, adding more delays to the the release.

[jterstriep]

First, everyone can make this change. It's a simple change the analytics.json document for each dataset. It should be fairly self-explanatory in the json document (i.e. change the URL parameter). There is documentation in the README of the Datasets repository.

Second, you should be able to test this without actual changes to the Elasticsearch DB using a request generated by curl.

If you need help with either of these please let me know.

[tmcphillips]

Is there documentation for the proper way to reload the documents into Elasticsearch, clearing old data in Elasticsearch if needed, etc?

[jterstriep]

Actually, I take that back. I'm changing Datasets to better reflect how I want things done. I'll take care of providing the necessary fields.

I'm going to close this issue since it's unrelated to the user interface.

[tmcphillips]

Can we just make things work for this release--today--and then change Datasets as you envision later?

[ludaesch]

I concur: Can we please have the "fix" right now? (it probably won't take very long, right?) Then let's just do it. After the beta release we can redesign the dataset schema...

[jterstriep]

I'm fixing things as expeditiously as possible.

On Fri, May 18, 2018 at 4:37 PM Bertram Ludaescher notifications@github.com wrote:

I concur: Can we please have "fix" right now? (it probably won't take very long, right?) Then let's just do it. After the beta release we can redesign the dataset schema...

— You are receiving this because you modified the open/close state.

Reply to this email directly, view it on GitHub https://github.com/openskope/skope-interface/issues/176#issuecomment-390339209, or mute the thread https://github.com/notifications/unsubscribe-auth/ACdK7MbcERnZXJZXQojSn7z0Vv_VnGg_ks5tzz8mgaJpZM4TusA_ .

[Zodiase]

Client should be sending years instead of band IDs.

[Zodiase]

Should be resolved by 631251a3c52321ff5895f6351ce7c1297b65005f which should be out in v0.1.19.