Organize files under new folder structure (similar to US)

peterdesmet commented 5 years ago

Suggestion by @adokter to change directory structure to date first, radar next:

Currently:

be/jab/2018/                12 monthly bejab201809.zip
be/jab/2018/09/28/10/       12 vp files

Proposal:

vp/2018/03/21/bejab/        12 x 24 = 288 vp files
vpts/2018/03/bejab/         1 vpts txt file

Differences:

Similar to US structure
Year first: quicker overview of temporal scope
Country + radar together, to have quicker list of all available radars for that month
No hour directories
Don't mix file types: vp vs vpts. Also needed because vp data will be organized by day ( files/day) while vpts data will be shown monthly

@CeciliaNilsson709 @baptischmi @jshamoun or any other user of the repository... comments? If adopted we'll update the bioRad functionality to access the files.

baptischmi commented 5 years ago

Changes are fine to me. And i'm looking forwards to extra functionalities to quickly visualise file availability (or even VPTS) time-wise or spatial-wise.

peterdesmet commented 5 years ago

@baptischmi, we'll be supporting that by having vpts txt files per month (updated daily)

jshamoun commented 5 years ago

Thoughts on this: Benefit is it would be similar to US Would be easier to collect data for a single date Would be harder to collect a time series for a single radar So which is better depends on the research question. Having the vpts would help a lot. Having a database would be even better ☺

From: Peter Desmet [mailto:notifications@github.com] Sent: Wednesday, 10 April 2019 10:59 AM To: enram/data-repository data-repository@noreply.github.com Cc: Shamoun-Baranes, Judy J.Z.Shamoun-Baranes@uva.nl; Mention mention@noreply.github.com Subject: [enram/data-repository] Organize files under new folder structure (similar to US) (#54)

Suggestion by @adokterhttps://github.com/adokter to change directory structure to date first, radar next:

Currently:

be/jab/2018/ 12 monthly bejab201809.zip

be/jab/2018/09/28/10/ 12 vp files

Proposal:

vp/2018/03/21/bejab/ 12 x 24 = 288 vp files

vpts/2018/03/bejab/ 1 vpts txt file

Differences:

Similar to US structure
Year first: quicker overview of temporal scope
Country + radar together, to have quicker list of all available radars for that month
Don't mix file types: vp vs vpts. Also needed because vp data will be organized by day ( files/day) while vpts data will be shown monthly

@CeciliaNilsson709https://github.com/CeciliaNilsson709 @baptischmihttps://github.com/baptischmi @jshamounhttps://github.com/jshamoun or any other user of the repository... comments? If adopted we'll update the bioRad functionality to access the files.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/enram/data-repository/issues/54, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHVIzJlrBex3fZ9oU8r9U5NUrP88Li4vks5vfafQgaJpZM4cmePU.

peterdesmet commented 4 years ago

See S3 bucket (and structure) for NOAA data: https://s3.amazonaws.com/noaa-nexrad-level2/index.html

peterdesmet commented 2 years ago

The US folder structure has the disadvantage that radars don't have their directory. A new folder structure is suggested in #65. Closing this one.

enram / data-repository

Organize files under new folder structure (similar to US) #54