Tidy up "Analysis ready data"

ACDguide / BigData

Working with big/challenging data collections

https://ACDguide.github.io/BigData

Other

6 stars 5 forks source link

Tidy up "Analysis ready data" #94

Open paigem opened 1 year ago

paigem commented 1 year ago

Analysis ready data section

Name is not clear what this page is
Lots of overlap with content in the “Software list and advice: Python” page
What content should be on this page (if any)?

paolap commented 1 year ago

I'm thinking of changing this to

Data catalogues: i.e. resources to find and access data which are currently available, so mostly new content but moving the content on existing intake catalogues here + referring to platform like pangeo which are "data ready" I would also include ready to use analysis envs as the conda envs (possibly move them from software) and again refer to any platform like pangeo which includes installed software. This might eventually include esmValTool ? ILamb? other benchmarking/testing environments? Climdex (there's a tool on the website to calculate climate indexes, post processing tools. ???

Data access tools: Here we could cover the tools available to make the data ready, which is more like what the current session is. However, I agree with Paige that this might sit better in the software part.

paigem commented 1 year ago

Notes from our discussion:

Move all of Intake/STAC discussion to software section
Start "Accessing data" section with "where to find data"
List condo environments in "accessing data", but link to them from their respective computing platform page
Link to intake in accessing data. Can have a sentence saying that you can write your own intake catalog, and link out to software list.
Move the communities list to its own category under "working environments"
Add ACCESS-NRI to communities

Update "accessing data" name into two categories:

Software environments
Data catalogs

We should create the categories as if writing a story from start to finish - what would a user need to know to do their analysis?

hot007 commented 1 year ago

@paolap re tools, in CSIRO we're iterating toward some standards for extremes, although there's different tools needed in different cases... anyway, @AliciaTak is doing a lot of extremes work so if you do decide to include ilamb and climdex then Alicia's contributions would be useful in that section :)

AliciaTak commented 1 year ago

@hot007, thanks for thinking of me. I do more extreme value analysis than looking at extreme indices. But I am more than happy to take over these sections if needed.