Revise docs organization

[j08lue]

The current table of contents / sidebar of the docs has grown organically so far and could use a revision. Any structure we create we should also be reflected in the repo folder structure and file names and headings be aligned, so users can quickly find the same content across rendered docs and repo.

In the sidebar, it would be nice if some of the example notebook titles were by default visible, so it is more obvious for users at first glance what kinds of resources are available. I think users will more easily recognize what the headers mean if they can see at least a couple of samples for each. Since we do not have that much content yet, I guess we can just list them all.

Planetary Computer distinguishes between Quickstarts, Tutorials, Concepts, and Datasets. No shame in imitating that, but we should put Datasets in the TOC, too.

I suggest something along these lines:

.
├── Welcome
├── VEDA APIs (<- VEDA STAC API and Browser)
├── Getting JupyterHub Access (<- VEDA Analytics JupyterHub Access)
├── Geospatial Cloud Computing (<- External Resources)
└── Usage Examples (<- Example Notebooks)/
    ├── Concepts/
    │   ├── List STAC collections
    │   ├── Open and plot COGs
    │   ├── Open and visualize COGs
    │   ├── Visualize zarr
    │   ├── Searching VEDA STAC API and inspecting catalogs with intake
    │   └── Get Fire Perimeters from an OGC API
    ├── Tutorials/
    │   ├── Downsample zarr
    │   ├── Calculate timeseries from COGs
    │   ├── Get map from COGs - NO2
    │   ├── Get timeseries from COGs
    │   ├── Get tiles from COGs
    │   └── GIF generation using the TiTiler /cog/crop endpoint
    └── Datasets/
        └── Visualizing Ocean NPP time series for seasonal patterns

Acceptance criteria

• [x] As a user, I can at first glance get an overview of different types of usage examples available, so I can find content of interest to me without clicking around a lot.
• [x] As a user, I find the headings coherent and descriptive of jobs I want to get done with Analytics, so it is easy for me to understand what information about the service the docs contain.

[jsignell]

Just as a quick note the current structure splits the notebooks into those that use a more pythonic interface to grab the data from STAC and just work with it directly in the notebook (these need to run on a nasa hub - 2i2c or SMCE) and those that use the RASTER API to do computation server-side (these can run on binder).

I know that division isn't at all clear to the naked eye, but it would be nice to find a way to maintain the separation between these two very different approaches.

[j08lue]

I know that division isn't at all clear to the naked eye, but it would be nice to find a way to maintain the separation between these two very different approaches.

@jsignell I deliberately removed that distinction, thinking that users may not understand the difference (they may not even know what the Raster API is and how it is different from the STAC API). 🤷

But I do not know that for a fact and am happy to leave it up to you to change titles or arrangement somehow.

[kathrynberger]

I know that division isn't at all clear to the naked eye, but it would be nice to find a way to maintain the separation between these two very different approaches.

@jsignell I deliberately removed that distinction, thinking that users may not understand the difference (they may not even know what the Raster API is and how it is different from the STAC API). 🤷

But I do not know that for a fact and am happy to leave it up to you to change titles or arrangement somehow.

As a new and naive VEDA user, I think it's going to be important to point this information out - as, admittedly, it tripped me up. Are these notebooks always going to be run in the Jupyter Hub account? If not, it at least makes sense on the 'Basics' landing page that the 'pythonic' approach requires running a NASA hub (i.e., 2i2c). I think we could keep the proposed naming scheme the same, but just add a disclaimer on that page. Just a thought.

[j08lue]

Makes sense, @kathrynberger. Maybe we could reflect the approach in the title of the notebooks:

• Open and plot COGs from STAC assets
• Get map from COGs using map tiles API - NO2

Then we do not need to have a different hierarchy for the two approaches but make it clear from the title (and perhaps a brief description at the beginning of the notebook).

[jsignell]

Yeah I know it's confusing for a new user, definitely agree that what we currently have is not great. We are basically presenting 2 ways of doing each science goal. I just want to keep those stories separate if possible. Maybe the solution is to take them out of the hierarchy, give them longer and highly standardized titles (like Jonas is proposing), and then present several paths through the example on the landing page or something. Something like a "server-side" and "client-side" tutorial.

[kathrynberger]

So I've had some great discussions with @j08lue this morning about this framing and my experience with VEDA docs as a new user. Below, I've tried to outline some thoughts on how we might restructure, reorganize things a bit.

Some observations/ideas at a much higher level, before we rename/organize the repo:

• Who are the intended users here? We provide an External Resources page with reference links and introductory geospatial data science tutorials, etc - but we are kind of assuming they already know what STAC catalogs/differences between server/client side, etc.
• It is _more _likely__ that users will be more familiar with geospatial data and analyses, but new to cloud optimized geotiffs and how to use them. Adding a link to a page like this would be very helpful in a section like the Geospatial Cloud Computing proposed at the top of this ticket.
• Adding a very brief intro and distinguishing between the STAC catalog, the pythonic version, and the RASTER API would be extremely useful. This would be particularly helpful on the VEDA STAC API and Browser page.
• I really see these VEDA docs as a fantastic resource for education/exploration/etc, specifically getting users to use COGs, etc vs. traditional download the data and analyze it on your machine (which is still a large crowd of users out there!)
• I've added some brief thoughts, specifically on the Example Notebooks landing page for reorganization

In terms of organizing the notebook themselves:

• I do like what has been proposed above in terms of Concepts, Tutorials, and Datasets. However, still looks like it is a renaming of the Basic vs. Using the Raster API divide that we have currently. Is there anything we can do to explain when and why one would use one approach vs. another in the Geospatial Cloud Computing page? This would make our lives much easier.
• It would also make @j08lue's proposed longer renaming of the notebooks (which explicitly name which approach was used) that much clearer for end users to understand.

[kathrynberger]

An annotated version of Jonas's outline above with some additional proposed changes:

Welcome

VEDA APIs (<- VEDA STAC API and Browser)

• Here, we explain the purpose of each API, including when and why you might use one over the other

VEDA's Analytics Platform: JupyterHub (<- VEDA Analytics JupyterHub Access)

• Explain JupyterHub's purpose
• Explain dataset access requirements (i.e., you can access all datasets in this environment, outside of this environment you will be limited to what you can do locally or in Binder notebooks)
• Provide information on how one can gain access/credentials to JupyterHub

Geospatial Cloud Computing and Geospatial Analysis Resources (<- External Resources)

• In addition to the great list of external resources currently provided regarding python, git, and geospatial analyses, we provide links materials introducing STAC, COGs, etc

Usage Examples (<- Example Notebooks)/ |──── Concepts/ |──── Tutorials/ |──── Datasets/

Notes on above: Overall, I like @j08lue's proposed revision to the structure. I think the only issue and clarification needed is the subtle difference (and how best to explain it) between what is currently 'Basic' vs 'Using the Raster API'.

I would like the idea of separating concepts and tutorials like proposed above, but don't know how we untangle the notebooks as they currently area. The separation @j08lue described at the top of this ticket renames 'Basic' to Concepts and 'Using the Raster API' to Tutorials so far as I can tell. Is this true?

Once we sort this out, I think the longer naming of the notebooks as prescribed above will help greatly as well.

[jsignell]

However, still looks like it is a renaming of the Basic vs. Using the Raster API divide that we have currently. Is there anything we can do to explain when and why one would use one approach vs. another in the Geospatial Cloud Computing page? This would make our lives much easier.

I can try to address this question. The difference is mostly around permissions, where computation happens, and what the desired output is.

Using the Raster API:

For when you want to show outputs to other people or do standard processing

• no permissions required (aka can run on mybinder)
• computation happens somewhere else (user does not have to think about instance size)
• suitable for use within notebooks and from web app frontends (like dataset discoveries)

Accessing data directly (formerly called "Basic"):

For when you need to do niche data analysis or access the raw data

• permissions required (ideally run from 2i2c or SMCE)
• computation happens on the user's instance (user needs to think about instance size)
• suitable for use within notebooks

[jsignell]

I am looking at the planetary computer docs and if we borrow their hierarchy I think most of our notebooks actually map to Quickstarts with some of the longer ones being more of a Tutorial and the ones that Kathryn is doing being Datasets.

[kathrynberger]

@jsignell This is a fantastic description, thank you. I do think that separating out these explicitly for the end user will be helpful. Thanks for highlighting the differences. 🙌 I really like that idea described above. Let's rename it with the following under Usage Examples:

|---Quickstarts/

• Using the Raster API
• Accessing the data directly

|---Datasets

Not sure what notebooks would map out to tutorials just yet, but like the proposed breakdown above

[jsignell]

That sounds like a great plan to me! The one that I am thinking is maybe a turtorial is the GIF generation notebook.

[kathrynberger]

In total agreement with you Julia, that gif generation example was one of the few I think we could put in under the 'Tutorials' heading. Awesome, I'll get started on reconfiguring the repo under this approach/revised outline. 👍

[jsignell]

@kathrynberger are you planning on updating the titles of the notebooks as well before closing this ticket or should I close it now?

[kathrynberger]

That's a good question: I hadn't planned on changing the titles of the notebooks as they sub-headers more accurately described the access route to the data and the split (accessing the data directly vs. using the raster API) between the notebooks was unchanged. Happy to revise if you'd like, but hadn't anticipated to do so with the latest version.

[j08lue]

I consider this ticket as completed, thank you @kathrynberger.

It would be great to rename the actual Markdown and notebook files to match their titles and have the same folder structure, so the repo is easy to browse and less cluttered when we clone it into JupyterHub environments. I will open a separate ticket for that.