Guides / Documentation for PhysRisk Engineers, Modelers, and Administrators

[ericbroda]

Is your feature request related to a problem? Please describe.

This is a request to create/enhance documentation

Describe the solution you'd like

Problem Statement: The model developer experience today requires significant knowledge of the inner workings of the physrisk code base and the various data formats

The Goal: Documentation and utilities should be available to allow personnel with reasonable experience (python, modelling etc) to quickly ("in 5 minutes or less"):

• Get started quickly: get basic components working in their local environment easily and quickly
• Create/on-board data quickly: new ZARR (and geospatial indexes etc) can be created easily and quickly
• Create new models quickly: New models can be created easily from a "cookbook"; Existing models can be configured quickly and easily
• Monitor and manage a physrisk environment easily: Docker containers can be created easily for any component; Components can be easily started ("docker-compose up" or similar); Component state canbe easily monitored

Describe alternatives you've considered

Current methodology documents are well established. Recognizing the methodology is somewhat complex, it is expected that the code that implements this methodology by its nature will be also be complex. This complexity is currently addressed by creating:

• more documentation
• more test cases
• more notebooks

This is the current approach, but more needs to be done achieve our goal.

Our request is to augment existing documentation efforts and create/structure our documentation (and supporting utilities) into several guides, each targeted at specific (common) user groups:

• Getting Started Guide: Lets a modeler start the physrisk components and get sample models (with established sample data) working in 5 minutes or less all running on their local desktop/workstation or cloud instance
• PhysRisk Data Guide: Lets an engineer should be able to easily understand what data is available and how to view/query that data, again, in 5 minutes or less; an engineer should be able to create a loader for a new data source based upon several available well documented sample loaders
• PhysRisk Model Developer Guide: Lets a modeler create a brand new model, configure sample data, and execute their model using only resources on their desktop/workstation (ie. sample data infiles/ZARR on their local machine); Once done, a modeller can connect to "live" data sources to further test their models
• PhysRisk System Admin Guide: Lets an engineer build Docker image from code; Lets an engineer start and test the full set of physrisk services on their local environment (their workstation, their cloud instance); Lets an engineer/operator monitor state of the physrisk system components

Additional context

No additional context noted

[joemoorhouse]

Hi @ericbroda,

Also FYI @xbarra and @NickKellett.

Good initiative to add to the docs. As discussed, I think next step is to check what goes where, In https://physrisk.readthedocs.io/en/latest/ we currently have

• Getting started (that is indeed in line with the above so easy)
• Methodology
• User guide
• API reference I would suggest we keep that structure as is quite nice and simple. I think Data Guide, Model Developer Guide and System Admin should all be chapters within the User guide. Does that work for everyone? If so, I'll set up along those lines to encourage contributions.

Docs are now using .ipynb apart from a bit of glue logic. I think that is also good to encourage contributions as most people happy to write a notebook.

Thanks, Joe

[joemoorhouse]

Hi @ericbroda; hi @NickKellett,

By the way, I'm also writing more on hazard indicators storage. This includes the current Zarr set-up, but I also want to write this to include your spatial indexing work. The flow is that:

• Zarr format is our common currency.
• Hosting that directly on S3 is our go-to that gives acceptable performance in a number of cases (expanding on quick wins like put-it-on SSD-attached-to-compute instead).
• We then have the database-based, spatial indexing approach that we can use to give a substantial speed up, especially in the case of sparse data. And there idea is to have that behind an API and integrate into physrisk.

Would you have a half-pager or so I can integrate there on the spatial indexing part (I think we can then expand on that later)?

I think to start with I want to cover the main idea (i.e. which DB are we using, how it works with sparsity and how the smart look-up works.

Thanks, Joe

[joemoorhouse]

hi @ericbroda, @xbarra,

After (in-person) feedback, the above plan for sections seems a good place to start, so I restructured; please see updated here: https://physrisk.readthedocs.io/en/latest/ Please take a look, especially at the user guide part to check structure is OK. I also moved to ReadTheDocs theme by the way, mainly as seemed to do better job for table display.

Thanks, Joe