aria-jpl / coseismic_usgs_neic_evaluator

USGS NEIC evaluator
Apache License 2.0
0 stars 0 forks source link

FAQ #1

Open riverma opened 3 years ago

riverma commented 3 years ago

Is your feature request related to a problem? Please describe. Sometimes operations questions come up regarding this codebase, it'd be great to have a section within the README providing an answer to frequently asked questions.

Describe the solution you'd like An FAQ section with a sample question answered: "How do I (re)generate aoitrack-earthquake datasets manually using this PGE?"

Describe alternatives you've considered n/a

Additional context See example section from @al-niessner 's FAQ: https://github.com/aria-jpl/coseismic_enumerator/tree/readme#faq

cmarshak commented 3 years ago

Questions:

We ingest multiple EQ events eg usgs_neic_pdl_origin_7.3_us7000e54r*. We have suffixes _pending, _orange, and all in our system.

  1. Do we run the displacement evaluator on all three events?
  2. If yes to 1., do we have a way of deduping/short-circuiting other PGEs downstream so we don’t re-run topsapp?
  3. If no to 1., what happens to the three events and how do we determine which event has associated AOITracks?
  4. If yes to 2., how is the short-circuiting occurring? We know that topsapp does have short circuiting for GUNWs (meaning if we have two ifg-cfs with the same SLC data, we won't reprocess the same GUNW unless the two jobs were spawned in close succession of one another and then we will likely get clobber errors from AWS).

Other questions:

  1. How and where is the "extended event polygon" created? Which PGE is this polygon created in specifically? The circle in this image is an example of this so-called extended event polygon.
  2. Can the radius of the "extended event polygon" be changed? Is this radius dependent on the size of the EQ?
  3. What are the current filters that are being used to generate AOITracks (aoitrack-earthquake) from events (usgs_neic_pdl_origin)?
  4. Can such filters be modified and how?
  5. What is the naming convention of AOITracks?
  6. What is the difference between these two AOITracks?
    • AOITRACK_eq_usgs_neic_pdl_us7000dfku_56_Trnavos_Greece_102
    • AOITRACK_eq_usgs_neic_pdl_us7000dfku_58_Verdikossa_Greece_102
marjo-luc commented 3 years ago

We ingest multiple EQ events eg usgs_neic_pdl_origin_7.3_us7000e54r*. We have suffixes _pending, _orange, and all in our system.

1. Do we run the displacement evaluator on all three events? This is a David question. These are all distinct products ingested from USGS. https://earthquake.usgs.gov/data/ground-failure/background.php

2. If yes to 1., do we have a way of deduping/short-circuiting other PGEs downstream so we don’t re-run topsapp? Running the displacement evaluator over these different datasets should not produce new acqlists unless new data were ingested. If topsapp jobs you feel should be deduped are not deduping, then that is a different issue.

3. If no to 1., what happens to the three events and how do we determine which event has associated AOITracks? NA

4. If yes to 2., how is the short-circuiting occurring? We know that topsapp does have short circuiting for GUNWs (meaning if we have two ifg-cfs with the same SLC data, we won't reprocess the same GUNW unless the two jobs were spawned in close succession of one another and then we will likely get clobber errors from AWS). This is part of hysds-core.

Other questions:

**How and where is the "extended event polygon" created? Which PGE is this polygon created in specifically? The circle in this image is an example of this so-called extended event polygon. Can the radius of the "extended event polygon" be changed? Is this radius dependent on the size of the EQ? The radius is a function of the magnitude. You can change it here: https://github.com/aria-jpl/coseismic_usgs_neic_evaluator/blob/95a7448895e2f383eec84e100e751b16e2fd4774/event_evaluator.py#L341

What are the current filters that are being used to generate AOITracks (aoitrack-earthquake) from events (usgs_neic_pdl_origin)?** Some are specified in the cron job inputs, others in the code/config.

Can such filters be modified and how? Yes. Most of the filters are defined here: https://github.com/aria-jpl/coseismic_usgs_neic_evaluator/blob/main/event_evaluator.py

What is the naming convention of AOITracks? AOITRACK_eq_usgs_neicpdl if there are issues with the extraction of the data from the pdl event, it defaults to AOITRACK_eq_usgs_neicpdl You can find more details here: https://github.com/aria-jpl/coseismic_usgs_neic_evaluator/blob/main/event_evaluator.py

What is the difference between these two AOITracks? AOITRACK_eq_usgs_neic_pdl_us7000dfku_56_Trnavos_Greece_102 AOITRACK_eq_usgs_neic_pdl_us7000dfku_58_Verdikossa_Greece_102 The earthquake magnitude. One is M5.6 -- the other is M5.8.

cmarshak commented 3 years ago

We ingest multiple EQ events eg usgs_neic_pdl_origin_7.3_us7000e54r*. We have suffixes _pending, _orange, and all in our system.

1. Do we run the displacement evaluator on all three events? This is a David question. These are all distinct products ingested from USGS. https://earthquake.usgs.gov/data/ground-failure/background.php

I think @dbekaert could eventually advise. I still don't think I have an answer for my original question. I thought we facet on one of the events to produce an AOITrack. Is this correct? Or do we single out one of them? Or will each of these events produce the exact same AOITrack dataset and that's why we don't care? I think you are hinting at this in the answer to your next question.

Looking at the event metadata in AOITracks (there are 6 produced here), we have something like this:

..."event_metadata": {
"depth": 10,
"event id": "us7000e54r",
"label": "M 7.3 - Southern Qinghai, China",
"latitude": 34.5864,
"location": "Southern Qinghai, China",
"longitude": 98.2548,
"magnitude": 7.3,
"orbit_direction": "asc",
"pager_status": "orange",
"track_number": 26,
"tsunami warning": 0,

Is it coincidence we select orange the current state of the system? I just want to get confirmation.

  1. If yes to 1., do we have a way of deduping/short-circuiting other PGEs downstream so we don’t re-run topsapp?

    Running the displacement evaluator over these different datasets should not produce new acqlists unless new data were ingested. If topsapp jobs you feel should be deduped are not deduping, then that is a different issue.

Ok, maybe in this answer, all the different events are producing a AOITrack, meaning the _orange and _pending produce the same AOITrack (or maybe there is some other de-duping). Again, just want to get understanding of how the system is working.

  1. If yes to 2., how is the short-circuiting occurring? We know that topsapp does have short circuiting for GUNWs (meaning if we have two ifg-cfs with the same SLC data, we won't reprocess the same GUNW unless the two jobs were spawned in close succession of one another and then we will likely get clobber errors from AWS).

    This is part of hysds-core.

A general explanation as it relates to this PGE would be appreciated.

**How and where is the "extended event polygon" created? Which PGE is this polygon created in specifically? The circle in this image is an example of this so-called extended event polygon.

Can the radius of the "extended event polygon" be changed? Is this radius dependent on the size of the EQ? The radius is a function of the magnitude. https://github.com/aria-jpl/coseismic_usgs_neic_evaluator/blob/95a7448895e2f383eec84e100e751b16e2fd4774/event_evaluator.py#L341

Could you explain/document the said formula?

Can such filters be modified and how?

Yes. Most of the filters are defined here: https://github.com/aria-jpl/coseismic_usgs_neic_evaluator/blob/main/event_evaluator.py

I am seeing the readme on the front of this repo.

I am seeing you expose the positional arguments through some hysds and the job spec.

I see these in the on-demand GUI:

image

Would be nice to have some explanation of these are roughly doing for an operator especially if they are changed. For example, assuming the distance from land is in km's? What does the create_aoi_version do? How does days_pre_event and days_post_event relate to this PGE? Why is the water_mask parameter an integer?

riverma commented 3 years ago

@rverma to scope out the issues and recommend which should go in a FAQ.

riverma commented 3 years ago

Reassigned to @marjo-luc to make the decision of what goes in the FAQ (generic questions).