Make animation of a 48-hour period

[ehmatthes]

This is the main issue for making an animation of 48-hour periods of rainfall. Will want to make several animations focusing on periods where a landslide occurred, and periods that came close to being critical but where no slide occurred.

[ehmatthes]

Tasks

• [x] Identify a set of readings to focus on.
  • Start with current readings. Will need 48 hours of readings before start point as well.
• [x] Adapt existing plotting code to generate animation.
  • Use plot_critical_forecast_mpl_extended() in utils/plot_utils_mpl.py
  • Simple approach first: generate png files, use ImageMagick to generate mp4 from png files.
  • [x] fix ordering
    • maybe issue with memory warning? -> close handle to image file before returning from fn
    • was simple issue of ffmpeg using alphabetical ordering, so 1 10 11 12 2 20 21...
    • solution: zero-pad numbers in filenames

[ehmatthes]

• (not critical) adjust so it's an actual 48-hour period
  • [x] should adjust source file, so can rerun animation plotting code
  • for 48 hour period at 15-min intervals, need 48 hours before and 48 hours after initial lead point (far right point on plot)
  • this is 48*4 + 48*4 = 384
  • for hourly readings, this is simply 48*2 = 96
• [ ] If going back to this, already have analysis_utils.get_recent_readings() that should make this easy.

[ehmatthes]

• [ ] Generate animation for Kramer slide event.
  • [ ] Grab readings, probably from irg_analysis work.
  • They're in sitka_irg_analysis/other_output/reading_dump_mmddyyyy.pkl files, but probably need more readings?
  • [x] Rerun code to generate pkl file, generating 96 hours of readings rather than 48. Probably need 3*48, or 4*48.
  • Note: Seems to work best to set start_index=72, end_index=24 in sitka_irg_analysis/analysis_utils.get_48hr_readings().
  • [x] Adjust animation code to work from either .txt file (usgs) or .pkl file.
  • [ ] This really needs to show when the slide occurred.
  • [ ] But this is good enough to reach out to sssc team about making the real-time analysis public.
  • [ ] Perhaps putting the animation code in the historical project would be better.

[ehmatthes]

• [ ] Choose other significant periods to animate.

[ehmatthes]

Refinements

• [ ] For historical animations, may want to plot critical region for as much of the period shown as possible, ie ~40 hours rather than just last 6.
• Could use multiprocessing. Create a function that takes in start and end index, then calls pcfme(). Use pool to call that function.

[ehmatthes]

Moving this work to sitka_irg_analysis.

[mansourhas]

hi @ehmatthes . i'm very interested in your project. i couldn't flow along. could you provide a readme doc explaining what is the project is heading at

and pleas let me know how can i help

[ehmatthes]

Hi @mansoor96g. What interests you about this project?

[mansourhas]

i want to contribute to this project to enhances my datascience and machine learning skills.

i recently finished coursera applied data science specialization. and this project uses matplotlib (my favorite) with a real data in a web app. pules it has the right size and complexity for me to be challenging enough but not overwhelmed.

what i could do:

• i could build more interactive visualizations ( although i'm not sure how that is done in a web environment) .

• i can apply data analysis and put more emphasis on the massage intended or provide a supplement summary charts .

• provide a dashboard to interact with the charts.

although i'm not a developer per say. i hope that it's possible to apply my skills sit and learn at the same time.

and thanks for you response

[ehmatthes]

Hi again @mansoor96g. Thanks for clarifying. This web-focused project isn't really ready for collaboration yet. If you are intested in this project, this is a better one to contribute to. That project is an analysis of historical data that justifies this project. That project is in a pretty messy state because I just finished the exploratory phase of the project. I'm going to clean up that project over the next few weeks, and write up the process. If you want to get involved, feel free to open a "how can I help" issue on that project, and I'll fill you in more there.

There has been some suggestion to apply ML techniques to that dataset, and if you're interested I'd be happy to see any results you obtain in that direction. I'm not sure if it's a large enough dataset to benefit from ML techniques, but I don't know much at all about practical ML application yet.

Also, that project does have a proper readme that will help you get started.

[mansourhas]

thanks.

i will defiantly follow your advice check that out.