search

pvlib / pvlib-python

A set of documented functions for simulating the performance of photovoltaic energy systems.
https://pvlib-python.readthedocs.io
BSD 3-Clause "New" or "Revised" License
1.18k stars 996 forks source link

get PRISM rainfall data #1293

Closed mikofski closed 1 year ago

mikofski commented 3 years ago

Problem PRISM doesn't have an intuitive API. It has rainfall and other info, but he interface is an AJAX call to a PHP backend which is hidden by JavaScript here which I can't figure it out. Maybe there's a way to use the JavaScript to get the data and add it to iotools. Then the data could be used for soiling. See HSU soiling example

Solution in iotools, get_prism_rainfall() that uses the JavaScript https://prism.oregonstate.edu/explorer/dataexplorer/js/explorer.js to get the rainfall

Describe alternatives you've considered Just point users to the website, it also has a bulk download tool. Maybe they don't want folks to use the site programmatically?

Additional context JavaScript modules used:

base URL of AJAX post call:

there's also a web service to download an entire monthly or daily grid:

maybe that's the better way to go?

kandersolar commented 3 years ago

I have a function for fetching daily precip and temp from that php endpoint. I just used the browser devtools to examine requests the webpage makes instead of trying to reverse engineer the webpage's JS though, so there's a chance it's missing some edge case. I think we should ask them before making a public function -- does anyone have a good contact at PRISM? I'll email their general contact address if not.

Note that PRISM also has gridded data available via FTP. I haven't used it myself but I'd guess it'd be pretty easy to parse with numpy or tables or something. https://prism.oregonstate.edu/documents/PRISM_downloads_FTP.pdf edit: I just noticed you linked this FTP doc as well :)

mikofski commented 3 years ago

I think we should ask them before making a public function -- does anyone have a good contact at PRISM? I'll email their general contact address if not.

PTAL @annalisemckenzie I think you know some folks there, and also you could get the code from @kanderso-nrel for retrieving rainfall using the PRISM data explorer.

Kevin, I'd also like to learn how you did it. I tried to use the dev console in Chrome, but I'm inexperienced, and didn't know where to look to get the actual payload data sent to the POST. That would be a neat trick to learn.

Thanks!

wholmgren commented 3 years ago

@mcrimmins and @ben-mcmahan from CLIMAS recommended RCC ACIS to get PRISM data through an API:

http://www.rcc-acis.org/docs_webservices.html

http://builder.rcc-acis.org/

Thanks guys!

Like most things that are more about general weather and less about irradiance, I'm a little skeptical that pvlib is the right place for this. Maybe MetPy/Siphon or another package?

kandersolar commented 3 years ago

didn't know where to look to get the actual payload data sent to the POST

Sent via email (it's not hard, but I'm still nervous about sharing publicly without permission from PRISM)