open-austin / azure-indigent-defense

4 stars 2 forks source link

Http trigger scraper function #8

Closed zimzoom closed 1 year ago

zimzoom commented 1 year ago

This adds an HTTP trigger version of scraper function. Also moves the write_to_blob function to the shared module. Although its function.json is allowing both GET and POST requests, the scraper functionality expects a POST request with valid json in the body.

See README for instructions on how to call the function. Nutshell version: After running func start, run curl -XPOST http://localhost:7071/api/http-scraper -d @path/to/payload.json (or whatever localhost port it gives you), where payload.json contains something like this:

{
    "start_date": "2022-11-11",
    "end_date": "2022-11-13",
    "county": "hays",
    "judicial_officers": ["Updegrove, Robert"]
}
zimzoom commented 1 year ago

If y'all would prefer to make this a GET that expects URL parameters ala /api/scraper?start_date=blabla&county=blabla etc., that would be very easy to implement because I already wrote it, and then changed my mind because I thought the json method would be more readable and flexible. We could even have it accept both.