search

sfbrigade / sf-lifeline

GNU Affero General Public License v3.0
7 stars 4 forks source link

Create a web-scraper API library for checking EMT/Paramedic licenses #21

Closed francisli closed 9 months ago

francisli commented 10 months ago

https://emsverification.emsa.ca.gov/Verification/Search.aspx

francisli commented 10 months ago

I think the use case for us is having first responders enter their license number during sign up, then we hit the endpoint to gather data and store it for verification. If we can scrape name, license type, status and return it as a JSON object in our scraper library, we can then store it in the user record for display to the staff who are verifying sign-ups.

samau3 commented 10 months ago

@francisli
I'm looking at using Puppeteer for the web scraping, but it would require modifications to our Docker files to allow for a headless browser. Should I continue with using Puppeteer or should I try a different option?

francisli commented 10 months ago

@francisli I'm looking at using Puppeteer for the web scraping, but it would require modifications to our Docker files to allow for a headless browser. Should I continue with using Puppeteer or should I try a different option?

I'm not necessarily against adding that to our project, but it does seem a bit heavyweight for this particular ask... I'm wondering if just using fetch (now included with Node) to make the POST request to the page and then maybe some regexps to parse what we want from the HTML would suffice...?