To aid with creating db migrations, we have a rake task which scrapes the Legistar documentation and determines what the corresponding rails database schema types should be for a given endpoint.
This recently broke, apparently because of a change to the scraper system being used.
Here's what we get now (broken):
rake legistar_matters:structure
I, [2017-04-19T13:10:20.636123 #3877] INFO -- : Connection opened to http://webapi.legistar.com and logging to log/fetch-legistar.log
E, [2017-04-19T13:10:21.000020 #3877] ERROR -- : Failed fetching for query_url: http://webapi.legistar.com/Help/Api/GET-v1-Client-Matters, body: { "input": { "webpage/url": "http://webapi.legistar.com/Help/Api/GET-v1-Client-Matters"}}; status: 400, error:
So two very related issues, one being the broken scraper and the other being logging output that doesn't help us realize what the problem is caused by.
To aid with creating db migrations, we have a
rake
task which scrapes the Legistar documentation and determines what the corresponding rails database schema types should be for a given endpoint.This recently broke, apparently because of a change to the scraper system being used.
Here's what we get now (broken):
Here's what we expect - something like:
So two very related issues, one being the broken scraper and the other being logging output that doesn't help us realize what the problem is caused by.