WeberLab-UW / 2022-Election-Material

Fork of @BlakeRMills repo for Web scraping, candidate data, and election visualizations - to modify for governors and mayors races 🤞
0 stars 0 forks source link

Tracking data via Blacklight #3

Open nniiicc opened 1 year ago

nniiicc commented 1 year ago

Given a list of URLs from political candidates we want to

There is a command line tool we can use to develop reports for political candidates

peiwenf commented 1 year ago

@nniiicc Here is the script that I found in the blacklight-collector repository that might doing the counting work https://github.com/the-markup/blacklight-collector/blob/main/src/parser.ts#L45 And I believe here is where they fit in data https://github.com/the-markup/blacklight-collector/blob/383eb2fbafa5e17585c2f0b1e6976cade91e3ecf/__tests__/parser.ts#L15

Here is the JSON file the black light collector produce https://github.com/peiwenf/campaign-access-eval/blob/main/inspection.json And this is the result I got for the same link on their website, which is surprisingly different. My guess is that the website only scans the base page while the json file contains three subpages. https://themarkup.org/blacklight?url=reg.recreation.uw.edu

Here is the script that Tor Browser Friendliness project calculating their score https://github.com/askumar2903/tor_browser_friendliness/blob/main/src/score-calculation.ts

And here is the link to the drop box folder I created https://www.dropbox.com/scl/fo/4619d5qiknngk0njjgbgj/h?dl=0&rlkey=g3s6xbmu2i6nl3b7r5fb5kvgu

peiwenf commented 1 year ago

@nniiicc Here is the black light collector reports generated by the Github Action, which are three examples from the attorney general data frame

And here is a description for the JSON file

I believe we are most interested in the inspection.json

Questions about the next step: