SPIKE: programmatically reach the top 60 banks' sites - Githubissues

CorrelAidxNL / BankTrack

Collaboration between BankTrack and CorrelAid Netherlands

GNU General Public License v3.0

0 stars 0 forks source link

SPIKE: programmatically reach the top 60 banks' sites #8

Open andrewsutjahjo opened 2 years ago

andrewsutjahjo commented 2 years ago

We've seen that some websites contain javascript. This spike is to find the programmatic reachability of the top 60 banks' websites.

Spike is done when we can say something about how well we can reliably get website data from the top 60 banks. Probably want to check

[x] Get starting links for top 60 banks' csr pages (or equivalent
[x] get some kind of metric for parsability of a site using an automatic tool (can_find_hits_for_href_regex_in_output?)
[x] try different scraping tools to get parsable data from links of 60 banks
[x] simple top line write up in wiki