search

lesliePhD / open_funders_canada

We are creating an open tool to help find out who is funding who, for what, and when over time. This will help both funders and nonprofits do their work more effectively.
MIT License
1 stars 0 forks source link

Potential Contributors can message me here #6

Open lesliePhD opened 7 years ago

TyJK commented 7 years ago

Hey Leslie, I'm doing some research on scraping myself using the Scrapy library. I was curious if you're still using import.io, how you're finding it and if you had a predefined list of sites you wanted to scrape? I think the Sprint will be something of a crash course in scraping for me, and being broke and (currently) unfunded, I need to make it myself. We should keep each other updated on what we've learned and see if that makes life easier for the both of us :)

lesliePhD commented 7 years ago

@TyJK Hi Tyler! Hope you are well! I would be using import.io as we paid for the subscription and its much easier given my 5% skillset in python. Import.io is like the drag-and-drop of scraping. Some other ppl suggested scrapy or beautiful soup, but then my boss told me we already had a subscription to import.io.

I didn't have a set of websites I want to scrape - I was going to wait for some contributors and see if anyone wanted to do a set of funders in a city or on a specific topic (ie. indigenous affairs, or the environment for example). We are pretty open, but the most useful to us is something that will allow us to get some insights into funding trends. I put in some more info here: https://github.com/lesliePhD/open_funders_canada/issues/4. You can use whatever tool you like as described there.