Closed daaronr closed 4 years ago
That sounds like too long - I think you should be able to filter the fundraisers by date in just_giving_data_pull.R
Can;t remember exactly but the fundraiser_search_data_2018 code block could be filtered even more and then used in the subsequent code .
issue is no longer relevant but I'm about to post a related one.
I ran 'Just_giving_data_pull.R' in the Github 'sponsorship_design_analysis' repo. It now seems to work fine, capturing the fundraisers for correctly-identified charities (see below; especially CARE, Oxfam, WaterAid, Unicef). [Note that I've edited it to capture only those charities that match to the 'justgiving_id' I have checked separately.]
However, the runtime is a lot longer than I recall. It seemed to require 8 hours or more (not sure how to track this, but I had to leave it overnight), taking a particularly long time to process donation_data. If it takes this long each time, it may make our experiment more work to run (we have to set it running every night). I was wondering if the program could be set up to only download fundraisers from after our start date. Perhaps this should be a different version of the program, or we should have this as an argument ('earliest date'). For the purposes of scoping/power analysis it's good we are getting everything, but for the actual day-to-day running of the experiment it would be better to have a faster runtime.
Thoughts? Shall I go ahead and try to program this?