Closed cuducos closed 6 years ago
BTW:
FIx #165
And maybe it's useful to test the fetch_latest_backup
function too:
from serenata_toolbox.datasets import fetch_latest_backup
fetch_latest_backup('data/')
That way we test if these new datasets are being downloaded by default in a default Serenata installation ; )
:tada:
What I did to test this PR:
Cloned the project:
$ git clone git@github.com:datasciencebr/serenata-toolbox.git
Change to its folder:
$ cd serenata-toolbox
Change to @cuducosβ branch:
$ git fetch origin
$ git checkout -b cuducos-donation-data origin/cuducos-donation-data
$ git merge master
Run the python fetch
script:
>>> from serenata_toolbox.datasets import fetch, fetch_latest_backup
>>> files = (
'2017-11-30-donations-candidates.xz',
'2017-11-30-donations-committees.xz',
'2017-11-30-donations-parties.xz'
)
>>> for filename in files:
fetch(filename, 'data/')
The result:
Downloading 2017-11-30-donations-candidates.xz: 100%|β| 239M/239M [02:34<00:00, 1.54Mb/s]
Downloading 2017-11-30-donations-committees.xz: 100%|βββββββββββββββββββββββββββββββββββββββββββββββββ| 5.64M/5.64M [00:03<00:00, 1.69Mb/s]
Downloading 2017-11-30-donations-parties.xz: 100%|ββββββββββββββββββββββββββββββββββββββββββββββββββββ| 6.47M/6.47M [00:03<00:00, 1.72Mb/s]
And for fetch_latest_backup
script:
>>> from serenata_toolbox.datasets import fetch_latest_backup
>>> fetch_latest_backup('data/')
Good! π
What is the purpose of this Pull Request? Add the electoral campaign donation datasets to the toolbox downloader.
What was done to achieve this purpose? Outside the repo I uploaded the
.xz
files to S3 and here I added the files to theLATEST
constant.How to test if it really works?
And check if the filer were downloaded successfully ; )
Who can help reviewing it? @anaschwendler