lucasmation / microdadosBrasil

Reads most common Brazilian public microdata (CENSO, PNAD, etc) easy and fast
165 stars 59 forks source link

Run tests for all datasets #104

Closed nicolassoarespinto closed 7 years ago

nicolassoarespinto commented 8 years ago

I'm running test for all datasets. Will use this issue to store all problems that I found.

nicolassoarespinto commented 8 years ago

PNAD

Import

IBGE seems to have updated files from 2014, this changed some names and so the test failed for 2014. Download works.

 period time_domicilios size_domicilios error_domicilios time_pessoas size_pessoas error_pessoas
1    2001      0.03127003            0 Mb            FALSE   0.06249809       0.2 Mb         FALSE
2    2002      0.04688501            0 Mb            FALSE   0.03125501       0.2 Mb         FALSE
3    2003      0.03002095            0 Mb            FALSE   0.05228090       0.2 Mb         FALSE
4    2004      0.07812500          0.1 Mb            FALSE   0.06251597       0.2 Mb         FALSE
5    2005      0.03125596            0 Mb            FALSE   0.05674815       0.2 Mb         FALSE
6    2006      0.03125310            0 Mb            FALSE   0.05276799       0.2 Mb         FALSE
7    2007      0.03124905            0 Mb            FALSE   0.04687595       0.2 Mb         FALSE
8    2008      0.03125405            0 Mb            FALSE   0.10073495       0.4 Mb         FALSE
9    2009      0.03125000          0.1 Mb            FALSE   0.07123804       0.2 Mb         FALSE
10   2011      0.06251621            0 Mb            FALSE   0.04687595       0.2 Mb         FALSE
11   2012      0.04689097            0 Mb            FALSE   0.05407095       0.2 Mb         FALSE
12   2013      0.03125095          0.1 Mb            FALSE   0.04686284       0.2 Mb         FALSE
13   2014      0.01562595            0 Mb             TRUE   0.01562500         0 Mb          TRUE
> 

Note: Test used samples.

Download

> test_PNAD_download
   period time_download error_download
1    2001      211.1185          FALSE
2    2002      140.9034          FALSE
3    2003      268.4109          FALSE
4    2004      249.3289          FALSE
5    2005      209.9573          FALSE
6    2006      159.6092          FALSE
7    2007      154.6441          FALSE
8    2008      225.0543          FALSE
9    2009      167.4503          FALSE
10   2011      119.8831          FALSE
11   2012      137.4362          FALSE
12   2013      150.5334          FALSE
13   2014      182.4195          FALSE
lucasmation commented 8 years ago

that is expected: they recently launched the "Suplemento da PNAD 2014"

nicolassoarespinto commented 7 years ago

New workflow, now all tests are stored at:

https://github.com/lucasmation/microdadosBrasil/blob/master/vignettes/test_report_download.md

https://github.com/lucasmation/microdadosBrasil/blob/master/vignettes/test_report_import_sample.md