search

datazoompuc / datazoom_social_Stata

Data Zoom provides free packages in Stata to access microdata from IBGE's household surveys. Data Zoom provides tools that help researchers standardize surveys across time, build panel databases and deflate nominal variables. Para descrição em Português, acesse: http://www.econ.puc-rio.br/datazoom/index.html
22 stars 9 forks source link

Unable to use Census function for 2000 data #119

Open ts724 opened 3 months ago

ts724 commented 3 months ago

End up with error right away saying: "v0103 not found"

IgorRigolon commented 3 months ago

Hi, this error has been reported before on #86. I ran the function on the data I already had on my computer and it worked fine.

I think the error is caused by the data not being read properly, and then the v0103 variable just doesn't exist. Could you share the code you are using or which states you're loading?

You should have all Domxx.txt, Pesxx.txt, or FAMIxx.txt files saved in the same folder for the function to read them properly, they can't be stored in folders inside that folder.

ts724 commented 3 months ago

Thanks for your response. 2010 worked for me without any issues. Here is the code I am using for 2000:

datazoom_censo, years( 2000 ) ufs( RO AC AM RR PA AP TO MA PI CE RN PB PE AL SE BA MG ES RJ SP PR SC RS MS MT GO DF ) original(/home/ecco_rais/data/interwrk/ts724/Data/Census 2000/Raw) saving(/home/ecco_rais/data/interwrk/ts724/Data/Census 2000/Datazoom) all english

I have tried to reduce to a smaller number of states and had the same issue. The previous issue says that the issue is with paths but I used essentially the same paths for the 2010 data and it worked fine. And to confirm, yes I have all the files saved in the same folder and not within other folders.

IgorRigolon commented 3 months ago

Hi, I still couldn't reproduce the error. First I though it was because of the space in the path (the folder /Census 2000/), but I tried adding a space here and it still works. I've uploaded the data I'm using to a Google Drive folder. Please check if your data matches it.

If it still doesn't work, please follow the instructions at the bottom of this page to produce a log file that can show what exactly is causing the error.

ts724 commented 3 months ago

Hi Igor, thanks for your help again. I downloaded your files, they are the same except for the fact that you have five fewer Pes files than I did. Upon running it again, I have the same error as before with your files. I am attaching the log you requested below, thanks! datazoom_error.txt

IgorRigolon commented 2 months ago

Hi, thank you for sharing the logs and sorry for the delay, this is quite a tricky problem to debug. I did some testing and from the log, it seems that the problem is arising during

quietly cap infile using /tmp/St2735106.000003, using("/home/ecco_rais/data/interwrk/ts724/Data/Census_2000/Censo 2000/pes11.txt") clear

where the tempfile is a dictionary and the pes11.txt is the actual data. It can't read this file properly, which leads to problems down the road (because the quietly capture suppresses it for the moment).

If you may, I suggest you download this dictionary file. Then you can directly run

infile using path/to/censo2000pes_en.dct, using("/home/ecco_rais/data/interwrk/ts724/Data/Census_2000/Censo 2000/pes11.txt") clear

This should at least reveal what the actual error is, could be something wrong with the dictionary of with the .txt file. I tried downloading all the Census data from the IBGE website onto another PC and couldn't replicate the error.