wbuchanan / StataJSON

JSON IO operations for Stata using the Jackson Java Library
http://wbuchanan.github.io/StataJSON
21 stars 6 forks source link

Stata crashes after running “ggeocode” #21

Closed yichenx closed 7 years ago

yichenx commented 7 years ago

I'm trying to geocode some addresses and was using the jsonio package. It didn't work very well-stopped working when the address cannot be found. So I split all my addresses into files with only one address and try to geocode it using "ggeocode" command. DO.txt

It worked for the first several files but later on stata refuses to take any ggeocode command. My query java result also turned into this:

`query java

  Java system information
  Java initialized     no
Advanced Java settings
  set java_heapinit    63999m
  set java_heapmax     64000m
  set java_vmpath      C:\Program Files\Java\jre1.8.0_131\bin\server\jvm.dll`
wbuchanan commented 7 years ago

@yichenx Just looking at the file that you attached my guess is that you are hitting a rate limit on the API. You can also simplify the code quite a bit by letting the program handle iterating over the records in the file for you. It will execute much faster without all of the file reading/loading as well. What is the last record that returns the geocoding info? If it is 2,500 after you've not queried the API for 24 hours then it is definitely a rate limit.

local   maxid1  =   468
local    maxid2    =    1176
forvalues x = 1/2{
    clear all
    di `x'
    use temp\geocode`x'.dta, clear
    ggeocode address, ret(viewport address bbox)
    save temp\geocode`x'.dta, replace
}
yichenx commented 7 years ago

I am not quite sure how many records I have in the past 24 hours since I ran part of the code several times. But it should be around 2,500. Now my stata also crashes when I'm running code like import delimited for .csv files-the program quit without giving any output. Not sure if the Java initialized no is part of the reason? I'm using Stata 14 on my windows 10 computer. Thank you!

wbuchanan commented 7 years ago

From the info on the StataList post, you might be having issues due to the java_heapinit parameter being set so high.

Try:

set java_heapinit 4096, perm

To reduce the amount of memory allocated to the JVM when it spins up. That might help a bit.

yichenx commented 7 years ago

I set it to 4096 now. But it still shows Java initialized no. It seems like ggeocode stops working whenever there's an error in the address list. It will leave missing value for the remaining records in the files. For example, "2952 Saunders Settlement Road, Cambria township, NY 14132" is the first address but Google cannot find this address, ggeocode will stop here and put missing value in the coordinates for the rest of the list. That's why I kept only one address for each file. Thank you!

yichenx commented 7 years ago

I tried reinstall my stata, java and 14.1 updates. All problems solved. Have tried all these last night but didn't help. I guess your suggestion of set java_heapinit 4096, perm helped me out. Thanks a lot!

wbuchanan commented 7 years ago

@yichenx If you want to, submit a different ticket with the issue you mentioned above regarding the missing records and I'll check into that. I'll close this issue since it sounds like you have this one solved.