search

UDST / synthpop

Synthetic populations from census data
BSD 3-Clause "New" or "Revised" License
99 stars 47 forks source link

add function that repeats census query till it gets a valid answer #56

Closed msoltadeo closed 4 years ago

msoltadeo commented 4 years ago

Add a function that queries the census api till it returns a valid answer

smmaurer commented 4 years ago

I approve, but there might be another approach that would be a little safer --

It looks like the census library has built-in support for retrying after transient errors, although i don't know if it actually works, or would solve the specific problems you're having.

In census_helpers.py#L11, you could try changing census.Census(key) to census.Census(key, retries=10), or some other number, and see if it helps. This would avoid the problem of an infinite loop if there's an unanticipated problem with a query.

(I found this from looking through the functions here, which I think is what we're using for the census connection.)

jessicacamacho commented 4 years ago

Yes, agree with @smmaurer! I did this as a temporary solution in my code and it worked for me, but was planning to improve it because of this risk of something else failing. I didn't want to modify the census library to add retrials often used in get or post requests, but if we can pass the retries directly as an argument to the census library, that would be ideal!

smmaurer commented 4 years ago

Closing this since it's been superseded by PR #57.