smart-on-fhir / cumulus-etl

Extract FHIR data, Transform with NLP and DEID tools, and then Load FHIR data into a SQL Database for analysis
https://docs.smarthealthit.org/cumulus/etl
Apache License 2.0
11 stars 2 forks source link

Support HTTP proxies for web requests #135

Closed dogversioning closed 4 months ago

dogversioning commented 1 year ago

Pretty standard territory on this one - https://www.scrapingbee.com/blog/python-requests-proxy/ , among other resources explaining the same - we just need to decide how we want to inject it (perhaps a .ENV file).

mikix commented 1 year ago

I thought this stuff was baked into python libraries (with code overrides allowed). Like, that blog post suggests you can just set:

export HTTP_PROXY='http://10.10.10.10:8000'
export HTTPS_PROXY='http://10.10.10.10:1212'

I'd hope that would work as-is...? If not, yeah let's fix that.

mikix commented 1 year ago

Oh ah -- you are thinking about how to drive the compose file. This is a docker-setup ticket, yeah?

dogversioning commented 1 year ago

Yeah - there's basically two best practices ways to do it in docker:

kangnari commented 1 year ago

Does the cTAKES container also require any outbound access?

mikix commented 1 year ago

Does the cTAKES container also require any outbound access?

I believe so yes, because it downloads the UMLS database.

dogversioning commented 1 year ago

yeah, that might be a bit trickier to wrangle.