search

thegraphnetwork / EpiGraphHub

Software platform to Gather, transmform, harmonize and store epidemiological data for analytical purposes.
https://epigraphhub.org
GNU General Public License v3.0
8 stars 10 forks source link

chore(airflow): Copernicus API weather collection DAG for InfoDengue (Catchup) #162

Closed luabida closed 1 year ago

luabida commented 1 year ago

This DAG fetches data from Copernicus API monthly since 2000-01-01 towards.

Needs extra configuration because it will store the data at InfoDengue SQL server, I've been using a .env file at the same directory of the DAG, any thoughts about this approach? @xmnlab

xmnlab commented 1 year ago

The .env should be read by the container. So it should be in the main .env ... soon I will move each .env file to its own container folder

luabida commented 1 year ago

@xmnlab > So it should be in the main .env

After this, in order to pass on CI, I would have to expose AD's postgres config, is there any better solution to this?

xmnlab commented 1 year ago

sorry .. not sure if I am following you here, why do you need to expose that?

luabida commented 1 year ago

@xmnlab sorry for the delay, I had a exhausting docker package error (not sure if k8s deconfigured it somehow)

I'm referring about the github CI ENV variables for the workflow, I'll send you mine .env to exemplify