coderxio / sagerx

Open drug data pipelines curated by pharmacists.
https://coderx.io/sagerx
Other
49 stars 13 forks source link

Max workers 3 #320

Closed jrlegrand closed 1 month ago

jrlegrand commented 1 month ago

Explanation

Sometimes when I run rxnorm_historical I get a SIGKILL error and the DAG runs forever, never stopping. https://github.com/coderxio/sagerx/issues/307

This doesn't fully resolve that issue, but it makes it better for my machine.

Also added a counter for logging every 1000 responses.

Rationale

I kept having to do this manually and at least one other person said this fixed the bug for them when they did it manually.

Tests

Ran RxNorm Historical.