HERE pull `token_expired` error

CityofToronto / bdit_data-sources

Data sources used by the Big Data Innovation Team

https://github.com/orgs/CityofToronto/teams/bigdatainnovationteam

GNU General Public License v3.0

38 stars 8 forks source link

HERE pull `token_expired` error #1073

Open gabrielwol opened 2 weeks ago

gabrielwol commented 2 weeks ago

When pull_here and pull_here_path task get_download_link runs for more than an hour (example: DAG run 2024-09-13, 17:00:00 EDT), the token for the pull expires and subsequent retries on this task fail immediately with token_expired error. We could fail immediately in this case or retry from beginning of DAG.

radumas commented 2 weeks ago

Duplicate of https://github.com/CityofToronto/bdit_data-sources/issues/204

I should have added more info in that issue :|