move-coop / parsons

A python library of connectors for the progressive community.
Other
255 stars 125 forks source link

Issue 910 - Added timeout to BigQuery load_job.result() #981

Closed bear-jordan closed 1 month ago

bear-jordan commented 5 months ago

Overview

This PR addresses Issue #910. A timeout is added to prevent indefinite hanging.

From the original issue:

Per Google BQ query job documentation, it's possible to pass a default timeout parameter to the synchronous call that awaits a response.

This change would require choosing a reasonable maximum default (30s?), setting that up as a configurable parameter, and passing that parameter into load_job.result.

Changes

27/1/2024

31/1/2024 — @Jason94

bear-jordan commented 5 months ago

Hi, looked into why the Mac build is failing (see here). Github isn't reporting any outages — not too sure what is going on

Warning: Failed to download action 'https://api.github.com/repos/actions/cache/tarball/e12d46a63a90f2fae62d114769bbf2a179198b5c'. Error: Operation timed out (codeload.github.com:443)
Warning: Back off 26.77 seconds before retry.
Warning: Failed to download action 'https://api.github.com/repos/actions/cache/tarball/e12d46a63a90f2fae62d114769bbf2a179198b5c'. Error: nodename nor servname provided, or not known (codeload.github.com:443)
Warning: Back off 21.133 seconds before retry.
Error: Operation timed out (api.github.com:443)
Jason94 commented 5 months ago

Hi, looked into why the Mac build is failing (see here). Github isn't reporting any outages — not too sure what is going on

Warning: Failed to download action 'https://api.github.com/repos/actions/cache/tarball/e12d46a63a90f2fae62d114769bbf2a179198b5c'. Error: Operation timed out (codeload.github.com:443)
Warning: Back off 26.77 seconds before retry.
Warning: Failed to download action 'https://api.github.com/repos/actions/cache/tarball/e12d46a63a90f2fae62d114769bbf2a179198b5c'. Error: nodename nor servname provided, or not known (codeload.github.com:443)
Warning: Back off 21.133 seconds before retry.
Error: Operation timed out (api.github.com:443)

Sometimes this happens with these mac tests :'( I'm rerunning it and hopefully it'll pass.

bear-jordan commented 5 months ago

Ominous that it's a timeout error 😂

shaunagm commented 4 months ago

Looks like the tests are passing now! What's the status of this PR? Are we waiting on re-review, or on changes?

bear-jordan commented 4 months ago

I might have missed something, but I think all changes have been addressed.

shaunagm commented 4 months ago

Awesome! @Jason94 whenever you get a chance can you re-review?