We're getting a pickling error: Can't pickle local object 'if_exception_type.<locals>.if_exception_type_predicate' on each concurrent invocation of cli.query._backfill_query. e.g.. I haven't been able to reproduce this locally.
Multi-threading tends to require less pickling so here's an attempt to fix it.
As an aside, most of concurrent functionality is I/O bound (API requests to GCP) so using threads probably makes more sense anyway. 🤞
We're getting a pickling error:
Can't pickle local object 'if_exception_type.<locals>.if_exception_type_predicate'on each concurrent invocation ofcli.query._backfill_query. e.g.. I haven't been able to reproduce this locally.Multi-threading tends to require less pickling so here's an attempt to fix it.
As an aside, most of concurrent functionality is I/O bound (API requests to GCP) so using threads probably makes more sense anyway. 🤞
┆Issue is synchronized with this Jira Task