Open bumegha opened 2 years ago
atee12
commented
2 years ago i do have similar erro as well:
ERROR:apache_beam.runners.direct.executor:Giving up after 4 attempts.
WARNING:apache_beam.runners.direct.executor:A task failed with exception: tzinfo
Traceback (most recent call last):
File "/home/g12_sean_aye/code_sub_utility.py", line 132, in
atee12
commented
2 years ago Manage to resolve this by upgrading the google cloud core, and assigning the correct access role. Follow some of the Pub/Sub toutorial to eaable the APIs and
I was running the .py code from gcp cloud console.
Name: google-cloud-core Version: 2.3.2
I have set following roles in the service account and the key from that service acc is set as enviroment variable in the gcp cloud console. BigQuery Admin Dataflow Admin Pub/Sub Admin Storage Object Admin
Hope this can help to resolve! Good Luck!
ERROR:apache_beam.runners.direct.executor:Giving up after 4 attempts. WARNING:apache_beam.runners.direct.executor:A task failed with exception: tzinfo WARNING:apache_beam.runners.direct.executor:A task failed with exception: tzinfo Traceback (most recent call last): File "/home/gcpuser/data-science-on-gcp/04_streaming/realtime/./avg01.py", line 82, in
run(project=args['project'], bucket=args['bucket'], region=args['region'])
File "/home/gcpuser/data-science-on-gcp/04_streaming/realtime/./avg01.py", line 61, in run
(all_events
File "/home/gcpuser/.local/lib/python3.9/site-packages/apache_beam/pipeline.py", line 598, in exit
self.result.wait_until_finish()
File "/home/gcpuser/.local/lib/python3.9/site-packages/apache_beam/runners/direct/direct_runner.py", line 588, in wait_until_finish
self._executor.await_completion()
File "/home/gcpuser/.local/lib/python3.9/site-packages/apache_beam/runners/direct/executor.py", line 432, in await_completion
self._executor.await_completion()
File "/home/gcpuser/.local/lib/python3.9/site-packages/apache_beam/runners/direct/executor.py", line 480, in await_completion
raise update.exception
File "/home/gcpuser/.local/lib/python3.9/site-packages/apache_beam/runners/direct/executor.py", line 370, in call
self.attempt_call(
File "/home/gcpuser/.local/lib/python3.9/site-packages/apache_beam/runners/direct/executor.py", line 416, in attempt_call
result = evaluator.finish_bundle()
File "/home/gcpuser/.local/lib/python3.9/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 704, in finish_bundle
data = self._read_from_pubsub(self.source.timestamp_attribute)
File "/home/gcpuser/.local/lib/python3.9/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 693, in _read_from_pubsub
results = [_get_element(rm.message) for rm in response.received_messages]
File "/home/gcpuser/.local/lib/python3.9/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 693, in
results = [_get_element(rm.message) for rm in response.received_messages]
File "/home/gcpuser/.local/lib/python3.9/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 681, in _get_element
timestamp = Timestamp.from_utc_datetime(message.publish_time)
File "/home/gcpuser/.local/lib/python3.9/site-packages/apache_beam/utils/timestamp.py", line 106, in from_utc_datetime
if dt.tzinfo != pytz.utc and dt.tzinfo != datetime.timezone.utc:
AttributeError: tzinfo