PrefectHQ / prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
https://prefect.io
Apache License 2.0
16.1k stars 1.57k forks source link

push_project_to_gcs is not working with credentials stored in a prefect block #13059

Open uuazed opened 1 year ago

uuazed commented 1 year ago

I've created a new project using the docker-gcs template. The credentials of my GCP service account are stored in a prefect block. According to the prefect project tutorial, this is the recommended setup. This is the push section in my deployment.yaml :

push:

When initiating a deploy via prefect deploy --name ... , the step fails with AttributeError: 'str' object has no attribute 'keys'

Expectation / Proposal

Using credentials stored in a prefect blocks works. Even better, I can use a prefect gcs block directly.

Traceback / Example

uuazed commented 1 year ago

recreating the credentials block solved the issue :shrug:

desertaxle commented 1 year ago

An older version of the GcpCredentials block saved the service_account_json fields as JSON string instead of an object, which is the root cause of this error. Recreating the GcpCredentials block will use the new version and save the service_account_json field in with the correct type. To guard against service_account_json values that are saved as JSON strings, we should attempt to parse the service_account_json field value if it is a string.