Unable to specify S3 endpoint_url for storage in Prefect v2.10.5

[todayisgood]

First check

• [X] I added a descriptive title to this issue.
• [X] I used the GitHub search to find a similar request and didn't find it.
• [X] I searched the Prefect documentation for this feature.

Prefect Version

2.x

Describe the current behavior

Hi Prefect team,

I'm currently using Prefect version 2.10.5 to build some services, and have encountered an issue with Prefect not allowing me to specify the endpoint_url for the S3 storage backend.

It seems that this is preventing me from properly storing Flow code in an S3 bucket for deployment. I use a custom S3-compatible storage solution that requires a specific URL.

Can you please add support for specifying a custom S3 endpoint_url when using the S3 Block?

Thank you in advance for your time and assistance.

Best regards,

Describe the proposed behavior

support for specifying a custom S3 endpoint_url when creating the S3 Block

Example Use

No response

Additional context

No response

[desertaxle]

Thanks for the issue @todayisgood!

If you need to use a custom endpoint URL for S3-compatible storage use can use the S3Bucket from prefect-aws. The S3Bucket block can be configured with a AwsCredentials block which has the option to specific a custom endpoint URL.

Let me know if the S3Bucket block works for your use case, or if there's additional configuration that's missing there as well.

[todayisgood]

Thanks for the issue @todayisgood!

If you need to use a custom endpoint URL for S3-compatible storage use can use the S3Bucket from prefect-aws. The S3Bucket block can be configured with a AwsCredentials block which has the option to specific a custom endpoint URL.

Let me know if the S3Bucket block works for your use case, or if there's additional configuration that's missing there as well.

Thanks Reply @desertaxle

I want to publish the code to remote storage through prefect deployment, but the prefect deployment build -sb parameter does not support prefect-aws

[desertaxle]

The S3Bucket block can be used as a storage block with a deployment because it implements the WritableDeploymentStorage interface, so I would recommend trying it and seeing if it satisfies your use case!

[discdiver]

Added PR to clarify the help description.

[todayisgood]

@desertaxle However, S3Bucket cannot specify an endpoint_url.

I think there are two possible solutions to consider. Either the prefect deployment build -sb supports specifying AwsCredentials block type , or the S3Bucket supports specifying an endpoint_url.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. To keep this issue open remove stale label or comment.

[github-actions[bot]]

This issue was closed because it has been stale for 14 days with no activity. If this issue is important or you have more to add feel free to re-open it.

[todayisgood]

您好！来信已收到。祝您生活愉快～