Update Vertex orchestrator to allow for custom disk size and type

strickvl commented 8 months ago

Open Source Contributors Welcomed!

Please comment below if you would like to work on this issue!

Contact Details [Optional]

support@zenml.io

What happened?

Currently, the Vertex orchestrator in ZenML does not provide options to configure custom disk size and type. This limitation restricts users from optimizing their cloud resources according to specific needs, such as handling large datasets or requiring faster disk speeds.

Task Description

Update the Vertex orchestrator's configuration/settings in ZenML to include options for specifying custom disk size and type. This enhancement will allow users more flexibility and control over their cloud resources, leading to better performance and cost optimization.

Expected Outcome

The Vertex orchestrator in ZenML should allow users to specify custom disk sizes and types as part of its configuration.
Users should be able to configure these options easily and have them applied correctly to their Vertex AI environments.
The documentation should be updated to guide users on how to use these new configuration options.

Steps to Implement

Review and update the Vertex orchestrator's codebase to include settings for disk size and type.
Ensure that these settings are correctly applied when deploying pipelines on Vertex AI.
Test the implementation with various disk sizes and types to ensure compatibility and correct behavior.
Update the ZenML documentation to include these new settings and provide usage examples.

Additional Context

Allowing for custom disk size and type configurations aligns with ZenML's philosophy of providing flexible and scalable MLOps solutions. This update will cater to a broader range of use cases and performance requirements.

Code of Conduct

[ ] I agree to follow this project's Code of Conduct

npv12 commented 8 months ago

I'll like to help around here. Could you guide me around on how to get started?

strickvl commented 8 months ago

Hi @npv12. I'd suggest you read the CONTRIBUTING.md guide which has general instructions. Then if you're unfamiliar with ZenML, the top of the docs will be useful and then of course you'll need to have and use the Vertex orchestrator, documented here. Also let us know if any of the description doesn't make sense.

AryaMoghaddam commented 5 months ago

I can take this issue if it's not assigned, or completed

strickvl commented 5 months ago

@AryaMoghaddam let us know if you have any questions along the way!

zenml-io / zenml