tensorflow / tfx

TFX is an end-to-end platform for deploying production ML pipelines
https://tensorflow.github.io/tfx/
Apache License 2.0
2.11k stars 707 forks source link

Documentation for custom artifact in Vertex AI #6102

Open jeongukjae opened 1 year ago

jeongukjae commented 1 year ago

URL(s) with the issue:

https://www.tensorflow.org/tfx/api_docs/python/tfx/v1/dsl/Artifact

Description of issue (what needs changing):

Clear description:

For now, custom artifact should satisfy the schema title regex (^[a-z][a-z0-9-_]{2,20}[.][A-Z][a-zA-Z0-9-_]{2,49}$).

https://github.com/tensorflow/tfx/blob/2fd4f95ce3e631e7763611fd8ed631d59bc053d8/tfx/orchestration/kubeflow/v2/compiler_utils.py#L302-L335

And custom artifact should be accessible with its title (top-level module) to be resolved in KubeFlowV2's container entry point.

https://github.com/tensorflow/tfx/blob/2fd4f95ce3e631e7763611fd8ed631d59bc053d8/tfx/orchestration/kubeflow/v2/container/kubeflow_v2_entrypoint_utils.py#L219-L234

But this information does not documented in the Artifact class. So it will be helpful for the developers who want to extend TFX with Vertex AI.

singhniraj08 commented 1 year ago

@jeongukjae,

Thank you bring this up for enhancing our documentation. Let me take this internally and update this thread with updates. Thank you.