Cannot write model card when pipeline root is in GCS - Githubissues

tensorflow / tfx-addons

Developers helping developers. TFX-Addons is a collection of community projects to build new components, examples, libraries, and tools for TFX. The projects are organized under the auspices of the special interest group, SIG TFX-Addons. Join the group at http://goo.gle/tfx-addons-group

Apache License 2.0

125 stars 64 forks source link

Cannot write model card when pipeline root is in GCS #261

Closed jeongukjae closed 1 year ago

jeongukjae commented 1 year ago

Expected Behavior

When setting pipeline_root to the Google Cloud Storage path, I couldn't write the model card properly.

Actual Behavior

Model Card Toolkit cannot write model card files to remote storage.

Code Flow:

ModelCardToolkit.export_format: https://github.com/tensorflow/model-card-toolkit/blob/74d7e6d8d3163b830711b226491ccd976a2d7018/model_card_toolkit/core.py#L363-L419
ModelCard.render: https://github.com/tensorflow/model-card-toolkit/blob/74d7e6d8d3163b830711b226491ccd976a2d7018/model_card_toolkit/model_card.py#L548-L580
template_utils.render: https://github.com/tensorflow/model-card-toolkit/blob/74d7e6d8d3163b830711b226491ccd976a2d7018/model_card_toolkit/utils/template_utils.py#L43-L72
io_utils.write_file: https://github.com/tensorflow/model-card-toolkit/blob/74d7e6d8d3163b830711b226491ccd976a2d7018/model_card_toolkit/utils/io_utils.py#L29-L44
```
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, mode) as f:
  f.write(content)
```

Steps to Reproduce the Problem

1. 1. 1.

Specifications

Version: model-card-toolkit==2.0.0, tfx-addons 0.6.0
Platform: vertex ai

jeongukjae commented 1 year ago

https://github.com/tensorflow/tfx-addons/blob/b2151e53b30be11774bf72a875002ec9ef1103b4/tfx_addons/model_card_generator/executor.py#L137-L138

I think this line should be changed. First, save the model card files to local and copy them to gcs