zenml-io / zenml

ZenML 🙏: The bridge between ML and Ops. https://zenml.io.
https://zenml.io
Apache License 2.0
3.93k stars 429 forks source link

[BUG] training_pipeline.view_anomalies() raises error in the "quickstart" example #61

Closed zyfzjsc988 closed 3 years ago

zyfzjsc988 commented 3 years ago

Describe the bug training_pipeline.view_anomalies() raises error in the "quickstart" example

To Reproduce I tried the "quickstart" example and anomalies function as well.

I just run training_pipeline.view_anomalies() after training_pipeline.run()

Expected behavior like screenshots

Screenshots I try to change source code will get right return (maybe) image

Stack Trace `

TypeError Traceback (most recent call last)

in ----> 1 training_pipeline.view_anomalies() /opt/conda/lib/python3.8/site-packages/zenml/pipelines/training_pipeline.py in view_anomalies(self, split_name) 338 schema_uri = self.get_artifacts_uri_by_component( 339 GDPComponent.SplitSchema.name)[0] --> 340 detect_anomalies(stats_uri, schema_uri, split_name) 341 342 def steps_completed(self) -> bool: /opt/conda/lib/python3.8/site-packages/zenml/utils/post_training/post_training_utils.py in detect_anomalies(stats_uri, schema_uri, split_name) 108 def detect_anomalies(stats_uri: Text, schema_uri: Text, split_name: Text): 109 schema = get_schema_proto(schema_uri) --> 110 stats = get_statistics_dataset_dict(stats_uri) 111 if split_name not in stats: 112 raise Exception(f'{split_name} split not present!') /opt/conda/lib/python3.8/site-packages/zenml/utils/post_training/post_training_utils.py in get_statistics_dataset_dict(stats_uri) 69 """Get DatasetFeatureStatisticsList from stats URI""" 70 result = {} ---> 71 for split in os.listdir(stats_uri): 72 stats_path = os.path.join(stats_uri, split, 'stats_tfrecord') 73 serialized_stats = next( TypeError: listdir: path should be string, bytes, os.PathLike, integer or None, not list ` ** Context (please complete the following information):** - OS: Ubuntu 18.04 - Python Version: 3.8.2 - ZenML Version: 0.3.5 **Additional information** Add any other context about the problem here.
htahir1 commented 3 years ago

@zyfzjsc988 You are right, indeed the answer is as you described. PR #62 fixes this, and will be included in the next release. Please override the view_anomalies() method in the meanwhile as you suggest