search

Galileo-Galilei / kedro-pandera

A kedro plugin to use pandera in your kedro projects
https://kedro-pandera.readthedocs.io/en/latest/
Apache License 2.0
33 stars 4 forks source link

AttributeError due to missing `metadata` parameter in datasets #67

Closed michal-mmm closed 2 months ago

michal-mmm commented 2 months ago

Description

Some kedro-datasets do not have a metadata parameter. This causes kedro-pandera to throw an error, even if there is no schema validation for the affected dataset.

Context

The bug prevents me from using datasets without a metadata parameter, disrupting my data pipeline. Even some official kedro datasets are missing this parameter (e.g. ManagedTableDataset, EagerPolarsDataset)

Steps to Reproduce

  1. Use the spaceflights-pandas starter.
  2. Use a custom CSV dataset from kedro-datasets.
  3. Delete or comment out the metadata parameter from the custom dataset.

Expected Result

The custom dataset should work without throwing an error, even if it lacks a metadata parameter.

Actual Result

An AttributeError is thrown: AttributeError: 'CSVDataset' object has no attribute 'metadata'

Additional Information

Code causing the error: catalog._datasets[name].metadata is not None