no10ds / rapid

Project rAPId
https://rapid.readthedocs.io/en/latest/
MIT License
23 stars 6 forks source link

Schema creation fails #59

Closed JOrjales-CDDO closed 11 months ago

JOrjales-CDDO commented 11 months ago

I may have misunderstood how to use the patterns, but on attempting to use the suggested pattern to create a dataset gives the following error

rapid = Rapid()

raw_data = [{"a": 1, "b": 2, "c": 3}, {"a": 10, "b": 20, "c": 30}]
df = pd.DataFrame(raw_data)

metadata = SchemaMetadata(
    layer="default",
    domain="test",
    dataset="test_jose",
    owners=[Owner(name="Jose", email="Jose.Orjales@digital.cabinet-office.gov.uk")],
    sensitivity=SensitivityLevel.PRIVATE.value,
)

try:
    dataset.upload_and_create_dataset(
        rapid=rapid,
        df=df,
        metadata=metadata,
        upgrade_schema_on_fail=False,
    )
except ColumnNotDifferentException:
    print("Columns not different.")

Gives the following error

Screenshot 2023-11-06 at 17 37 52

I've tried using:

rapid = Rapid()

raw_data = [{"a": 1, "b": 2, "c": 3}, {"a": 10, "b": 20, "c": 30}]
df = pd.DataFrame(raw_data)

schema = rapid.generate_schema(
    df=df,     
    layer="default",
    domain="test",
    dataset="test_jose", sensitivity="PUBLIC",
)

print("Domain ", schema.metadata.domain)
print("Columns ", schema.columns)

rapid.create_schema(schema=schema)

but that gives an error

Screenshot 2023-11-06 at 17 40 12

It's not clear to me based on generate_schema where I should specify the default owner.