✨ catalog: add ds.tables to get list with all tables

[lucasrodes]

When working with a dataset with multiple tables, we sometimes need to load all of them. This is specially true for Grapher steps when we just want to push propagate all tables coming from garden to Grapher.

The idea would be that, instead of doing:

# Load dataset
ds_garden = paths.load_dataset("leading_causes_deaths")

# Read all tables
all_tb = [ds_garden[tb_name for tb_name in ds_garden.table_names]

# Create a new grapher dataset with the same metadata as the garden dataset.
ds_grapher = create_dataset(
    dest_dir,
    tables=all_tb,
    check_variables_metadata=True,
    default_metadata=ds_garden.metadata,
)

one can simply write

# Load dataset
ds_garden = paths.load_dataset("leading_causes_deaths")

# Create a new grapher dataset with the same metadata as the garden dataset.
ds_grapher = create_dataset(
    dest_dir,
    tables=ds_garden.tables,
    check_variables_metadata=True,
    default_metadata=ds_garden.metadata,
)

[owidbot]

Quick links (staging server): Site	Admin	Wizard

Login: ssh owid@staging-site-ds-attribute-tables

chart-diff: ✅

No charts for review.

Edited: 2024-06-11 10:35:07 UTC Execution time: 3.61 seconds

[lucasrodes]

Unsure where the unit-test errors are coming from. It looks as if method named tables is not allowed?

ERROR tests/apps/wizard/utils/test_env.py - AttributeError: 'function' object has no attribute 'Table'
ERROR tests/backport/datasync/test_data_metadata.py - AttributeError: 'function' object has no attribute 'Table'
ERROR tests/data_helpers/test_geo.py - AttributeError: 'function' object has no attribute 'Table'
ERROR tests/test_command.py
ERROR tests/test_converters.py - AttributeError: 'function' object has no attribute 'Table'
ERROR tests/test_datadiff.py - AttributeError: 'function' object has no attribute 'Table'
ERROR tests/test_etl.py
ERROR tests/test_grapher_helpers.py - AttributeError: 'function' object has no attribute 'Table'
ERROR tests/test_grapher_model.py - AttributeError: 'function' object has no attribute 'Table'
ERROR tests/test_helpers.py - AttributeError: 'function' object has no attribute 'Table'
ERROR tests/test_metadata_schemas.py - AttributeError: 'function' object has no attribute 'Table'
ERROR tests/test_prune.py - AttributeError: 'function' object has no attribute 'Table'
ERROR tests/test_snapshot.py - AttributeError: 'function' object has no attribute 'Table'
ERROR tests/test_steps.py - AttributeError: 'function' object has no attribute 'Table'
ERROR tests/test_version_tracker.py - AttributeError: 'function' object has no attribute 'Table'

[Marigold]

You can actually write

ds_grapher = create_dataset(
    dest_dir,
    tables=list(ds_garden),
    check_variables_metadata=True,
    default_metadata=ds_garden.metadata,
)

which could be enough.

[lucasrodes]

@Marigold right! that makes sense ー i'll close this issue for now then.