frictionlessdata / pilot-catalyst

Pilot with the Catalyst cooperative

2 stars 2 forks source link

Investigate/implement resource groups (logical partitioning) #12

Closed roll closed 4 years ago

roll commented 4 years ago

Overview

Instead of using multi-part resources it was decided to use one resource per partition (e.g. by year). There are some requirements:

ability to export to database
ability to work with using datapackage-py

Tasks

[x] investigate the complexity of this decision
[x] get back to Catalyst regarding the point above
[x] implement resource.group support for datapackage-py reading/etc
[x] implement resource.group support for SQL export
[x] ~~implement resource.group support for SQL import (if not complex)~~

roll commented 4 years ago

It's done - will be released after a review https://github.com/catalyst-cooperative/pudl/issues/364#issuecomment-523445496