frictionlessdata / frictionless-py

Data management framework for Python that provides functionality to describe, extract, validate, and transform tabular data
https://framework.frictionlessdata.io
MIT License
701 stars 147 forks source link

Regression from version 5.7.2 #1565

Open jze opened 1 year ago

jze commented 1 year ago

Overview

I have code that works with version 5.7.2:

from frictionless import Package, Resource
r1 = Resource('https://opendata.schleswig-holstein.de/dataset/8049b860-6d5c-4ad8-a584-43bec41a6220/resource/13269a62-71af-499e-b4b2-e28882c4d6c1/download/badegewasser-stammdaten-aktuell.json')
r2 = Resource('https://opendata.zitsh.de/frictionless/badegewasser-messungen.json')
r3 = Resource('https://opendata.zitsh.de/frictionless/badegewasser-einstufung.json')
r4 = Resource('https://opendata.zitsh.de/frictionless/badegewasser-saisondauer.json')
r5 = Resource('https://opendata.zitsh.de/frictionless/badegewasser-infrastruktur-aktuell.json')
package = Package(
    name='bathing-water-quality-sh',
    title='Bathing Water Quality Schleswig-Holstein',
    resources=[r1, r2, r3, r4, r5 ]
)

# add foreign keys to connect the resources
f_keys = []
f_keys.append({
  "fields": ["BADEGEWAESSERID"],
  "reference": {
      "resource": "badegewasser-stammdaten-aktuell",        
      "fields": ["BADEGEWAESSERID"]
  }            
})
package.resources[1].schema.foreign_keys = f_keys
package.resources[2].schema.foreign_keys = f_keys
package.resources[3].schema.foreign_keys = f_keys
package.resources[4].schema.foreign_keys = f_keys

package.publish('sqlite:///badewasserqualität.sqlite')

With an up-to-date version 5.15.0 I get this error message:

frictionless.exception.FrictionlessException: [package-error] The data package has an error: descriptor is not valid (The data package has an error: 'https://opendata.schleswig-holstein.de/dataset/8049b860-6d5c-4ad8-a584-43bec41a6220/resource/13269a62-71af-499e-b4b2-e28882c4d6c1/download/badegewasser-stammdaten-aktuell.json' is not of type 'object' at property 'resources/0')
shashigharti commented 1 year ago

Thank you @jze for reporting!. We will investigate.

roll commented 1 year ago

We found the root cause but it will take some time to fix it