frictionlessdata / frictionless-ci

Data management service that brings continuous data validation to tabular data in your repository via Github Action
https://repository.frictionlessdata.io
MIT License
36 stars 12 forks source link

Failing validation by zipped resources #22

Closed AFoletti closed 2 years ago

AFoletti commented 2 years ago

Overview

I am using the pattern suggested here to describe zipped CSVs.

However, the validation fails with an "Encoding Error" (see here for an example report) while the very same datapackage validate the very same files if they are unzipped and referenced one by one.

You can find my datapackages (both the zipped and unzipped resources version) in the attached zip datapackages.zip


Please preserve this line to notify @roll (lead of this repository)

roll commented 2 years ago

Thanks for reporting!

roll commented 2 years ago

Hi @AFoletti,

Sorry for the slow issue handling.

I checked the Python framework and it works fine for a data package like this:

{
  "profile": "data-package",
  "resources": [
        {
          "path": "data.zip",
          "innerpath": "data.csv",
          "format": "csv",
          "name": "data"
        },
        {
          "path": "data.zip",
          "innerpath": "data2.csv",
          "format": "csv",
          "name": "data2"
        }
  ]
}

It might be a problem with Frictionless Repository. Do you still experience it?

roll commented 2 years ago
frictionless validate https://www.uvek-gis.admin.ch/BFE/ogd/10/datapackage.json
# -----
# valid: https://www.uvek-gis.admin.ch/BFE/ogd/10/ogd10_energieforschungstatistik_ch.csv
# -----
# -----
# valid: https://www.uvek-gis.admin.ch/BFE/ogd/10/ogd10_energieforschungstatistik_iea.csv
# -----
# -----
# valid: https://www.uvek-gis.admin.ch/BFE/ogd/10/ogd10_catalogs.zip
# -----
# -----
# valid: https://www.uvek-gis.admin.ch/BFE/ogd/10/ogd10_catalogs.zip
# -----
# -----
# valid: https://www.uvek-gis.admin.ch/BFE/ogd/10/ogd10_catalogs.zip
# -----
# -----
# valid: https://www.uvek-gis.admin.ch/BFE/ogd/10/ogd10_catalogs.zip
# -----
# -----
# valid: https://www.uvek-gis.admin.ch/BFE/ogd/10/ogd10_catalogs.zip
# -----
AFoletti commented 2 years ago

Hello @roll , and thanks for chicking this out. Actually yes, I still have the same issue. And strangely, the report does not show up anymore 🤔

I am using the github action for my workflow. You can find here the action report. Am I doing something blatantly wrong?

roll commented 2 years ago

Thank you for the feedback @AFoletti, I'll continue investigating

roll commented 2 years ago

Hi @AFoletti,

I created a PR in your repo to resolve the problem which is that Frictionless can't recognize the data package (thinks it's a resource) because it has datapackage_ZIP.json file name. We will think about making the error message is more helpful in this case...

Please re-open if it doesn't help

roll commented 2 years ago

Here is a CLI demonstration:

# ERROR
frictionless validate https://www.uvek-gis.admin.ch/BFE/ogd/staging/datapackage_ZIP.json
# VALID
frictionless validate --type package https://www.uvek-gis.admin.ch/BFE/ogd/staging/datapackage_ZIP.json
AFoletti commented 2 years ago

Hi @roll Many thanks for your help! That works perfectly