Some views broken on datahub.io/core datasets post upgrade to v3

mohamedsalem401 commented 2 months ago

Tasks

Review the main items below first (headings of sections in the "Notes" part below) and decide what can be done or not and then add tasks for fixing:

[ ] Only views with specType: "simple" are supported at the moment. (see the related section in the notes below)
[x] view.spec.type of column not supported ✅2024-06-27 looks like all of these are about column type. i think we can convert to bar or line without much problem
[ ] resources field of views in form of array of objects are not supported (see the related section in the notes below)

See DataPackage Views spec for reference https://specs.frictionlessdata.io/views

Notes

Only views with specType: "simple" are supported at the moment.

Other spec types will cause the following error:

List of affected datasets:

https://datahub.io/core/co2-fossil-by-nation ( 2 Data previews )

view.spec.type of column not supported

Some datasets have a view with an unsupported view.spec.type.
The Vega component is not displaying any errors; it's just showing an empty data preview.

unsupported view.spec.type of type column 
( Valid values: "arc", "area", "bar", "image", "line", "point", "rect", "rule", "text", "tick", "trail", "circle", "square", "geoshape" )

List of affected datasets

https://datahub.io/core/employment-us ( 2 views ) - uses type: column

  {
    "name": "graph",
    "resources": ["aat1"],
    "title": "US unemployment rates since 1940",
    "specType": "simple",
    "spec": {
      "type": "column",
      "group": "year",
      "series": [
        "unemployed_percent"
      ]
    }
  },

https://datahub.io/core/gdp-us ( 1 view )
interest-rates-gb ( 1 view )
investor-flow-of-funds-us ( 1 view )

`resources` field of views in form of array of objects are not supported

Currently, we only support views with simple resources (an array of strings):

"resources": [ resource1, resource2 ]

Some datasets have resources in form of arrays of objects:

List of affected datasets

core/co2-fossil-by-nation ( 2 Data previews )
core/co2-fossil-by-nation
core/co2-fossil-global
core/exchange-rates
core/household-income-us-historical
core/pharmaceutical-drug-spending
core/population
core/population-global-historical
core/population-growth-estimates
core/usa-education-budget-analysis

olayway commented 2 months ago

@mohamedsalem401 so in the spec it says that resource in the resources array of the view field can be an object as well:

https://specs.frictionlessdata.io/views/#data-source-spec

However, I'm not sure if we want to support transforms atm. Probably not. And in the spec it also says that "Data Transforms" are "In progress".

@rufuspollock any thoughts?

rufuspollock commented 3 weeks ago

I think this is a relatively simple item that would be good for someone who wants to work on datasets.

datopian / datahub