InseeFrLab / onyxia

🔬 Data science environment for k8s
https://onyxia.sh
MIT License
458 stars 80 forks source link

Data Explorer: Support for URLs without .parquet file extensions #809

Open trygu opened 4 months ago

trygu commented 4 months ago

Description:

The Data Explorer component in Onyxia currently does not support accessing parquet data from URLs that do not have the .parquet file extension. This limitation hinders the ability to fetch and explore parquet data provided by APIs, which often do not include file extensions in their URLs.

Problem:

When attempting to access parquet data from URLs provided by APIs, such as the example below, the Data Explorer fails to process the data correctly:

Example URL:

https://data.qa.ssb.no/pxapi2-beta/api/v2/tables/12462/data?lang=no&valueCodes[ContentsCode]=Indeksnivo&valueCodes[Tid]=2023M01,2023M02,2023M03,2023M04&codelist[NaringUtenriks]=vs_NaringPPI1&outputformat=parquet

Steps to Reproduce:

  1. Open the Data Explorer component.
  2. Attempt to load parquet data from the example URL above.
  3. Observe that the Data Explorer fails to load the data.

Expected Behavior: The Data Explorer should correctly handle URLs pointing to parquet data regardless of whether they include a .parquet file extension.

Additional Context: This feature is essential for working with data APIs that dynamically generate and serve parquet files without including the .parquet extension in the URL.