dr-leo / pandaSDMX

Python interface to SDMX
Apache License 2.0
127 stars 59 forks source link

NotImplementedError when using Eurostat's DSD #242

Open mephinet opened 1 year ago

mephinet commented 1 year ago

I recently upgraded to pandasdmx[cache]>=1.10.0 after the recent Eurostat API change. At the same time, I improved my code to follow the walkthrough document and use structure-specific data.

However, now I run into a NotImplementedError when accessing Eurostat.

The minimal reproducer is:

import pandasdmx

estat = pandasdmx.Request("ESTAT")
resource_id = "demo_gind"
flow_msg = estat.dataflow(resource_id)
dsd = flow_msg.dataflow[resource_id.upper()].structure
data = estat.data(resource_id=resource_id, dsd=dsd)

When I remove the dsd argument from the data call, the code runs without issues.

beto-Sibileau commented 1 year ago

Hi @dr-leo, adding some more references to this one, essentially the same error to a similar flow:

Code snippet below:

import io

import pandasdmx as pdsdmx
import requests

# pdsdmx requests objet (ESTAT)
Estat = pdsdmx.Request("ESTAT")
# ESTAT dataflow
d_flow = "hlth_silc_08".upper()

# ESTAT structure query (pdsdmx)
Dsd_estat = Estat.datastructure(d_flow)

# ESTAT data query
api_param = {
    "startPeriod": "2022",
    "endPeriod": "2022",
    "format": "SDMX_2.1_STRUCTURED",
}
url_estat = f"https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/{d_flow}/.PC..NO_UNMET...BE+BG"
indicator_xml = requests.get(url_estat, api_param, timeout=15)

buffer_read = io.BufferedReader(io.BytesIO(indicator_xml.content))

try:
    xml_message = pdsdmx.read_sdmx(
        buffer_read, format="XML", dsd=Dsd_estat.structure[d_flow]
    )
    data_df = xml_message.to_pandas(dtype=str, attributes="o", rtype="rows")
    print("OK")
except Exception as error:
    print(error)