dr-leo / pandaSDMX

Python interface to SDMX
Apache License 2.0
129 stars 60 forks source link

Source contains replaced ABS service #221

Closed Chowti closed 2 years ago

Chowti commented 2 years ago

Hi,

The ABS source url leads to a decommissioned service "https://stat.data.abs.gov.au/sdmx-json". This has been replaced by "https://api.data.abs.gov.au". The new service is SDMXML 2.1 compliant and could now also support the datastructure resource.

Thanks

dr-leo commented 2 years ago

Thanks. This should be added. Guide says that API is in beta. So maybe the JSON version should remain the default for some time.

I've added a new "ABS_XML" source. It is in the v1.7.1dev branch.

Quick test with dataflow query failed with error 404.

Chowti commented 2 years ago

I think the 404 could be due to the agency ID. With all of the dataflows at that endpoint having an ABS agency ID.

I get the same, HTTPError: 404 Client Error: Not Found for url: https://api.data.gov.au/dataflow/ABS_XML/latest

While the url: https://api.data.abs.gov.au/dataflow/ABS/latest works. When adjusting the ABS source to:

  {
    "id": "ABS",
    "data_content_type": "JSON",
    "url": "https://api.data.gov.au",
    "name": "Australian Bureau of Statistics",
    "documentation": "https://www.abs.gov.au/"
  },

I was able to successfully return content via:

import pandasdmx as sdmx

req = sdmx.Request('ABS')

flow_msg = req.dataflow(force=True)
flow_msg

With https://stat.data.abs.gov.au/sdmx-json returning 410 gone for everything, I believe it would be ok to redirect the "id":"ABS" source?

Cheers

dr-leo commented 2 years ago

Thanks and sorry, my mistake. New aBS service has id ABS_XML, old ABS remains. Documented as deprecated though. Does this make sense to you?

Chowti commented 2 years ago

Sweet, the new ABS_XML is working well for me now. And the path as you've documented https://pandasdmx.readthedocs.io/en/v1.0/sources.html#abs-xml-australian-bureau-of-statistics-xml-based-api makes sense to me. Thanks again.