Add APIs not listed in the API catalogue

MansMeg commented 2 years ago

There are APIs available that are currently not listed in the package. Several APIs listed here are not in our catalogue (yet): https://www.scb.se/en/services/statistical-programs-for-px-files/px-web/pxweb-examples/

pitkant commented 1 year ago

I went through the list:

PxWeb data bases in some countries/organisations

[x] Croatian Bureau of Statistics
[x] Eustat, Basque
[ ] National Statistics, Republic of China (Taiwan) (This seems to be superseded in favour of a homegrown solution)
[ ] National Statistical Service of the Republic of Armenia
[x] State Statistical Office of the Republic of Macedonia (makstat.stat.gov.mk)
[x] Statistics and Research Åland (pxweb.asub.ax)
[x] Statistics Estonia (andmed.stat.ee)
[x] Statistics Faroe Islands (statbank.hagstova.fo)
[x] Statistics Finland (statfin.stat.fi)
[x] Helsinki city (stat.hel.fi)
[x] Nordstat (Nordic cities) (accessible through stat.hel.fi)
[x] Statistics Greenland (bank.stat.gl)
[x] Statistics Iceland (px.hagstofa.is)
[x] Statistics Latvia (data.stat.gov.lv)
[x] Statistics Norway (data.ssb.no)
[x] Statistical Office of the Republic of Slovenia (pxweb.stat.si)
[x] Statistics Switzerland (www.pxweb.bfs.admin.ch)
[x] UNECE (w3.unece.org)
[x] Nordic Statistics database (Nordic Ministries Nordic countries database) (pxweb.nordicstatistics.org)
[x] Grande Région database (www.grande-region.lu)
[x] INSTAT, Mali

PxWeb examples in Sweden

It is also possible to use components of PxWeb together with selections from an SQL database. (These are maybe not attainable in this package but maybe it's worth exploring)

[ ] Statistics Denmark
[ ] Statistics New Zeeland
[ ] Albania, The Institute of Statistics (INSTAT)

I also found some databases done in pxweb that were not in the list above:

Databases listed in the pxweb package catalogue but not included in the list on SCB website:

Other notes:

Västra Götaland seems to have at least 3 different API urls: https://pxweb2022.vgregion.se, pxwebb2017.vgregion.se and pxwebb2014.vgregion.se. http://www.vgregion.se/statistikdatabas redirects to pxweb2022. All of these, at the time of writing, seem to be out of service so I can't compare them now

MansMeg commented 1 year ago

Great! Could you open up a PR with ehe ones that are missing?

pitkant commented 1 year ago

Entities that did not have API available:

National Statistical Service of the Republic of Armenia
Nordic Health and Welfare Statistics

Entities with broken APIs:

Örebro kommun (old pxweb API version)
EUSTAT (redirects to net-inter-eustat-45.ejgvdns, reported to their helpdesk)
INSTAT, Mali (sometimes fails, sometimes produces an error message, sometimes works, unstable internet connection?)
Icelandic Centre for Retail Studies (px.rsv.is) (no longer existing, timeouts?)

API works fine but produces a warning:

Sundsvall kommun

Error in curl::curl_fetch_memory(url, handle = handle) : 
  Could not resolve host: net-inter-eustat-45.ejgvdns

###

This is an old PXWEB API not supported by pxweb R package
(version PX-Web 2014 Dec R1 or later is supported): 
http://pxweb.orebro.se/api/v1/sv/

###

1: 34
Error: 
This is an old PXWEB API not supported by pxweb R package
(version PX-Web 2014 Dec R1 or later is supported): 
https://pxweb.instat-mali.com/PXWeb/api/v1/fr/BaseDeDonnees/

1: 34
Error: 
This is not a PXWEB API: 
https://pxweb.instat-mali.com/PXWeb/api/v1/fr/BaseDeDonnees/

==============================================================================
 R PXWEB: Content of 'pxweb.instat-mali.com'
==============================================================================
 [ 1 ] : v1
==============================================================================
Enter your choice:
('esc' = Quit, 'i' = Show id) 

###

http://px.rsv.is/PXWeb/api/v1/en
Error in curl::curl_fetch_memory(url, handle = handle) : 
  Timeout was reached: [px.rsv.is] Failed to connect to px.rsv.is port 80 after 10009 ms: Timeout was reached

http://px.rsv.is/PXWeb/api/v1/is
Error in curl::curl_fetch_memory(url, handle = handle) : 
  Timeout was reached: [px.rsv.is] Failed to connect to px.rsv.is port 80 after 10004 ms: Timeout was reached

###

Warning message:
PXWEB URL CHANGE:
 from: https://m02-http-pxwebb.login.sundsvall.se/PXWeb_Ext/api/v1/sv/
 to:   https://m02-http-pxwebb.login.sundsvall.se/PXWeb_Ext/api/v1/sv/

MansMeg commented 1 year ago

Strange warning from Sundsvall.

I think we can skip the ones that doesnt work. They can still use pxweb but we end up with unnessecary maintenance.

rOpenGov / pxweb

Add APIs not listed in the API catalogue #254