IATI / D-Portal

http://d-portal.org/
Other
30 stars 23 forks source link

502 Bad Gateway on search for COVID-19 activities #592

Closed markbrough closed 3 years ago

markbrough commented 3 years ago

Hello! Thanks again for all of your help earlier this year in helping me understand how to access COVID-19 activities from D-Portal.

The COVID-19 visualisation is still using D-Portal as the source for COVID-19 data. However, the following URL is leading to a 502 Bad Gateway error at the moment. Please could you help (or let me know if there is a different URL I should be using)?

Many thanks!

This is the URL that is leading to a 502 Bad Gateway error:

https://d-portal.org/q.xml?from=act&limit=-1&distincton=aid&text_search=COVID-19

notshi commented 3 years ago

Hi @markbrough we will take a look at this but you're better off using the new dquery stuff as this is our old method of querying the database and was only designed for d-portal and not external use.

This URLOR(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Ftitle%2Fnarrative%27%20AND%0A%09to_tsvector(%27simple%27%2C%20xson-%3E%3E%27%27)%20%40%40%20to_tsquery(%27simple%27%2C%27COVID-19%27)%0A)OR(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Fdescription%2Fnarrative%27%20AND%0A%09to_tsvector(%27simple%27%2C%20xson-%3E%3E%27%27)%20%40%40%20to_tsquery(%27simple%27%2C%27COVID-19%27)%0A)OR(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Ftransaction%2Fdescription%2Fnarrative%27%20AND%0A%09to_tsvector(%27simple%27%2C%20xson-%3E%3E%27%27)%20%40%40%20to_tsquery(%27simple%27%2C%27COVID-19%27)%0A)%0A%0A) should get you what you need.

Or visit https://d-portal.org/dquery and under 'EXAMPLES', choose COVID-19 then hover over 'DOWNLOAD XSON' to choose which formats you need. Let me know if this works (or not) for you.

The above error could possibly be due to a timed out error or it has hit a memory limit but we'll need to look at it further.

markbrough commented 3 years ago

Thanks a lot for this @notshi !

This looks good - though I am now trying to work out how to return the full XML data of the activities. When I click "DOWNLOAD XSON" and then "XML" it returns JSON data.

I thought I could just pipe these results to q.xml, but when I run this query I also get a 502 Bad Gateway error:

http://d-portal.org/q.xml?sql=%0ASELECT%20DISTINCT%20aid%20FROM%20xson%20WHERE%0A(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Fhumanitarian-scope%27%20AND%0A%09xson-%3E%3E%27%40type%27%3D%271%27%20AND%0A%09xson-%3E%3E%27%40vocabulary%27%3D%271-2%27%20AND%0A%09xson-%3E%3E%27%40code%27%3D%27EP-2020-000012-001%27%0A)OR(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Fhumanitarian-scope%27%20AND%0A%09xson-%3E%3E%27%40type%27%3D%272%27%20AND%0A%09xson-%3E%3E%27%40vocabulary%27%3D%272-1%27%20AND%0A%09xson-%3E%3E%27%40code%27%3D%27HCOVD20%27%0A)OR(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Ftag%27%20AND%0A%09xson-%3E%3E%27%40vocabulary%27%3D%2799%27%20AND%0A%09xson-%3E%3E%27%40vocabulary-uri%27%20IS%20NULL%20AND%0A%09UPPER(xson-%3E%3E%27%40code%27)%3D%27COVID-19%27%0A)OR(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Ftitle%2Fnarrative%27%20AND%0A%09to_tsvector(%27simple%27%2C%20xson-%3E%3E%27%27)%20%40%40%20to_tsquery(%27simple%27%2C%27COVID-19%27)%0A)OR(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Fdescription%2Fnarrative%27%20AND%0A%09to_tsvector(%27simple%27%2C%20xson-%3E%3E%27%27)%20%40%40%20to_tsquery(%27simple%27%2C%27COVID-19%27)%0A)OR(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Ftransaction%2Fdescription%2Fnarrative%27%20AND%0A%09to_tsvector(%27simple%27%2C%20xson-%3E%3E%27%27)%20%40%40%20to_tsquery(%27simple%27%2C%27COVID-19%27)%0A)%0A%0A&limit=-1

Many thanks!!

markbrough commented 3 years ago

Sorry - I remembered this other thread and worked out how to do this. So this is now working:

http://d-portal.org/dquery?form=xml&sql=%0ASELECT%20*%20FROM%20xson%20WHERE%20root%20%3D%20%27%2Fiati-activities%2Fiati-activity%27%20AND%20%20aid%20IN%20(%0A%0A%0ASELECT%20DISTINCT%20aid%20FROM%20xson%20WHERE%0A(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Fhumanitarian-scope%27%20AND%0A%09xson-%3E%3E%27%40type%27%3D%271%27%20AND%0A%09xson-%3E%3E%27%40vocabulary%27%3D%271-2%27%20AND%0A%09xson-%3E%3E%27%40code%27%3D%27EP-2020-000012-001%27%0A)OR(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Fhumanitarian-scope%27%20AND%0A%09xson-%3E%3E%27%40type%27%3D%272%27%20AND%0A%09xson-%3E%3E%27%40vocabulary%27%3D%272-1%27%20AND%0A%09xson-%3E%3E%27%40code%27%3D%27HCOVD20%27%0A)OR(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Ftag%27%20AND%0A%09xson-%3E%3E%27%40vocabulary%27%3D%2799%27%20AND%0A%09xson-%3E%3E%27%40vocabulary-uri%27%20IS%20NULL%20AND%0A%09UPPER(xson-%3E%3E%27%40code%27)%3D%27COVID-19%27%0A)OR(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Ftitle%2Fnarrative%27%20AND%0A%09to_tsvector(%27simple%27%2C%20xson-%3E%3E%27%27)%20%40%40%20to_tsquery(%27simple%27%2C%27COVID-19%27)%0A)OR(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Fdescription%2Fnarrative%27%20AND%0A%09to_tsvector(%27simple%27%2C%20xson-%3E%3E%27%27)%20%40%40%20to_tsquery(%27simple%27%2C%27COVID-19%27)%0A)OR(%0A%09root%3D%27%2Fiati-activities%2Fiati-activity%2Ftransaction%2Fdescription%2Fnarrative%27%20AND%0A%09to_tsvector(%27simple%27%2C%20xson-%3E%3E%27%27)%20%40%40%20to_tsquery(%27simple%27%2C%27COVID-19%27)%0A)%0A%0A%0A%0A%0A)

This issue can be closed if you wish - thanks a lot!

notshi commented 3 years ago

Glad to hear it's working, @markbrough