csv extract on participating-org/@ref publisher list

[stevieflow]

via https://www.iaticonnect.org/topic/who-else-uses-organisation-reference-use-new-d-portal-feature-check

I notice the CSV extraction doesn't include the participating org ID and name/description. If that could be added we may be able to see the various names given to the same code and help improve some of the data quality issues.

I'm not exactly sure where best such a CSV would be served, but this could be run through a dquery - eg

https://d-portal.org/dquery/#--/*%20Display%20all%20the%20names%20for%20a%20participating-org%20published%20by%20reporting-org%0A%0ASELECT%0A%0Axson-%3E%3E'@ref'%20AS%20%22@ref%22%20,%0Axson-%3E'/narrative'-%3E0-%3E%3E''%20AS%20%22/narrative%22%20,%0Apid%20as%20%22reporting-org%22,%0Acount(*)%20AS%20count%0A%0AFROM%20xson%20WHERE%20root='/iati-activities/iati-activity/participating-org'%20%0AAND%20xson-%3E%3E'@ref'='ET-COA-152'%0A%0AGROUP%20BY%201,2,3%0A%0A

@robredpath @simon-20 @odscjames I'm also guessing such a query could be run on the datastore --> which might make interesting synergies

@xriss @notshi as always, welcome your thoughts

[simon-20]

@stevieflow, you can do a straightforward query on the Datastore searching by participating_org_ref, and when running this now it shows 82 activity results, which is similar to the number of activities shown by that dquery, 88 activities.

You wouldn't be able to do the grouping of results using the Datastore front end website. You may be able to do something close to that using the Datastore API, using a facet search.

But anyway, it's good that the number of activities returned by the two systems is quite close.

[Michelle-IOM]

@simon-20 - I was the one asking and what might be a straightforward datastore search for someone like @stevieflow, is a long and painful exercise for me, a non-techie. This is why I was asking if it could be added to the extraction routine of what we already see. But if there is a better way to do it that doesn't require serious data mining skills then I don't really care how we get there. I'm just trying to see how we can improve data quality so however we get there which is easy for any unskilled user is fine by me.

[simon-20]

Hi @Michelle-IOM, I see, thanks for the clarification. Yes, I think at the moment this isn't possible without interacting with one of the APIs, but perhaps this could be the next feature to be added somewhere.

[stevieflow]

Thanks @Michelle-IOM.

Yes, I was not expecting you to go here and follow this - just that your comment in Connect was very useful, and great evidence of some of the things we need to work on. Thanks!

[Michelle-IOM]

Yes gentlemen, I understood you were talking techie talk and knew that I'm lost in that world. I will patiently wait for a d-portal enhancement whenever it is ready.

[notshi]

Many thanks, all for the conversations and follow ups.

We did propose something like this at the top of the page where we could serve a csv download link - https://github.com/IATI/D-Portal/issues/642#issuecomment-1079713762

However, this will be an entirely new section to be added to the site.