Closed stevieflow closed 3 years ago
Pinging @HermanvanLoon on this, as I know he is interested in such a service
Does this: https://datastore.codeforiati.org/api/1/access/activity.xml?iati-identifier=1022405|1022474 pickup the two identifiers in all the various places or just the activity-ids? Sorry, could probably check this but quicker to ask!
If not, might iatikit be an option? https://iatikit.readthedocs.io/en/stable/examples.html#find-an-activity-by-its-identifier also not sure if it searchs all 4 places?
Thanks @matmaxgeds
I think I was thinking along simpler lines!
A - if we pass a string to the Datastore (say 1022474) - can we receive a response to understand if that matches any of the iati-identifier
in the store (true/false)? I think that's possible...
B (which probably comes before A in any workflow!) - any validation service would look for those strings in any of the following attributes in an activity:
Hi Steven, Agree with you, provided the datastore has recent data and its performance is good enough.
Regards, Herman
Hey - so trying to use the datastore with the activity ID 'XI-IATI-WBTF-P147521' as that project has a related activity included to test with.....I think this query should look in all four of the places: https://datastore.codeforiati.org/api/1/access/activity.xml?iati-identifier=XI-IATI-WBTF-P147521|transaction_provider-org.provider-activity-id=XI-IATI-WBTF-P147521|transaction_receiver-org.receiver-activity-id=XI-IATI-WBTF-P147521|related-activity=XI-IATI-WBTF-P147521 - but when I run it, it only returns the activity matching the activity_id field, whereas https://datastore.codeforiati.org/api/1/access/activity.xml?related-activity=XI-IATI-WBTF-P147521 matches the other activity that should be picked up.......I guessed at the use of '|' for 'or', maybe that is not valid?
I also don't know how to get the datastore to return a yes/no result, although returning any conflicting activities might be useful in case there is a conflict found.
@HermanvanLoon Datastore Classic is updated every night now - last update was 5 hours ago. See here: https://datastore.codeforiati.org/ @stevieflow we can think about how we could add a new endpoint to just return a list of matching IATI activity IDs rather than the full XML
You can currently do this through the dQuery interface, for example 1022474 gets you two results. You can click on Browse Activities to view these activities on d-portal.
It takes a while to load because it uses an expensive wildcard search.
Something similar was raised here as well https://github.com/devinit/D-Portal/issues/510.
In this case, the wildcard %
is only placed after the search term to get results that begin with the search term.
I’ve had a go at a purpose-built thing here: https://activity-id-checker.codeforiati.org
It currently looks very ugly… It’s pretty fast, though, because it just does this one thing. The big list of IDs is auto-updated daily.
That's superfast! It's also really cool to see the identifiers as you type. Very handy when checking questionable looking identifiers!
If it's helpful, you can also use this url to get d-portal pages (much faster than the dquery option): http://d-portal.org/ctrack.html?aid_like=%251022474%25#view=main
%25
id looks for activities that end with this id.
id%25
looks for activities that start with this id.
%25
id%25
looks for activities that has this id within it.
If it's helpful, you can also use this url to get d-portal pages (much faster than the dquery option): http://d-portal.org/ctrack.html?aid_like=%251022474%25#view=main
Oh wow – that’s great! Yes, that’s super fast!
Very handy when checking questionable looking identifiers
Yeah, I was wondering if there’s value in using this to flag good/bad identifiers. E.g.:
This identifier is good because it:
- uses publisher ID as a prefix
- doesn’t contain any special characters
- uses an org-id prefix?
- doesn’t use lowercase characters?
That sort of thing.
I think we can reasonably mark this as “done”!
Agreed / huge thanks all (esp @andylolz !)
Two follow-ups:
This should be listed on https://codeforiati.org/ ?
Kk, sorted now!
I'll create a new ticket about an API on/for this -- which would need thoughts around batch checking etc
So Activity ID checker currently offers no API! (There sort of is one, but it is baffling). The site is just a glitzy frontend.
I can probably add an API… But if someone wants to batch check identifiers, they’re much better off using d-portal or DSClassic.
In d-portal, it would be this sort of thing: https://d-portal.org/q.json?aid=GB-CHC-800672-GTF309
In DSClassic, this sort of thing: https://datastore.codeforiati.org/api/1/access/activity.json?iati-identifier=GB-CHC-800672-GTF309
Rationale
For many organisations trying to implement some form of traceability, being able to validate the activity
iati-identifier
they cite in various attributes, exists would be a valuable service.IATI documentation even includes:
Currently, there is no viable way for someone to check if the identifier they include in their data is "valid"
Proposal
Two fold:
1 - a "list"/service of current IATI activity identifiers.
Whilst this cannot possibly be a "codelist" (of over a million codes) to navigate, it'd be interesting to know if the lists of current activity identifiers (pulled into Datastore Classic) could be queried by a user (to return a true/false). Is that function already in place?
2 - Simple validation of an IATI XML file If I pass an IATI XML file, there are four distinct places where I can cite an external IATI activity identifier:
A validation service would check whether any of these fields are populated, and then -- using the service in [1] as to whether they currently exist
The output to the user could then be whether any false matches were found (and even the positive matches)