search

codeforIATI / iati-ideas

💡 Ideas for new codeforIATI projects and blogs
https://ideas.codeforiati.org
0 stars 0 forks source link

[PROJECT IDEA] Create a registry API backup, by scraping and serving the existing API #15

Closed andylolz closed 3 years ago

andylolz commented 4 years ago

I think this was the idea @markbrough proposed on the codeforIATI call.

Did I get this right, @markbrough?

andylolz commented 3 years ago

On discord, @notshi said:

That would be nice if the Registry is just a single file that you can download because it's a small enough amount of data that it should be!

I’ve had a go at implementing that here: https://registry.codeforiati.org

notshi commented 3 years ago

Thanks, @andylolz - I meant the Registry could be replaced, not just mirrored :}

We've been grabbing this data and it's in git so there's a history attached to it, if you're interested.

https://github.com/devinit/D-Portal/blob/master/dstore/json/publishers.json https://github.com/devinit/D-Portal/blob/master/dstore/json/packages.json

andylolz commented 3 years ago

We've been grabbing this data and it's in git so there's a history attached to it, if you're interested.

Ha! I should have known d-portal would have already done it, years ago! :)

I think I did know that you already make this available, but completely forgot I’m afraid.

Okay, so… https://registry.codeforiati.org is duplicative of that effort, but I think I’ll leave it running anyway, because:

I’m clutching at straws a bit!

I meant the Registry could be replaced, not just mirrored :}

Gotcha, right. Definitely interested in exploring this idea a bit more. As well as what I posted on discord about this: I think the registry provides a pretty bad UX for data users currently, but I gather it does provide a bunch of services to publishers.

notshi commented 3 years ago

We really don't have a problem with duplicative efforts - that's why the internet stays up :} We are unsure why this is seen as a bad thing.

You can still provide a service for the publishers but it could be in a known IATI format, at the very least. The CKAN has its own standard, separate from IATI (why?!) and the API is also frustratingly slow. This doesn't help new users coming in cold.