Questions about IATI Plus graph database

[FreiburgBill]

Just starting to look your IATI/Refresher project. Great work! A great improvement would be an option to use something other than the Azure Storage, e.g. local storage or AWS S3 buckets.

[FreiburgBill]

I am not a bot, honest.... or is that what a bot would say??
I want to have your application run on AWS. After I have it up and running, maybe we could touch basis, to ether merge some of my changes or talk if I should make a branch project. I can see you have put a lot of thought into the logic, good work!

[nosvalds]

Apologies for the misunderstanding. What benefits are you hoping to gain by running this on application your own on AWS? It would likely cost $1k's/month. We are providing the service for free along with Public APIs:

IATI Validator Datastore

Public APIs

-Nik

[FreiburgBill]

Hi Bill and Nik,

I just finished my work week and will be updating our project repo more over the next days. Nik, it's actually great timing to have you on the thread today.

Relative to the database, behind the scenes we've been cultivating interest at Amazon in piloting an Alexa Prize style competition challenging university teams and others to see how open data published by aid organizations and donors can be used by virtual assistants to answer complex queries relevant to humanitarian operations. We need a database sandbox to support the project that we can add experimental element and attribute nodes and relationships to, other types of data and that we can keep up-to-date up to the minute or hour so to speak with endpoints that can not only power test assistants but serve natural language processing and machine learning model training and testing needs. The point is that the database will need to be responsive and able to quickly adapt to support team needs and interests of the teams. Also I've been long interested in seeing how IATI could look graphed and in the Open Cypher query language initiative. The Ukrainian crisis sparked us to return to work on the database and see how it could potentially support relief operations.

One of our first tasks will involve thinking about the overall high level graphical structure of the database and loading it with Organization name nodes and the URL addresses of their XML files and file last-update info. As a fall-back we thought we'd just scrape the info from the IATI Registry. Maybe you could advise us on approaches and code to drop into a Lambda instance and automate?

Brent

On Wed, Apr 20, 2022 at 10:01 AM Bill Worthington < @.***> wrote:

Hi Nick,

I volunteered to help Brent Phillips Brent Phillips @.***> https://github.com/Brentophillips

to make a searchable Graph Database of the IATI database. https://github.com/Humanitarian-AI/IATIPlus

What he has listed for the first sprint

https://github.com/Humanitarian-AI/IATIPlus/blob/main/Documentation/Sprint01.md

looks like you have already done with your project.

Brennt is currently in the north east of the USA, eastern standard time (est) I am located in Switzerland, central european time zone (cet).

I am very flexible with time. If we could arrange a video conference with Brennt, I am sure you will be able to give this project a tremendous kick start!

@Brennt: Here is the project that Nick has already done: https://github.com/IATI/refresher If you could arrange a video call with him, that would be great!

Regards Bill Worthington

On Tue, Apr 19, 2022 at 4:58 PM Nik Osvalds @.***> wrote:

Apologies for the misunderstanding. What benefits are you hoping to gain by running this on application your own on AWS? It would likely cost $1k's/month. We are providing the service for free along with Public APIs:

IATI Validator https://iativalidator.iatistandard.org Datastore https://datastore.iatistandard.org

Public APIs https://developer.iatistandard.org

-Nik

— Reply to this email directly, view it on GitHub https://github.com/IATI/refresher/issues/110#issuecomment-1102759084, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK2YK5DUFKJQCLYI3IEE3BDVF3CZ5ANCNFSM5TNRBXOA . You are receiving this because you authored the thread.Message ID: @.***>

[nosvalds]

Appreciate the additional context and sounds like an interesting project! Would be happy to discuss with you on a call as that would probably be the quickest way to get on the same page and offer our guidance. Our team is based in the UK and US Eastern time zones. You can email me: code at iatistandard.org to coordinate. Thanks.

[FreiburgBill]

Hi Nick,

I volunteered to help Brent Phillips Brent Phillips @.***> https://github.com/Brentophillips

to make a searchable Graph Database of the IATI database. https://github.com/Humanitarian-AI/IATIPlus

What he has listed for the first sprint https://github.com/Humanitarian-AI/IATIPlus/blob/main/Documentation/Sprint01.md

looks like you have already done with your project.

Brennt is currently in the north east of the USA, eastern standard time (est) I am located in Switzerland, central european time zone (cet).

I am very flexible with time. If we could arrange a video conference with Brennt, I am sure you will be able to give this project a tremendous kick start!

@Brennt: Here is the project that Nick has already done: https://github.com/IATI/refresher If you could arrange a video call with him, that would be great!

Regards Bill Worthington

On Tue, Apr 19, 2022 at 4:58 PM Nik Osvalds @.***> wrote:

Apologies for the misunderstanding. What benefits are you hoping to gain by running this on application your own on AWS? It would likely cost $1k's/month. We are providing the service for free along with Public APIs:

IATI Validator https://iativalidator.iatistandard.org Datastore https://datastore.iatistandard.org

Public APIs https://developer.iatistandard.org

-Nik

— Reply to this email directly, view it on GitHub https://github.com/IATI/refresher/issues/110#issuecomment-1102759084, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK2YK5DUFKJQCLYI3IEE3BDVF3CZ5ANCNFSM5TNRBXOA . You are receiving this because you authored the thread.Message ID: @.***>

[nosvalds]

Hi there,

We'd be happy to share any knowledge we can about the IATI Unified Platform and Datastore. As I said before, best to coordinate a time by emailing us at code at iatistandard.org.

If you're looking to engage with other technical and data use experts of IATI. I would recommend checking out IATI Connect: https://iaticonnect.org/dashboard/welcome-iati-connect .

[akmiller01]

If we want to continue the discussion on Github, I've created a discussion board for it here: https://github.com/IATI/refresher/discussions/238

Thanks.