Integration of IATI registration

[praweshsth]

As per the delivery in ToR and regarding the automatic integration of the IATI registry,

1. What are the fields that we need to show during account registration in publisher so that it will be enough for IATI registration?
2. Since registration in IATI registry requires manual verification, what would the verification flow of registry be like? Will the registration process go into moderation?

[praweshsth]

Regarding the registration in IATI registry while creating user account in IATI publisher, there are two possible scenarios:

1. New user hasn't registered in IATI registry:

• The publisher will provide a form along with the create account form to input data that will be required to create organization ID in the IATI registry
• The publisher will communicate with IATI registry and automate the registration process via API
• The publisher will be able to see if the registration has been verified by IATI provided the API is available.
• The user will be able to explore, change the settings, add activities and other tasks in the system except for publishing in IATI registry.

2. New user has already registered in IATI registry:

• The publisher will provide publisher ID while creating the account
• The tool will use API to pull and store the API key of that organization provided the API provide this service

In both case if there is issue with organization ID from IATI registry, inform user to contact IATI to create/ fix the Organization ID issue.

Audrey to talk to Andy regarding the API

[anjesh]

In the first scenario New user hasn't registered in IATI registry, lets mention that the user will be able to start publishing once verified, and until then the user will be able to do everything except for publishing.

The second scenario New user has already registered in IATI registry: needs revisit as we shouldn't allow the API to pull the API key (now called API token) based on the publisher-id as anyone will be able to publish on behalf of anyone in such instances as publisher-id is visible and could also be guessed.

I think we need a flow where the user will be asked if they have already account in IATI Registry or not, if yes - then they will need to pull the publisher-id and API token from registry and paste into this system. Else if they are just registering in the system for the first time, then only we will allow them to create their account in registry and pull API token from the registry.

With regards to org-id, we need a separate mechanism to deal with missing org-id-registrar. For example if any org-id list is missing from https://org-id.guide/results?structure=all&coverage=all&sector=all, then we need a different approach during the registration process as well.

[andreaszenasidi]

As promised, I am sharing the mapping between the registration form and the Registry API endpoint parameters. I checked with the CKAN API endpoints and realised that some fields are indeed tailored for IATI e.g. publisher_iati_id

Publisher Information Publisher Id -> name Publisher Name -> title IATI Organisation Identifier -> publisher_iati_id Publisher Type -> publisher_organization_type Data License -> license_id Country -> publisher_country Publisher Logo URL -> image_url State -> state This field is only visible for sysadmins, this is the Approved vs. Pending state of the account Description -> publisher_description

Contact Information Contact Email -> publisher_contact_email Website -> publisher_url Address -> publisher_contact

Publishing Additional Information Source -> publisher_source_type First published date -> publisher_first_publish_date Implementation Schedule -> publisher_implementation_schedule Organisations / agencies covered -> publisher_agencies Timeliness of Data -> publisher_timeliness Frequency of publication -> publisher_frequency_select Frequency of publication (extra comments) -> publisher_frequency Units of Aid -> publisher_units Segmentation of Published Data -> publisher_segmentation Data Definitions and References -> publisher_refs Field Exclusions -> publisher_field_exclusions Record Exclusions -> publisher_record_exclusions Thresholds -> publisher_thresholds
Other Constraints -> publisher_constraints Data Quality -> publisher_data_quality User Interface -> publisher_ui User Interface Website -> publisher_ui_url

[praweshsth]

Thank you @andreaszenasidi. We will inform you if we have any questions regarding the same.

[praweshsth]

@andreaszenasidi cc: @anjesh @man-git007 @manibibek @anjhero

As per our meeting last week, we have questions regarding the IATI registry API integration in IATI Publisher. We have come up with flows for two different scenarios, which are explained below. And along with the flow, we want to understand if the API provides what the flows require:

1. Scenario 1: The new user in IATI Publisher doesn't have registered in IATI Registry Flow: The new user adds information about the organization and user information (this is for registering only in IATI Publisher) and submits. Upon that IATI Publisher calls the API endpoint and provides the organization information to IATI Registry to create a new publisher in it and in response the API provides confirmation to IATI publisher. Questions

• In IATI Registry, to create a new publisher, the user has to sign up first and then only she/he can create a new publisher in it. If it is the case, then is there an API endpoint for creating user in IATI Registry?
• Is there an API endpoint for creating API token (which is done manually in IATI Registry) and returns the API Token in response?
• If the user's 'IATI Organization Identifier' doesn't match, does the API respond to that as well?

2. Scenario 2: The new user in IATI Publisher is already registered in IATI Registry Flow: The new user adds information about the IATI Registry (such as publisher ID and other details) and user information and submits. Upon that IATI Publisher calls the API endpoint and provides the information about the IATI Registry. The API endpoint provides the API token. Question

• Is there is provision in API to respond back with API token for the publisher?

Based on this information, we will be able to finalize if the IATI registry API integration is possible or not?

[andreaszenasidi]

@praweshsth cc: @anjesh @man-git007 @manibibek @anjhero

1. Scenario 1: The new user in IATI Publisher doesn't have registered in IATI Registry

• In IATI Registry, to create a new publisher, the user has to sign up first and then only she/he can create a new publisher in it. If it is the case, then is there an API endpoint for creating user in IATI Registry? _This is possible, the endpoint is user_create._

• Is there an API endpoint for creating API token (which is done manually in IATI Registry) and returns the API Token in response? _This is possible, the endpoint is token_create._

• If the user's 'IATI Organization Identifier' doesn't match, does the API respond to that as well? When a new publisher account is created this will be automatically in a Pending state until the BAs manually verify the account for the accuracy of data and verify if the IATI Organisation Identifier is formed correctly. When the data is correct the publisher account status is updated manually in the Registry from Pending to Approved. The Registry will automatically send out an email to the publisher that their account was approved.

2. Scenario 2: The new user in IATI Publisher is already registered in IATI Registry

• Is there is provision in API to respond back with API token for the publisher? Could you please clarify the flow. What type of information does the user enter about the Registry account? Can the user just enter the publisher ID and API Token? Then the Publishing Tool can verify if these are correct, similar to what AidStream dows now.

[praweshsth]

@andreaszenasidi we will check the endpoints for creating user, creating publisher and creating token in Registry that you have provided to define the flow of registration in IATI publisher and IATI registry.

Regarding the second scenario, we can ask for the publisher ID and Token key while registration or after sign up in setting.

[praweshsth]

@andreaszenasidi cc: @manibibek @man-git007 @anjesh

We tried to use the endpoints to create a user, it's organization (publisher) and API token. Here's the summary along with questions:

1. We can not create a new user as we were not authorized to do so. It requires an authorized API token, lets say a 'Master API token', which is also mentioned in the CKAN API documentation. Is it possible to provide such Master API token for us to test if it exists? And do we use the same token in the future while creating user from IATI Publisher?
2. Similarly, we can not create a new API token for an existing user without an authorized API token. Is it possible to use the same Master API token to create the API token for an existing user? And do we use the same token in the future while creating a new API token from IATI Publisher?
3. We were successful in creating a new organization (publisher) for an existing user and it's API token.

If there is a master API token, then provide us so we can test it again and make sure the following flow works:

1. Create a new user with master API token
2. Create an API token for the new user using the user detail and the master API token
3. Create a new organization (publisher) for the new user using organization details, new API token and user details

[andreaszenasidi]

@praweshsth I reached out to the Registry developers for information about the master token. I also requested for them to add you to the Registry Staging environment so you can continue your testing there. I will give you an update as soon as I hear back.

[praweshsth]

That would be great. Thank you @andreaszenasidi

[praweshsth]

@andreaszenasidi Is there any update on this? We need to figure this out soon as this will decide the flow of the user registration in IATI publisher and the design for it as well.

[andreaszenasidi]

@praweshsth, unfortunately, no answer yet. I followed up with the dev team, let's hope they will get back to me soon.

[praweshsth]

@andreaszenasidi Please inform us as soon as you get any information on this.

[andreaszenasidi]

@praweshsth just got off a meeting with the Registry team. Could you please share the email you want to use for the admin account for the staging envrionment?

[praweshsth]

@andreaszenasidi prawesh.shrestha@yipl.com.np We would also need your help on using the staging environment as well.

And what about the queries about the API - https://github.com/younginnovations/iatipublisher/issues/8#issuecomment-1031165924? Did they provide you with any information on this?

[andreaszenasidi]

@praweshsth thanks, I let you know when staging is setup.

You will need some sort of master API Token, but I am talking to the developers now to clarify it.

[praweshsth]

Thank you @andreaszenasidi

[andreaszenasidi]

@praweshsth You have received a new email that informs you that you are a new admin in the Registry staging environment. However, after further discussions with the developers, I requested them to add you as a sysadmin, because only with this permission will you be able to create new publishers. Apparently, the master token you will need to use is basically a token associated with this sysadmin.

[praweshsth]

@andreaszenasidi Yes, I got the email but it was in spam so didn't know for a while. But the link didn't work as I was promted to provide my username and password instead of changing password. And also, we will need to endpoints of staging of IATI registry to test the user, publisher and api token creation. How can we get that?

[andreaszenasidi]

@praweshsth As I mentioned in the call I a trying to create a sysadmin user for you to have the necessary permissions for all the operations you will do. Please ignore the email you received for now, since that user did not have all the permissions you will need.

Staging endpoint: https://staging.iatiregistry.org/ E.g. https://staging.iatiregistry.org/api/action/organization_show?id=cecipub

[praweshsth]

@andreaszenasidi any update on this? we are ready to test it out and finalize it.

[andreaszenasidi]

@praweshsth not yet. I just requested an update from the devs, I will keep you posted. Thanks for your patience.

[andreaszenasidi]

@praweshsth did you receive an email that you were added as a sysadmin and instructing you to setup a password?

[praweshsth]

@andreaszenasidi I haven't received any email for sysadmin.

[andreaszenasidi]

@praweshsth I emailed you the basic authentication parameters you need to add to the staging API endpoint calls, as well as to login to the Staging site.

I was mistaken about the email. The Registry team informed me that they kept the same user they created the first time (the first email you received) and increased the user's permission to sys admin. I notified them that you couldn’t set the password for that user because the link didn’t work and requested them to set a temporary password for you to be able to sign in.

[praweshsth]

@andreaszenasidi I am able to login now and our team will work on testing the endpoints for creating user, publishers and api token. However, we need to know what is the process for a sysadmin to approve a new publisher in the staging of IATI Registry. That will be of great help in our testing.

[andreaszenasidi]

@praweshsth When a user creates a new publisher account, then all the sysadmins will receive an email that a new publisher account was created and it needs to be checked and approved. Since you are a sysadmin now you will be receiving these emails. Then you can click the link in the email which will open the Pending publisher account in the Registry (staging). The yellow banner (see screengrab) will indicate that the publisher is waiting for approval. Then you will click the Manage button (see screengrab) and scroll down to the State field, change it from Pending to Active and click Update Publisher at the bottom of the page. This is the full process of approving a publisher if their profile is correctly filled.

For example, when uxtest1 new publisher account is created, you will receive an email with this link: https://staging.iatiregistry.org/publisher/uxtest1

[andreaszenasidi]

@praweshsth the email you received when your first account was setup should prompt you to change your password without requiring you to have an existing password. Could you please confirm that this is how it's working? If not could you please forward me the email you received? andrea.szenasi@devinit.org

[praweshsth]

Thank you @andreaszenasidi. I am able to login to the staging site as sysadmin. Yesterday, during meeting, Audrey and Amy helped me to learn a few things about what I have asked you.

[praweshsth]

@andreaszenasidi We have successfully tested the creation of user, publisher and token api in IATI Registry using the 'Registry Basic authentication param' and an API Token, which was created by using a sysadmin user. So that means it is possible to do all these in production of IATI Registry from IATI Publisher.

We want to confirm two things for production:

1. if we require similar 'Registry Basic authentication param' for the production too?
2. Will system admin of IATI Registry provide the API Token for the authentication?

Also, your idea on checking if the publisher has been approved in IATI Registry has also worked.

CC: @man-git007 @sriza

[andreaszenasidi]

@praweshsth 1. if we require similar 'Registry Basic authentication param' for the production too? You won't need a basic auth param in production.

2. Will system admin of IATI Registry provide the API Token for the authentication? Yes, we can provide one for the tool. Let me know when you anticipate you will need this.

[praweshsth]

@andreaszenasidi we have come up with an idea about signup in IATI publisher and IATI Registry as we have mentioned yesterday in our meeting. We can discuss this tomorrow in our meeting. But for now, here's the flow for reference:

https://www.figma.com/proto/upgUy5i65LYC3iiqYVnACW/IATI-Publisher-Platform?page-id=2359%3A26662&node-id=2360%3A26666&viewport=306%2C48%2C0.44&scaling=scale-down-width

[andreaszenasidi]

@praweshsth after seeing the flow actually it makes sense and it's a good solution to make the registration faster!

[praweshsth]

@andreaszenasidi That flow is basically the setup flow for the user after signing up and signing in and here's the signup flow:

This flow is for quick signup for new user in IATI publisher and IATI Registry. After it is done, they can follow the above flow to setup publishing information and other settings required for publishing data in IATI.

What do you think about this? Should we go ahead with these two flows?

[praweshsth]

@andreaszenasidi After our last meeting, here's the final flow for user registration for both situations: Not registered in IATI Registry and Registered in IATI Registry

06dacb1f0.png)

And here's the flow for Publisher settings after sign up including other setup flows:

https://www.figma.com/proto/upgUy5i65LYC3iiqYVnACW/IATI-Publisher-Platform?page-id=2359%3A26662&node-id=2360%3A26666&viewport=306%2C48%2C0.44&scaling=scale-down-width