Closed ca-liu closed 1 year ago
connected with #1010
@ca-liu @EduardoRamirezBC We need to set up a sync call with the PeopleSoft team developers (Mike / Patrick) and Digital Journey developers (Chris, Bhumin, Andrew).
From ticket #1010 got to know from Mike that we are revising database structure hence API endpoints work is pause for a while.
We are performing CRUD operations on Clinic, Contact, and Address tables. Mike will be developing API endpoints for the same. Hence we need to know when we(Digital Journey) can call which API endpoints from the Form(Digital Journey). It required to indicate which API and Form component are relate with each other. Also from ticket #1007 found that we need revision history. Question: If the data stores on PeopleSoft, Do we need to stores revisions on the Digital Journey database? or it will be handle on the PeopleSoft, and we(on Digital Journey) can retrive that data using GET API?
For reference how Peopesoft APIs are connected with Digital Journey system (High level diagram).
cc: @Stella-Archer @DonMoffatt @chriscaldwell-psa @MeghanStothers
@ca-liu @EduardoRamirezBC We need to set up a sync call with the PeopleSoft team developers (Mike / Patrick) and Digital Journey developers (Chris, Bhumin, Andrew).
From ticket #1010 got to know from Mike that we are revising database structure hence API endpoints work is pause for a while.
We are performing CRUD operations on Clinic, Contact, and Address tables. Mike will be developing API endpoints for the same. Hence we need to know when we(Digital Journey) can call which API endpoints from the Form(Digital Journey). It required to indicate which API and Form component are relate with each other. Also from ticket #1007 found that we need revision history. Question: If the data stores on PeopleSoft, Do we need to stores revisions on the Digital Journey database? or it will be handle on the PeopleSoft, and we(on Digital Journey) can retrive that data using GET API?
For reference how Peopesoft APIs are connected with Digital Journey system (High level diagram).
cc: @Stella-Archer @DonMoffatt @chriscaldwell-psa @MeghanStothers
H Bhumin, PeopleSoft is not part of this. We have a data warehouse containing the Influenza database and that Influenza database will be populated by this form: it is not connected to PeopleSoft. Just replace the PeopleSoft in your diagram with "Data Warehouse". Don
@DonMoffatt a quick question here. Who will work on "Data Warehouse" database and provide API endpoints so we can connect it to "Digital journey" system?
@DonMoffatt a quick question here. Who will work on "Data Warehouse" database and provide API endpoints so we can connect it to "Digital journey" system?
Hi Bhuman - I am constructing the new tables in the Influenza db on the DEV server in the Data Warehouse this morning and Mike is my contact for the API endpoints. I will notify him when it is ready. I will loop you in on related emails to Caleb et al. as this proceeds. I am juggling multiple projects but hope to complete this by noon our time.
@DonMoffatt Thank you for your quick response :)
@mikeshi-bcgov based on today's discussion.
We need below API Endpoints to get go in the ongoing sprint.
Note: In point 2 & 3. We send applicationID in the PUT request which help you to identify existing data in database. So you need to add "applicationId" column in worksite and clinic/registration table as well. We have done the same in "Senior leadership review form". Patrick worked on it.
I'm currently using below endpoints, ODS_Datamart_influenza_ministry (list all ministries) ODS_Datamart_influenza_city (list all cities) ODS_Datamart_influenza_worksite (list and filter worksites basedn ministryId[mID]).
@bhumin-fw
Based on the current version of ERD, API endpoints for all tables have been created as below, "telework" has GET/POST/PUT/DELETE permissions, please let me know what else you need.
https://analytics-testapi.psa.gov.bc.ca/apiserver/api.rsc/ods_datamart_influenza_contacts/
https://analytics-testapi.psa.gov.bc.ca/apiserver/api.rsc/ods_datamart_influenza_ministries/
https://analytics-testapi.psa.gov.bc.ca/apiserver/api.rsc/ods_datamart_influenza_registrations/
https://analytics-testapi.psa.gov.bc.ca/apiserver/api.rsc/ods_datamart_influenza_worksites/
@bhumin-fw
FYI: The old API endpoints based on the legacy schema have been removed:
ODS_Datamart_influenza_ministry ODS_Datamart_influenza_city ODS_Datamart_influenza_worksite
@mikeshi-bcgov
for ods_datamart_influenza_registrations/ and ods_datamart_influenza_worksites/ Will you be able to provide me endpoint that work like point 2 & 3 in above comment?
- POST and PUT: Worksite + clinic information (In the PSA Digital journey we have ApplicationID that help us to uniquely identify all the submission for create and update) #998
- POST and PUT: Clinic registration under existing worksite. (for all submission we have applicationID as unique field) #999
Note: In point 2 & 3. We send applicationID in the PUT request which help you to identify existing data in database. So you need to add "applicationId" column in worksite and clinic/registration table as well. We have done the same in "Senior leadership review form". Patrick worked on it.
@mikeshi-bcgov for point 2. in which I'll send worksite and registration both details for new record creation. You need to add separate endpoint.
for point 3, I guess I can use ods_datamart_influenza_registrations/ endpoint only.
Requirement: in POST it should accept applicationID and save on both worksite & registration table and in PUT request it accept applicationID to update data on ODS end points.
We have done the same in "Senior leadership review form". Patrick worked on it.
You can connect with Patrick regarding the same.
@bhumin-fw is the applicationID integer?
@bhumin-fw
field application_id has been added to both ods_datamart_influenza_worksites and ods_datamart_influenza_registrations, please check
@mikeshi-bcgov yes it is integer only. let me match schema with form so I can create sample request for you.
are you preparing new and point? to create worksite and registration in same endpoint. As that will be required, if worksite does not exist then we will have form that contain all fields for worksite and registration in single object.
@bhumin-fw We are working on this
@bhumin-fw The new endpoint for creating records for worksite and registration in single object has been created, please test.
Request Example:
var postdata = { "application_id": 1, "city": "city_2", "closed_reason": "closed_reason_3", "country": "country_4", "is_closed": "is_closed_5", "last_modified_date_time": "2023-04-12T13:17:57-07:00", "ministry_id": 7, "number_of_employees": 8, "place_id": "place_id_9", "postal_code": "postal_code_10", "primary_contact_id": 11, "province": "province_12", "relocating": "relocating_13", "secondary_contact_id": 14, "special_instructions": "special_instructions_15", "street_name": "street_name_16", "street_number": "street_number_17", "submitted_date_time": "2023-04-12T13:17:57-07:00", "year": 19};
Response:
{ "value": [{ "registrations_id": 1, "wm_id": 2, "worksite_id": 3}]}
@mikeshi-bcgov
The new endpoint for creating records for worksite and registration in single object has been created, please test. https://analytics-testapi.psa.gov.bc.ca/apiserver/api.rsc/ods_datamart_influenza_insert_worksite_registration/
Does above API endpoint works with POST (insert) and PUT (update) both request?
Your sample request object. { "application_id": 1, "city": "city_2", "closed_reason": "closed_reason_3", "country": "country_4", "is_closed": "is_closed_5", "last_modified_date_time": "2023-04-12T13:17:57-07:00", "ministry_id": 7, "number_of_employees": 8, "place_id": "place_id_9", "postal_code": "postal_code_10", "primary_contact_id": 11, "province": "province_12", "relocating": "relocating_13", "secondary_contact_id": 14, "special_instructions": "special_instructions_15", "street_name": "street_name_16", "street_number": "street_number_17", "submitted_date_time": "2023-04-12T13:17:57-07:00", "year": 19 }
Object that I can generate from PSA Digital journey form submission system. { / Worksite Details / "street_number": "2 is street number", "street_name": "abbot is street name", "city": "Vancouver", "postal_code": "234 vt5",
/ Primary and secondary contact details / "primary_first_name": "PFirst", "primary_last_name": "PLast", "primary_phone": "(111) 111-1111", "primary_email": "bhumin@freshworks.io", "secondary_first_name": "SFirst", "secondary_last_name": "Slast", "secondary_phone": "(222) 222-2222", "secondary_email": "bhumin@freshworks.io",
/ registration details / "number_of_employees": "45", "ministry_id": 8, "is_closed": "closed", "closed_reason": "justification on cloased", "relocating": "yes", "special_instructions": "Special instruction",
"application_id": "327",
/ extra fields / "application_status": "Resubmit", "change_fields": "No data has been changed", "do_not_see_your_worksite_hidden": "true", "is_registration_open": True, "your_ministry_or_organization": None, "select_your_worksite": None }
It throw 500 if I pass json that generated from form. maybe due to contact or other param details. Please check on your end and do let me know for the same.
Note:
Request to add logic for contact add/update. Here I can send full detail of primary and secondary contact details. You need to add logic that get contact id or create contact details in database, same like worksite and registration. Same apply for update (PUT) request. in put request url will look like /ods_datamart_influenza_insert_worksite_registration(327) and data object will be same as above.
@bhumin-fw
The log shows you are still calling the 3 old endpoints today, as I have communicated they have been replaced with new ones based on the new schema.
@mikeshi-bcgov all the old endpoints just stopped and replace it with new one. In recent couple hours it didn't trigger any old point. Do let me know if it trigged in last 2 hours so I can check DEV environment again for the same.
@mikeshi-bcgov In above object below data will be used to fetch primary_contact_id "primary_first_name": "PFirst", "primary_last_name": "PLast", "primary_phone": "(111) 111-1111", "primary_email": "bhumin@freshworks.io",
Pick primary_email and look up contacts table. If found, fetch and assign it to primary_contact_id key. If no data found for email on contacts table. Add data using above keys & data. retrive newly created records id and assign it to primary_contact_id.
Same apply for the secondary_contact_id too.
@mikeshi-bcgov Could you please let me know why I'm getting 500 error from ODS console/log? endpoint: https://analytics-testapi.psa.gov.bc.ca/apiserver/api.rsc/ods_datamart_influenza_insert_worksite_registration/
Note: I'm passing my suggested json data.
@bhumin-fw What is the time calling? I don't see in the log.
@mikeshi-bcgov Just triggered it, see below error on DEV. Now you might have log your end which you can check.
Error has been observed at the following site(s): *__checkpoint ⇢ 500 from POST https://analytics-testapi.psa.gov.bc.ca/apiserver/api.rsc/ods_datamart_influenza_insert_worksite_registration
@bhumin-fw Logged:
{ "error" : { "date" : "2023-04-13T10:57:49-07:00", "code" : "200", "message" : "[22001] Data too long for column 'relocating' at row 1 (1406)", "resource" : "/apiserver/api.rsc/ods_datamart_influenza_insert_worksite_registration", "pathinfo" : "/ods_datamart_influenza_insert_worksite_registration", "request" : "{\"primary_first_name\":\"PFirst\",\"relocating\":\"yes\",\"secondary_last_name\":\"Slast\",\"city\":\"Vancouver\",\"change_fields\":\"
No data has been changed<\/p>\",\"secondary_email\":\"bhumin1008@mailinator.com\",\"street_name\":\"abbot street name\",\"primary_phone\":\"(111) 111-1111\",\"app_base_url\":\"http://localhost:3000/\",\"secondary_phone\":\"(222) 222-2222\",\"closed_reason\":\"justification on closed\",\"application_status\":\"Resubmit\",\"secondary_first_name\":\"SFirst\",\"primary_last_name\":\"PLast\",\"is_closed\":\"closed\",\"is_registration_open\":false,\"special_instructions\":\"Special Utility\",\"application_id\":\"345\",\"do_not_see_your_worksite_hidden\":\"true\",\"number_of_employees\":\"45\",\"your_ministry_or_organization\":\"undefined\",\"select_your_worksite\":\"undefined\",\"street_number\":\"9887\",\"primary_email\":\"bhumin1007@mailinator.com\",\"ministry_id\":36,\"postal_code\":\"234 vt5\"}" } }
@mikeshi-bcgov
"message" : "[22001] Data too long for column 'relocating' at row 1 (1406)",
"relocating":"yes" its length is 3 only. In your example it shows "relocating": "relocating_13".
not sure why it throws this error, could you help me to understand this error if you have any idea.
@bhumin-fw
The API endpoint does not recognize data type bit from MySQL, have just adjusted to bool. Can you please pass values true/false or 1/0. and try again? for both fields is_closed and relocating
@bhumin-fw Still see many requests for old API endpoints, please check again
@mikeshi-bcgov still see 500, changed is_closed & relocating too boolean. Error has been observed at the following site(s): *__checkpoint ⇢ 500 from POST https://analytics-testapi.psa.gov.bc.ca/apiserver/api.rsc/ods_datamart_influenza_insert_worksite_registration [DefaultWebClient]
Sorry for the inconvenience here, Ministry and worksite are there on DEV image, added PR to remove that. Once PR approved on github I'll deploy new image and it will stop calling old API endpoints.
@mikeshi-bcgov still see 500, changed is_closed & relocating too boolean. Error has been observed at the following site(s): *__checkpoint ⇢ 500 from POST https://analytics-testapi.psa.gov.bc.ca/apiserver/api.rsc/ods_datamart_influenza_insert_worksite_registration [DefaultWebClient]
Sorry for the inconvenience here, Ministry and worksite are there on DEV image, added PR to remove that. Once PR approved on github I'll deploy new image and it will stop calling old API endpoints.
Iman will test.
@bhumin-fw Based on the data elements you are able to generate from PSA DJ form submission, the API endpoint was re-created.
var postdata = { "application_id": 1, "city": "city_2", "closed_reason": "closed_reason_3", "country": "country_4", "is_closed": "is_closed_5", "ministry_id": 6, "number_of_employees": 7, "postal_code": "postal_code_8", "primary_email": "primary_email_9", "primary_first_name": "primary_first_name_10", "primary_last_name": "primary_last_name_11", "primary_phone": "primary_phone_12", "province": "province_13", "relocating": "relocating_14", "secondary_email": "secondary_email_15", "secondary_first_name": "secondary_first_name_16", "secondary_last_name": "secondary_last_name_17", "secondary_phone": "secondary_phone_18", "special_instructions": "special_instructions_19", "street_name": "street_name_20", "street_number": "street_number_21"};
All fields included are required, if there is data please passed blank, for instance for Province and Country.
Below is an example what I have tested using Postman, it worked:
{"application_id": 999, "city": "Victoria", "closed_reason": "Not Closed", "country": "canada", "is_closed": "false", "ministry_id": 10, "number_of_employees": 5, "postal_code": "V4E2B3", "primary_email": "mike.shi@telus.com", "primary_first_name": "mike", "primary_last_name": "shi", "primary_phone": "604-111-1111", "province": "BC", "relocating": "false", "secondary_email": "mike.shi@gov.bc.ca", "secondary_first_name": "michael", "secondary_last_name": "shi", "secondary_phone": "778-111-1111", "special_instructions": "special_instructions_19", "street_name": "Second st", "street_number": "11111"}
@bhumin-fw Sorry, correct the saying in last comment: "All fields included are required, if there is NO data for that field please passed blank, for instance for Province and Country."
Thank you, Mike!
Tagging @iman-jamali-fw for awareness
Deployed Bhumin's PR so hopefully call to the old endpoint should not happen. Will test out the new endpoints today and will update the ticket.
Iman to focus on this ticket today - to ensure issue is resolved.
@mikeshi-bcgov
question: How to filter worksites based on the selected ministry/organization?
Context: In the form, there are two dropdowns. One allows users to select a ministry/organization and another one, lists all worksites. I wonder if there is a way to filter those worksites based on the selected ministry/organizations.
ministry/organizations are fetched from ods_datamart_influenza_ministries
endpoint and return list of following objects:
{
"ministry_id": 1,
"is_active": true,
"m_name": "Advanced Education, Skills and Training"
},
And worksites are fetched from ods_datamart_influenza_worksites
, returning list of objects with following schema:
{ "worksite_id": 1,
"city": "Victoria",
"country": "Canada",
"place_id": "ChIJpeEBgZF0j1QRtRjzzSgGSw0",
"postal_code": "V8W 9B5",
"province": "BC",
"street_name": "Humboldt St",
"street_number": "835",
"application_id": null
}
@mikeshi-bcgov This is great! Thank you
@mikeshi-bcgov
I was able to send a POST request to https://analytics-testapi.psa.gov.bc.ca/apiserver/api.rsc/ods_datamart_influenza_insert_worksite_registration
with applicationId
of 8384
. Could you please confirm that it actually received?
The initial issue, from our end, was that two fields were missing, country
and province
. I added them to the form, but not sure if it's a requirement. This a question for tomorrow's standup, I guess.
Also, there is some mismatch between what we currently send (in the following) and the example you shared in Postman in a few comments above. I listed them in the table below. Could you please check to see if we need to make any changes or if it's OK?
Fields | What we are sending | In your example |
---|---|---|
application_id | string | integer |
is_closed, relocating | boolean for example true |
boolean in double quotes for example "true" |
Another thing I learned is that for the current optional fields in the form (country, city, primary_phone, province, secondary_phone, special_instructions, street_number, postal_code) I can send null
values and post request pass with no problem. But one optional item, ministry_id
broke the request which I changed to a required field. Could you please check the mentioned optional fields to see if it is OK to send null values or do I need to make changes to the form?
@mikeshi-bcgov @ca-liu
Based on what I learned from tables shcema, what needs to pass for registration is worksite_ministry_id wm_id
. Currently only ministry_id
is passed which I think needs to change. Let me know what you think.
Iman was able to register a worksite but there are questions about field - work with Mike and Caleb to sort out details.
@iman-jamali-fw Confirm 3 registrations received, 8383/8384/8385
@iman-jamali-fw
For "registrations" table, for existing worksites, there is a standalone endpoint available which allow GET/POST/PUT/DELETE
https://analytics-testapi.psa.gov.bc.ca/apiserver/api.rsc/ods_datamart_influenza_registrations/
The assumption is on form side you already have wm_id, primary_contact_id/secondary_contact_id.
You may call https://analytics-testapi.psa.gov.bc.ca/apiserver/api.rsc/ods_datamart_influenza_contacts/ to retrieve contact_id.
@iman-jamali-fw
In table schema for "registrations" , there is field "year", do you have the value for it on the form?
@ca-liu is the "year" mandatory?
@iman-jamali-fw
In table schema for "registrations" , there is field "year", do you have the value for it on the form?
@ca-liu is the "year" mandatory?
Hey @mikeshi-bcgov, yes, we need the year as it will help filter data for 2024 registrations.
Thanks for checking!
@mikeshi-bcgov @ca-liu
I don't see year
in the form. Do you want it to be added by the user in the form or added in the background based on some logic, like the current year during submission?
@ca-liu
Another question, I forgot to mention in standup is about two required filed in the registration API, country
and province
. We currently don't have these fields captured (temporarily added to pass submission). in the form. Do you want me to add the fields to the form?
@mikeshi-bcgov @ca-liu
I don't see
year
in the form. Do you want it to be added by the user in the form or added in the background based on some logic, like the current year during submission?@ca-liu Another question, I forgot to mention in standup is about two required filed in the registration API,
country
andprovince
. We currently don't have these fields captured (temporarily added to pass submission). in the form. Do you want me to add the fields to the form?
Hey @iman-jamali-fw,
Thanks for catching this! We don't need a field for year, it will just be the current year that the form is submitted 🙂.
I've added the logic requirement to ticket #989 as well. Not sure if it was added to the FE.
We are using Google Maps (just getting approvals from Privacy right now), the Google Maps API response would provide the country and province: https://developers.google.com/maps/documentation/geocoding/requests-geocoding
Thank you!! 🙂
@ca-liu @mikeshi-bcgov
I added year
to the submission. It's an integer representing the current year during the submission.
One follow-up question for you @ca-liu Do you want the year to capture the initial submission or get updated each time a submission is resubmitted (if that is even supported for this form)?
@ca-liu @mikeshi-bcgov
I added
year
to the submission. It's an integer representing the current year during the submission.One follow-up question for you @ca-liu Do you want the year to capture the initial submission or updated it each time a submission is resubmitted (if that is even supported for this form)?
Hey @iman-jamali-fw,
Great question! year
would be the year that the form was first submitted. We also have a date_modified
row in the database to capture changes. 🙂
@iman-jamali-fw
The endpoint now will accept value for "year"
User Story A short, simple description of a feature or issue told from the perspective of the user. “As a user wanting to register my worksite for a clinic, I did find my worksite listed but I have reasons to register our team as a separate clinic”
Methods:
path = https://analytics-testapi.psa.gov.bc.ca/apiserver/api.rsc/ods_datamart_influenza_{CUSTOM_PATH}/
CUSTOM PATH: ministries HTTP METHOD: GET Output: return array of ministries Ex:
CUSTOM PATH: worksites_ministries_joined/ministry_id? HTTP METHOD: GET Output: return array of worksite_ministries Ex:
CUSTOM PATH: vm_registrations_worksites_joined?{worksite_id} HTTP METHOD: GET Output: return array of registrations Ex:
CUSTOM PATH: insert_worksite_registgration HTTP METHOD: POST Input: forms JSON Backend Logic: See documentation from Mike
New worksite: https://app.zenhub.com/files/439477060/e5d760ab-4fcc-4b75-a990-873ac1700f51/download
Existing worksite: https://app.zenhub.com/files/439477060/f0bcf722-feff-4772-b637-eca9aebb9878/download
Output: return http success / fail message
CUSTOM PATH: delete_registration/ HTTP METHOD: POST Input:
{ "application_id": 1}
Output: return http success / fail messageAcceptance Criteria