Closed henryvalentine closed 3 years ago
@henryvalentine please send me the backup of this facility in a private email.
@henryvalentine the second issue listed per StateCode is as a result of missing LGACode. Ensure LGACode is included and try again. That should resolve that part. I also think the other issues are as a result of wrong data in the database. Validating the enums hences throws the error as per WHOClinicalStage. You may want to cross-check with the data dictionary to confirm you have the accurate values and that the module has the correct enum values @wonderfulguru1 you can confirm this as well and let us know.
@henryvalentine thanks for the feedback once again. It seems you are not using the most update DB kindly refer to the online instance you will notice some of your forms are different. Especially the HIV Enrollment Form. Please compare the online instance and make necessary adjustments. Thanks
The vital signs of the care card form have a Pregnant/Breastfeeding Status field and this is also missing in the DB you sent. look here to see the correct form. https://nmrsdev.mgic-nigeria.org/openmrs/htmlformentryui/htmlform/enterHtmlFormWithStandardUi.page?patientId=0b4a8b06-88a4-4dfd-b004-ac9398254210&visitId=&formUuid=5d522e63-463e-4a9f-a2c1-7ebbe4069a49&returnUrl=%2Fopenmrs%2Fcoreapps%2Fclinicianfacing%2Fpatient.page%3FpatientId%3D0b4a8b06-88a4-4dfd-b004-ac9398254210%26. I am available for more clarifications.
@henryvalentine have you confirmed the issues pointed out by @wonderfulguru1. Let us know if that resolves your issue so we can close this issue or provide you additional support to get this done.
@wonderfulguru1 please I have taken my time to compare this database forms with the online reference you pointed at but they are exactly the same. If you are looking at it from a database explorer/management studio like SQLyog or workbench, I will suggest you do it from the forms on the interface. I was a bit excited when I saw your messages and was like maybe that's why the EDDandPMTCTLink values are not coming out but it is not so.
@caniekwe please where do I include the LGACode so that it will be extracted as it should? Also, I thought the extraction module should be able to pick the LGACode for each patient from the address2 column on the person_address table if it is provided.
Sorry guys that this response is coming late but I've been a bit unsettled since yesterday.
@henryvalentine and Team. Welldone for the effort. The coded values of lgas and states are stored address_hierarchy_entry table.
@caniekwe the export plugin picks it if available you can also edit for a patient from here https://nmrsdev.mgic-nigeria.org/openmrs/admin/patients/patient.form?patientId=62 for any patient that doesn't have
@henryvalentine please confirm you have breastfeeding field on the DB you sent me because I didn't see the field on the DB you sent me. If you have confirmed you can resend and ensure you password it. Please check the care card form see the link again for a recheck
Thanks
@henryvalentine thanks for the effort we might have to do a zoom then the same DB is giving this image
(REDACTED - because it contains private information)
@henryvalentine please when can we do zoom for us to check thing together, It will really help. thanks. hope you can update LGAcode now
@brightoibe thanks for your help and support please have you tested your feedback will really help.
@wonderfulguru1 I am currently in Lagos and my network is really not strong in the hotel i'm staying but we might give it a try now. Just give me a call or share a zoom link
@wonderfulguru1 please use a female patient and you will see the pregnancy status drop-down become visible. You are not seeing it because you selected a male patient according to the image you attached to your response
Morover, the LGACodes have always been available in the address_hierarchy_entry table except there were unpublished changes. What @brightoibe scripted and sent to me about an hour ago is what we have in our databases.
@wonderfulguru1 this is noted. I will get back to you soon.
@henryvalentine were any files produced when you run the NDR export? or does it just fail outrightly
@henryvalentine and Team please when sharing screenshot here, Kindly remove the name of the patient and the hospital, this is a public forum that anyone can run into. Let's keep the patient anonymous.
@henryvalentine please remove and re-upload the screenshot without the name @wonderfulguru1 same with your screenshot Thank you
@henryvalentine the error cvc-type.3.1.3: The value 'BF' of element 'EDDandPMTCTLink' is not valid.
is a warning that the value BF
should not be in that field because it's not in the xsd. here is the section defining EDDandPMTCTLink
in the xsd in the code.
<xs:annotation>
<xs:documentation xml:lang="en">
Code that represents
the EDD and PMTCT Link of the patient this visit
</xs:documentation>
</xs:annotation>
<xs:simpleType>
<xs:restriction base="CodeType">
<xs:enumeration value="P"/>
<xs:enumeration value="PMTCT"/>
<xs:enumeration value="NP"/>
<xs:enumeration value="NK"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
the xsd needs to be updated to the latest one with the value BF
listed
<xs:annotation>
<xs:documentation xml:lang="en">
Code that represents
the EDD and PMTCT Link of the patient this visit
</xs:documentation>
</xs:annotation>
<xs:simpleType>
<xs:restriction base="CodeType">
<xs:enumeration value="P"/>
<xs:enumeration value="PMTCT"/>
<xs:enumeration value="NP"/>
<xs:enumeration value="NK"/>
<xs:enumeration value="BF"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
I've attached the updated xsd, @wonderfulguru1 will update it and share a new build. Please note this error was a warning and will still append the value BF
in your file.
Looks like that's the only issue from the logs
the updated one is uploaded already https://github.com/nmrs-nigeria/files_and_resources/tree/master/Release/nigeriaemr/1.6.0
@wonderfulguru1 kindly replace the XSD we have in the source
@brightoibe that is how it is on our databases. I only responded to @caniekwe in case there is an extra step I have to take to make the LGACode extracted by the extraction plugin
@wonderfulguru1 please have a look at the three images attached to this response and you will see the selection of the three pregnancy statuses from my localhost using the same database I sent you
thanks @wandechris for pointing out the privacy breach.
Thanks @wandechris for the updates on the xsd...I look forward to testing again and providing feedback with a 'thumbs up'
the updated one is uploaded already https://github.com/nmrs-nigeria/files_and_resources/tree/master/Release/nigeriaemr/1.6.0
@wonderfulguru1 kindly replace the XSD we have in the source
@wandechris thanks for this
@henryvalentine @wonderfulguru1 has shared an updated build, kindly test again to confirm the fix
@wandechris on it
@wonderfulguru1 the 'BF' flag doesn't show up again. However, the following are still flagged during extraction:
cvc-type.3.1.3: The value 'I' of element 'WHOClinicalStage' is not valid. LINKED EXCEPTION: org.xml.sax.SAXParseException; lineNumber: 0; columnNumber: 0; cvc-type.3.1.3: The value 'I' of element 'WHOClinicalStage' is not valid.
cvc-enumeration-valid: Value 'I' is not facet-valid with respect to enumeration '[1, 2, 3, 4]'. It must be a value from the enumeration.
cvc-complex-type.2.4.b: The content of element 'OtherPatientIdentifiers' is not complete. One of '{Identifier}' is expected. LINKED EXCEPTION: org.xml.sax.SAXParseException; lineNumber: 0; columnNumber: 0; cvc-complex-type.2.4.b: The content of element 'OtherPatientIdentifiers' is not complete. One of '{Identifier}' is expected.
cvc-complex-type.2.4.a: Invalid content was found starting with element 'StateCode'. One of '{WardVillage, Town, LGACode}' is expected. LINKED EXCEPTION: org.xml.sax.SAXParseException; lineNumber: 0; columnNumber: 0; cvc-complex-type.2.4.a: Invalid content was found starting with element 'StateCode'. One of '{WardVillage, Town, LGACode}' is expected.
Then surprisingly, the name of the XML files have 'UNKNOWNUNKNOWN': "....openmrs\tmp\openmrs\downloads\NDR\20-08-2020\UNKNOWNUNKNOWN_uq98eMvdS5R_CT9196_200820.xml", including the completed data file: UNKNOWNUNKNOWN_uq98eMvdS5R_200820.zip
The log file is also attached here
@henryvalentine the UNKNOWN is showing because you've not configured partner_reporting_state
and partner_reporting_lga_code
in global property
see details here https://github.com/nmrs-nigeria/openmrs-module-nigeriaemr/issues/128#issue-682552114
I can't see the WHOClinicalStage
error in the log you sent.
and you're seeing the stateCode error because the patient doesn't have StateCode or LGACode in their data, (Please confirm this is the case).
Please configure partner_reporting_state
and partner_reporting_lga_code
in global property, then generate another file and send to us for validation.
@henryvalentine the UNKNOWN is showing because you've not configured
partner_reporting_state
andpartner_reporting_lga_code
in global propertysee details here #128 (comment)
I can't see the
WHOClinicalStage
error in the log you sent.and you're seeing the stateCode error because the patient doesn't have StateCode or LGACode in their data, (Please confirm this is the case).
Please configure
partner_reporting_state
andpartner_reporting_lga_code
in global property, then generate another file and send to us for validation.
@henryvalentine also ensure the patients you want to extract for have LGA and state select you can do that here
https://nmrsdev.mgic-nigeria.org/openmrs/admin/patients/patient.form?patientId=62
@henryvalentine the Letter I should be in that list, kindly confirm if you have the correct concept set for WHOClinicalStage
@wonderfulguru1 If I have to go to all the patients' record to ensure this is done before files can be generated, then I think this will pose some challenges as a lot of patients are likely not going to have these already. I was thinking it is about setting Partner Reporting Lga Code and Partner Reporting State from the 'Settings' page.
See the attached log file for the WHOClinicalStage flags
nmrs_1.6_extraction_flags_WHO_Stage.txt
@wandechris see the attached image
@henryvalentine you don't have to populate to generate the files, the error you're seeing in the logs are warnings
@henryvalentine please send the generated file and the team will see why there's a mapping issue on your end
@wandechris I know they are warnings. I think the place @wonderfulguru1 pointed to for State and LGACode to be set should be in the '.../openmrs/admin/maintenance/settings.list' page instead as the 'UNKNOWN' is now replaced with the LGA value I provided on the Settings page
Also, @wandechris you might consider telling some other persons to test this in debug mode using maven's mvn openmrs-sdk:run -DserverId=openmrs -Dfork=false -e on the command line interface. If they use this omod with their NMRS instance directly in tomcat, they are likely not going to see the warnings produced during generation of the patient data files. I will email you the data files generated before and after modifying the LGACode on the settings page
@wonderfulguru1 If I have to go to all the patients' record to ensure this is done before files can be generated, then I think this will pose some challenges as a lot of patients are likely not going to have these already. I was thinking it is about setting Partner Reporting Lga Code and Partner Reporting State from the 'Settings' page.
See the attached log file for the WHOClinicalStage flags
nmrs_1.6_extraction_flags_WHO_Stage.txt
@wandechris see the attached image
@henryvalentine thanks so much for these feedbacks there are two scenarios here
@henryvalentine you can find the list of LGA code in this doc https://github.com/nmrs-nigeria/files_and_resources/blob/master/NDR%20Data%20Dictionary%20Workbook.xlsx
partner_reporting_state
- the name of the state
partner_reporting_lga_code
- the code of the LGA from the sheet
@wandechris Received. Thanks
@wonderfulguru1 I think we are good with updating/setting the partner_reporting_state and partner_reporting_lga_code.
The name of the data file is now something like "Kaduna397_uq98eMvdS5R210820.zip". Where the '397' is the LGA code. Should you consider adding an extra underscore '' between the State name and LGA code like "Kaduna_397_uq98eMvdS5R_210820.zip"?
Well-done guys for the good job
@wonderfulguru1 I think we are good with updating/setting the partner_reporting_state and partner_reporting_lga_code.
The name of the data file is now something like "Kaduna397_uq98eMvdS5R210820.zip". Where the '397' is the LGA code. Should you consider adding an extra underscore '' between the State name and LGA code like "Kaduna_397_uq98eMvdS5R_210820.zip"?
Well-done guys for the good job
@henryvalentine thanks for the rigorious tests i will discuse your concerns on this matter with @caniekwe
@wonderfulguru1 I believe @henryvalentine request is valid. You can include it in your to do list. If the original issues reported in this thread has been resolved, then I suggest we close this.
@caniekwe thank you for the comments. Do you suggest we change to this Kaduna_397_uq98eMvdS5R_210820.zip.
@ahmedsylvanus suggested we have this way Kaduna397_uq98eMvdS5R_210820.zip
@wonderfulguru1 The former will be my preference. But I will recommend we check the NDR implementation guide and follow the guidance provided there.
@wonderfulguru1 and team well-done for the good job. For the latest version 1.6 omod released, I observed that once a patient is female, all the EDDAndPMTCTLink statuses have Breast Feeding (BF) as their values. When the patient is pregnant, it is marked as 'P' which is correct. Is BF now the default value for EDDAndPMTCTLink ?
@wonderfulguru1 and team well-done for the good job. For the latest version 1.6 omod released, I observed that once a patient is female, all the EDDAndPMTCTLink statuses have Breast Feeding (BF) as their values. When the patient is pregnant, it is marked as 'P' which is correct. Is BF now the default value for EDDAndPMTCTLink ?
@wonderfulguru1 please this is a gentle reminder on this observation
@henryvalentine we will test this further and ensure it's fixed in the next update
@wonderfulguru1 kudos for the good job and I can say we are getting there gradually.
I still have the following observations and concerns:
Though the LGACode and State Code are included as the names of the zip archive and also in each XML file, the message below is still persisting "MESSAGE: cvc-complex-type.2.4.a: Invalid content was found starting with element 'StateCode'. One of '{WardVillage, Town, LGACode}' is expected"
Also, if the LGACode is calculated from the DATIM code, it will be fine if the textboxes for it and the operating state are removed from the settings page
The EDDAndPMTCTCLink tag now contains the appropriate pregnancy status for each clinical visit.
In version 1.4.1 omod, I do see cases where the generated XML file is completely empty: no root element. The patient has encounters and is even active. Hope this won't be the case again this time around.
The error below shows up a lot at some point especially during comprehensive extraction and slows the extraction process down significantly
Sep 19, 2020 5:51:54 PM org.openmrs.module.nigeriaemr.ndrUtils.LoggerUtils log SEVERE: null java.nio.file.NoSuchFileException: C:\Users\hvalentine\NMRS_LOGS\logs_19-09-2020.log at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) at sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:230) at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) at java.nio.file.Files.newOutputStream(Files.java:216) at java.nio.file.Files.newBufferedWriter(Files.java:2860) at org.openmrs.module.nigeriaemr.ndrUtils.LoggerUtils.log(LoggerUtils.java:105) at org.openmrs.module.nigeriaemr.ndrUtils.LoggerUtils.write(LoggerUtils.java:122) at org.openmrs.module.nigeriaemr.fragment.controller.NdrFragmentController.startGenerateFile(NdrFragmentController.java:169) at org.openmrs.module.nigeriaemr.fragment.controller.NdrFragmentController.generateNDRFile(NdrFragmentController.java:120) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.openmrs.ui.framework.fragment.FragmentFactory.invokeFragmentAction(FragmentFactory.java:498) at org.openmrs.module.uiframework.FragmentActionController.handlePath(FragmentActionController.java:143) at org.openmrs.module.uiframework.FragmentActionController.handleUrlWithDotAction(FragmentActionController.java:103) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:177) at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:446) at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:434) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:943) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:877) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:857) at javax.servlet.http.HttpServlet.service(HttpServlet.java:621) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842) at javax.servlet.http.HttpServlet.service(HttpServlet.java:728) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.openmrs.module.web.filter.ForcePasswordChangeFilter.doFilter(ForcePasswordChangeFilter.java:60) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.openmrs.web.filter.GZIPFilter.doFilterInternal(GZIPFilter.java:64) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:72) at org.openmrs.module.owa.filter.OwaFilter.doFilter(OwaFilter.java:57) at org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:70) at org.openmrs.module.web.filter.ModuleFilter.doFilter(ModuleFilter.java:54) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.openmrs.web.filter.OpenmrsFilter.doFilterInternal(OpenmrsFilter.java:108) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.orm.hibernate4.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:150) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.openmrs.web.filter.StartupFilter.doFilter(StartupFilter.java:105) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.openmrs.web.filter.StartupFilter.doFilter(StartupFilter.java:105) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.openmrs.web.filter.StartupFilter.doFilter(StartupFilter.java:105) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:492) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:165) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1195) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:654) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:319) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)
Temiye, please have a look at the message log from the Extraction module (nigeriaemr-1.6.0-SNAPSHOT.omod) as attached here.
It seems the EDDAndPMTCTLink is having issues with Breast-feeding (BF) as a value as indicated by this error line:
"cvc-enumeration-valid: Value 'BF' is not facet-valid with respect to enumeration '[P, PMTCT, NP, NK]'. It must be a value from the enumeration." nmrs_1.6.omod_errors.txt
Also, there seems to be a value mismatch when "StateCode" is specified as indicated by "cvc-complex-type.2.4.a: Invalid content was found starting with element 'StateCode'. One of '{WardVillage, Town, LGACode}' is expected."
There are also some entries that point out that 'I' is not a valid value for WHOClinicalStage as the expected value must be an integer within [1,2,3,4].
These are just from a quick scan of the log attached here.