search

abtassociates / eva

Eva is a HUD application to aid HMIS Leads with data analysis. It is an open-source project intended for local use by HMIS Administrators in Continuums of Care (CoCs) around the U.S. and its territories.
GNU Affero General Public License v3.0
14 stars 4 forks source link

Eva crashed 11/07/2023 at 12:05pm EST #423

Closed TanyaPieterse closed 10 months ago

TanyaPieterse commented 11 months ago

Good morning, There was a few structural issues with a file from one of our customers, I corrected the errors and tried to re-upload but EAV crashed with the message: "Eva has crashed. Please submit an issue on GitHub..."

Thanks Tanya Pieterse Bell Data Systems, inc

kiadso commented 11 months ago

Hi Tanya, Thanks for reporting this! Can you tell me a little bit about what the structural issues were that were corrected? Maybe this will help us pin down the issue. Thank you- Genelle

TanyaPieterse commented 11 months ago

Hi Genelle,

Absolutely! Below were the original structural issues before correcting the data.

It was a quick fix, there were a handful of Service records where the UserID was a blank; and on one project they did not enter the Operating Start Date, HMIS Participation Status Start Date or CE Participation Status Start Date. It seemed to crash when the EVA Tool was building the data quality section.

Thanks,

Tanya Pieterse Bell Data Systems, Inc.

kiadso commented 11 months ago

Ok great-

You mention a null OperatingStartDate- is it possible that you fixed this by excluding records like this from Project.csv and then didn't also exclude any related references in other tables? Like if the project had a null in a non-nullable data element, then the .csv was corrected to exclude those, any enrollments or ProjectCoC records or anything else really that refers to that ProjectID would also have to be excluded.

Eva shouldn't crash because of this, but just thinking out loud here..

Thanks-

kiadso commented 10 months ago

Hi @TanyaPieterse,

We just updated Eva with a number of hotfixes. Can you try again?

Thanks- Genelle

TanyaPieterse commented 10 months ago

Hi Genelle,

I just tried it again (1:34pm EST), still crashing the EVA Tool. I've tried refreshing the browser, and tried Google Chrome and Edge. Please let me know if you need anything else to troubleshoot.

Thanks

alex-silverman commented 10 months ago

Hi Tanya,

It's possible that these crashes coincided with some server updates we were making, but to be sure, I've created a temporary test app for you to try, and I've added some debugging tools to help us diagnose the issue, if there still is one. If possible, please try uploading your file to this app:

https://hmis-stage.abtsites.com/eva-test/

Let me know when you've had a chance to test and I will look at the logs. At the very least, I should get a much better sense of where things are going wrong!

Alex

TanyaPieterse commented 10 months ago

Hi Alex,

I uploaded the exact same file to your test link above and it just crashed that one as well on the same spot progress bar section where it is accessing the Data Quality.

Thanks,

alex-silverman commented 10 months ago

okay, thanks for testing.

This is really puzzling. Based on the server logs, it seems like you are actually getting past the Data Quality step, into the PDDE Checker step, and even past that to a successful upload. If the app was crashing during the Data Quality step, I would expect NOT to see the logging statements that I added in the PDDE Checker step. And yet, there you are in the log below!

Can you confirm what browser and OS you're using? And can you screenshot the crashing message and attach it here?

SessionToken Datestamp Contact Msg
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network Running get export
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network Running dates
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network Running file structure analysis
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network duplicate combinations found of: PersonalID
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network duplicate combinations found of: EnrollmentID
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network duplicate combinations found of: HouseholdID
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network Running initial data prep
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network duplicate combinations found of: ProjectID
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network Running Data Quality
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network Running PDDE checker
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing subpopNotTotal
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing operating_end_missing
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing missing_CoC_Info
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing missing_CoC_Geography
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing missing_CoC_Address
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing missing_inventory_record
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing inventory_start_precedes_operating_start
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing operating_end_precedes_inventory_end
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing rrh_no_subtype
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing vsp_projects
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing vsps_in_hmis
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing projects_w_beds
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing res_projects_no_clients
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing rrh_so_w_inventory
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing overlapping_ce_participation
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network doing overlapping_hmis_participation
a1dc7e3eaaeeea28cd84b3be2593d3e4 11/10/2023 13:21 Bell Data Systems Inc. - Client Services Network successful upload
TanyaPieterse commented 10 months ago

Hi, I am using a Windows 11 Home pc, and the browser is Google Chrome

I just uploaded a different file from another system to your test link and it worked perfectly: image

Then I tried the other file again and watched the progress bar more closely, it went all the way to "Done!" then showed this crashed screen: image

Thanks for troubleshooting this!!

alex-silverman commented 10 months ago

Hi Tanya,

This is really helpful. This confirms that the crash happens after the file is successfully processed. I have a couple suspicions, so I added some logging statements in those places. If you have a chance to re-upload to the test app, that might help us narrow down where this is happening.

TanyaPieterse commented 10 months ago

Hi, Just uploaded it and crashed it again at 2:50pm EST. Thanks,

alex-silverman commented 10 months ago

For the moment, I'm not intending to fix anything yet - just trying to pinpoint where the code is failing, since we can't replicate. But we're getting closer! The code is failing for you after it processes some of the File Structure Analysis stuff that should appear on the page when an upload finishes. I just tried adding a couple more logging statements. If it crashes after that, I'll be pretty stumped.

If you'd like to try once more, perhaps while on a call, we can continue narrowing down the issue.

TanyaPieterse commented 10 months ago

Hi Alex, The issue is: Julia’s Thanks!

alex-silverman commented 10 months ago

Hi Tanya,

I just tried modifying a ProjectName in our internal test file to see if I could replicate the issue, but I could not. I'll try a few more things in case the character (Right-Single Quotation Mark) is getting translated somehow in the paste or by my Windows 10 OS (vs. your Windows 11), but I'm wondering if there's something else about this project that is causing the crash.

Alex

TanyaPieterse commented 10 months ago

Hi Alex, I've made it so that the problem-project will be included in the Export again, then wrapped this around the 2 fields where it generates the Project.csv file replace(ProjectName,'’','') as ProjectName, replace(ProjectCommonName,'’','') as ProjectCommonName,

Then reran the Export and it uploaded perfectly to your EVA Test Site - it is definitely this apostrophe causing the issue, we just have to catch it in both fields. Thank you AGAIN so much for all your troubleshooting today!!

Thanks,

alex-silverman commented 10 months ago

According to this HMIS CSV Specs, right single- and right double- quotation marks are not permitted in string fields. We will update the GitHub README to note this, and will likely update Eva to reject exports that include these.