Import Legacy complete, but no forward progress

[missyschoenbaum]

When I was testing #879, I imported a legacy scenario. The import went through the process and seemed to complete as expected. It leaves me on the Scenario description screen. I was able to edit and add a Disease name on the Disease Screen. However, when I attempt to validate, nothing happens. I closed and reopened, and still nothing happened. Here's the cmd window before I restarted.

[missyschoenbaum]

Here's cmd after I restarted, slightly different message

[BryanHurst]

The command lines output looks like everything is in good order. I'll see if I can replicate on this end.

[missyschoenbaum]

Maybe all that it is missing is pulling up the validation screen. I switched over to Sample Scenario and it is g doing the same thing.

[BryanHurst]

To attempt to replicate this, I created a new "Import NAADSM Legacy Scenario".
I selected SampleSpace_XML_para.xml and SampleSpace_XML_pop.xml for the import.
From there, I did three tests (re-importing between each):

1. Edit disease name, apply, open another tab, validate.
2. Edit disease name, apply, validate.
3. Validate

In each case after about 10 seconds I received this screen after clicking Validate:

Is there another step I can follow to get this to replicate?

[missyschoenbaum]

@BryanHurst not sure, right now I am locked out by #907. I'm glad you are not having same problem as me. What might I have done to get myself into a bad position? I think originally it was updating the unit_id.

[BryanHurst]

Since #907 is blocking you, I pushed an un-versioned compile with a potential fix for that issue. You will need to run the NPU manually if you can't open the application.

[missyschoenbaum]

Let me exercise through this again and see what happens.

[missyschoenbaum]

OK, this time I made it through as expect. Let me load Texas up just to confirm a bigger files works as expected.

[missyschoenbaum]

Texas seems to be the hangup. Try this with Texas, which is in the Github under PopParms. It takes 10 minutes or so to load. I am letting mine sit for a few minutes to see if it is still processing something; replicate this, I created a new "Import NAADSM Legacy Scenario". I selected Texas_nodepop.xml and Texa_pop_Y.xml for the import.

It did import, and I attempted to add Disease, name, then hit validate.

[missyschoenbaum]

I was able to hit the Save button (upper right), and it saved. But, my change to disease, name has still not applied, I can't change to a different screen, and Validate gives me this image:

[BryanHurst]

Okay, I am seeing this issue with the larger scenario.
It looks like it is working on something in the background still, I'll let it go for a while and see if it completes.

There might be a new validation that we introduced that is taking a long time on larger data sets. I'll have to profile it.

[BryanHurst]

Okay, I've narrowed it down to a new validation that ensures unit_id is unique.
This check is taking 10+ minutes.

[missyschoenbaum]

I was able to change between scenarios, but it took a long time. I did have a message in CMD that it might take a long time. Could we validate using a group by, count > 1 instead? Here's what that query could look like.
Select unit_id, count(unit_id) from ScenarioCreator_unit Group by unit_id Having Count(unit_id) > 1

[missyschoenbaum]

Or if we need to skip the validation for now, that would also be a work around. Taking 45 seconds to change between screens.

[BryanHurst]

Okay, brought this validation back down to sub second.

[missyschoenbaum]

Will test

[missyschoenbaum]

Looks good! Will continue with other testing.