RuntimeError: table 'taz_control_data' never checkpointed. Closing remaining open files:output\pipeline.h5...done

[issa-jtmt]

Dear All,

I am trying to run the popsim, and it seems all is working till one I get this message after the run_pop reads the files:

RuntimeError: table 'taz_control_data' never checkpointed. Closing remaining open files:output\pipeline.h5...done

Any help?

Regards,

Issa

[bettinardi]

That sounds like it is failing pretty early. It seems like the taz data control is not established correctly. How may minutes or seconds would you say that it ran before it got to this message?

[toliwaga]

the log file would be helpful

On Mon, Mar 16, 2020 at 12:06 PM Alex Bettinardi notifications@github.com wrote:

That sounds like it is failing pretty early. It seems like the taz data control is not established correctly. How may minutes or seconds would you say that it ran before it got to this message?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ActivitySim/populationsim/issues/116#issuecomment-599621036, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAOMQAICTXUVCIVBZYXIS3RHZFBVANCNFSM4LLLCKYA .

[issa-jtmt]

activitysim.log Please note that it takes me less than 30 seconds to have this message. I could not understand the problem there.

Any help would do.

thanks.

Issa

[bettinardi]

I'll leave it to Jeff to provide a more helpful response, but it looks like it read the control_taz.csv file fine, so I'm guessing there must be an inconsistency in the settings.ymal or controls.csv. Either that or it could be a data type in the control_taz file. In other issues we have found that PopulationSim doesn't do well with decimals in the control input files. You might ensure that everything is a round integer in your taz file and all your control files.

[issa-jtmt]

By the way, I am very desperate and disappointed by this error message, which does not have a clear meaning. I tried everything, still to no avail. Any help would be really appreciated.

I attached also the yaml file (in text here).

Issa ‏‏settings.txt

[bettinardi]

I would echo that PopulationSim would benefit from better error reporting in general, and one thing that would help that would be some standard checks up front for known issues so that those issues can be flagged and a more helpful error message can be provided. I see that taz is your lowest level of geography. That means that num_hh specifically needs to be called out and pointed to the correct field in your controls.csv. Again, I would check for and round off any decimals. Ensure all control files are integer only...

[issa-jtmt]

Well, I do not have any kind of decimals in the control file. This is not the issue here. Al the data types there are numerical. No letters or anything. I will post also controls and the geo_cross_walk files (moved them to text).

Thanks.

‏‏geo_cross_walk.txt ‏‏control_taz.txt

[bettinardi]

The other file that would need to be shared is the controls.csv in the config folder

[issa-jtmt]

‏‏controls.txt Attached.

Sorry for the mess, but I need to run this ASAP.

[bettinardi]

I don't know how the seed was processed. It looks like in your control file you are referencing a "hhnum" filed in your seed. I know we have had issues when we try and mess with the expression for "num_hh". You have this - households.hhnum > 0

We have this: (households.WGTP > 0) & (households.WGTP < np.inf)

I'll let you know if I think of anything else, but that's currently my best thought. Maybe Jeff or others can review and provide better instruction.

I would add that after you get this working, you are going to have conflicting controls. All your size controls are going to fight directly with your duplicate population control - population,taz,persons,1000000,pop,persons.hhnum >= 1

I don't think that population control will work as intended. I suggest removing that one, and letting the population by age controls do the work of that total population control.

Related to this, you might consider doing tests where you start with a smaller set of controls and then add controls and see how those impact the results (once you get it working).

[binnympaul]

Here is the original runtime error

Just reiterating what I sent in the email for better documentation.

PopulationSim expects you to be consistent with geography names, it is even case sensitive. In your settings file, where you list the geographies, “taz” is in lower case but while defining the taz control table, you use uppercase.

• tablename: TAZ_control_data filename : control_taz.csv

As per the error message it cannot find "taz_control_data" since "taz" was listed as a geography. However, you defined "TAZ_control_data" as the control table for that geography. Defining that table by the name "taz_control_table" should probably resolve this error. Please try using consistent geography names everywhere. Please update this thread if that fixes it.

[binnympaul]

@issa-jtmt : Since this has been resolved can we please close this issue? I will open another issue with an enhancement/documentation tag

[issa-jtmt]

Yes, all is fine.

From: Binny Mathew Paul notifications@github.com Sent: Tuesday, March 17, 2020 3:39 PM To: ActivitySim/populationsim populationsim@noreply.github.com Cc: Issa Zananiri Issa@jtmt.gov.il; Mention mention@noreply.github.com Subject: Re: [ActivitySim/populationsim] RuntimeError: table 'taz_control_data' never checkpointed. Closing remaining open files:output\pipeline.h5...done (#116)

@issa-jtmthttps://github.com/issa-jtmt : Since this has been resolved can we please close this issue? I will open another issue with an enhancement/documentation tag

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ActivitySim/populationsim/issues/116#issuecomment-600076646, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AO2WFQA4GWL7I3NAHYO576TRH54Q7ANCNFSM4LLLCKYA.