Notify user of percentage complete of Gridlab model.

[dpinney]

Instead of telling the user Gridlab is running, we should display percentage done.

Note that reading the stdout.txt from GridlabD doesn't interrupt the process. They've also got some process control stuff.

QUESTIONS

• What model is this for? gridlabMulti, solarEngineering, cvrDynamic.

[dpinney]

Back to Mannan.

We're looking for a new function. It shouldbe in omf.models.gridlabMulti. Mockup:

def checkStatus(pathToModelFolder):
  #implementation here.
  return floatPercentageStatus

[mannanj]

Problem: The stdout.txt file doesn't exist until a majority of the processing time has passed. On lines 74-78:

        # RUN GRIDLABD IN FILESYSTEM (EXPENSIVE!)
        with open(pJoin(workDir,'stdout.txt'),'w') as stdout, open(pJoin(workDir,'stderr.txt'),'w') as stderr, open(pJoin(workDir,'PID.txt'),'w') as pidFile:
            # MAYBEFIX: turn standerr WARNINGS back on once we figure out how to supress the 500MB of lines gridlabd wants to write...
            proc = subprocess.Popen([binaryName,'-w', glmName], cwd=workDir, stdout=stdout, stderr=stderr)
            pidFile.write(str(proc.pid))

This takes a majority of the model execution time, and the file doesn't exist until this is done executing. So reading the stdout.txt isn't how we'll want to estimate time. Is there another way we can do so?

Solution: ??

[dpinney]

Solution = watch stderr.txt instead of stdout.txt. GLD writes progress to that instead of stdout. Their approach is uncommon.

[mannanj]

Thanks for the suggestion. I tried it, to no luck. stderr doesn't return anything but a 1 liner with a time, nothing useful.

So when reading its contents every second:

• Time=the time at which the script is reading the fil
• line after stderr= is the contents of the file

Time= 1440000424.32
stderr=
Processing 2012-01-03 12:00:00 PST...

Time= 1440000425.32
stderr=
Processing 2012-01-03 16:00:00 PST...

Time= 1440000426.33
stderr=
Processing 2012-01-03 18:00:00 PST...

Time= 1440000427.33
stderr=
Processing 2012-01-03 19:00:00 PST...

Time= 1440000428.33
stderr=
Processing 2012-01-03 21:00:00 PST...

Time= 1440000429.33
stderr=
Processing 2012-01-03 23:00:00 PST...

Time= 1440000430.33
stderr=
Processing 2012-01-04 01:00:00 PST...

Time= 1440000431.33
stderr=
Processing 2012-01-04 02:00:00 PST...

Time= 1440000432.33
stderr=
Processing 2012-01-04 03:00:00 PST...

Time= 1440000433.33
stderr=
Processing 2012-01-04 03:00:00 PST...

[dpinney]

You have to parse the time.

On Aug 19, 2015, at 12:09 PM, mannanj notifications@github.com wrote:

Thanks for the suggestion. I tried it, to no luck. stderr doesn't return anything but a 1 liner with a time, nothing useful:

Time= 1440000424.32 stderr= Processing 2012-01-03 12:00:00 PST... Time= 1440000425.32 stderr= Processing 2012-01-03 16:00:00 PST... Time= 1440000426.33 stderr= Processing 2012-01-03 18:00:00 PST... Time= 1440000427.33 stderr= Processing 2012-01-03 19:00:00 PST... Time= 1440000428.33 stderr= Processing 2012-01-03 21:00:00 PST... Time= 1440000429.33 stderr= Processing 2012-01-03 23:00:00 PST... Time= 1440000430.33 stderr= Processing 2012-01-04 01:00:00 PST... Time= 1440000431.33 stderr= Processing 2012-01-04 02:00:00 PST... Time= 1440000432.33 stderr= Processing 2012-01-04 03:00:00 PST... Time= 1440000433.33 stderr= Processing 2012-01-04 03:00:00 PST... — Reply to this email directly or view it on GitHub.

Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copy, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.

[mannanj]

Got it. Misunderstanding on my part.

[dpinney]

BTW, there’s a parser already built in to python’s datetime module. I think it’s called strftime (or strptime?).

On Aug 19, 2015, at 12:18 PM, mannanj notifications@github.com wrote:

Got it. Misunderstanding on my part.

— Reply to this email directly or view it on GitHub.

Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copy, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.

[mannanj]

The strptime is the parser you were thinking of. I have a demo to show you, it estimates % complete for a feeder at a time since it seems gridlabD runs completely for one feeder before creating its stdErr.txt log and then runs for the next, making it impossible to get an average estimate for all feeders. I can only get one at a time. So there may be a design change in order so that we run gridlabD in parallel for all feeders at the same time. This is slightly complex and we will probably have to have a strategy talk.

[dpinney]

Great demo.

Move the functionality to a function in /omf/solvers/gridlabd/init.py defined as:

def checkStatus(pathToModelFolder):
   #implementation here.
   return floatPercentageStatus

And then we're good to close.

[mannanj]

Commit ee9bd057ec7842623d7a639414a569ff4c95a4ac

[mannanj]

With this, all my current issues closed! What's next!?