cvrStatic testing issues

[cshjin]

1. data validation In cvrStatic.html the data validation using regular expression is [\w\s]+, which caused error for float value input. probably need to change to [\w\s.]+ or other forms of valid float value input
2. cancel cvr modeling If some error caused a pause of cvr modeling, it cannot be cancelled by click cancel button, which cannot remove those files associated in the file system.

3. output error caused by feeder data Some specific data needed for cvr testing, I followed the _tests function in cvrStatic.py, using ABEC Columbia feeder, and set parameters in the _tests function, and

   Process Process-1:
   Traceback (most recent call last):
   File "C:\Program Files\Anaconda\lib\multiprocessing\process.py", line 258, in
   _bootstrap
   self.run()
   File "C:\Program Files\Anaconda\lib\multiprocessing\process.py", line 114, in
   run
   self._target(*self._args, **self._kwargs)
   File "C:\Users\jxh10\Documents\GitHub\omf\models\cvrStatic.py", line 325, in r
   unForeground
   p = output['Zregulator.csv']['power_in.real'][0]
   KeyError: 'Zregulator.csv'

   Probably, it is because of data in ABEC Columbia feeder. Running the model from gridlabd has the error like main.glm(19743): '0' is not a valid longitude\n.

4. unicode operation
   Pick some value for loads(peak 100, aver 50) and prices(all to be 1) for cvr, and run that model. It could have the error like the following (running it from Srivats PC:):

Process Process-1:
Traceback (most recent call last):
  File "C:\Anaconda\lib\multiprocessing\process.py", line 258, in _bootstrap
    self.run()
  File "C:\Anaconda\lib\multiprocessing\process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\sxs15\Documents\GitHub\omf\models\cvrStatic.py", line 388, in r
unForeground
    row['energyReductionDollars'] = row['energyReduction'] * (rates['wholesaleEn
ergyCostPerKwh'] - rates['retailEnergyCostPerKwh'])
TypeError: unsupported operand type(s) for -: 'unicode' and 'unicode'

[dpinney]

What does "unicode operation" mean?

[cshjin]

It should be "operand", probably it is because of row['energyReductionDollars'] = row['energyReduction'] * (rates['wholesaleEnergyCostPerKwh'] - rates['retailEnergyCostPerKwh']) is in unicode and unsupport - and * operands.

[cshjin]

Q1 was fixed by commit c919e7902b890695097206187e0208598727776d. And Q4 was fixed by commit 1979124b0d36e769cb5f022b5f8399932e5a0bcc.

The reason is the "rates" data from allInputData.json file is in unicode format, need to be converted to float, just as other attributes such as a "_Avg" and "_Peak". And for _test() function, since it is already in float data format, there won't be any errors.

[cshjin]

Now the model can be run from the web interface. Note please choose a proper feeder. Q3 was fixed automatically.

[cshjin]

For Q2: a proper model workflow should be considered. When a model is chosen to run, create a "PID.txt" file, and then prepare data for specific model. The problem is if there is an error before running gridlabd, which will leave "PID.txt" empty and cannot cancel the model.

[srivats0603]

One recommendation, after discussions with Hongwei : If we can create a fake PID everytime the system is in some error (when an exception occurs). We can read it when cancel process is initiated and take appropriate actions

[dpinney]

I think the fake PID might be a good idea.We also need to think about when to catch exceptions in model runs.

See below for my thoughts on the workflow steps we need to handle.

[cshjin]

Consider the following scenario: we already have some models finished, i.e. gridlabd, and we clicked "Re-Run Model", before it finished, click "cancel". Now we have the problem: the cancel function will kill the subprocess of gridlabd, but leave the runForeground function running in its parents process. At the end of the function, it will attempt to remove the "PID.txt" file, however, this file is already removed by cancel function. In this situation we will have the error like the following:

Process Process-1:
Traceback (most recent call last):
  File "C:\Program Files\Anaconda\lib\multiprocessing\process.py", line 258, in _bootstrap
    self.run()
  File "C:\Program Files\Anaconda\lib\multiprocessing\process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\jxh10\Documents\GitHub\omf\models\gridlabSingle.py", line 234, in runForeground
    os.remove(pJoin(modelDir,"PID.txt"))
WindowsError: [Error 2] The system cannot find the file specified: u'C:\\Users\\jxh10\\Documents\\GitHub\\omf\\data\\Model\\admin\\Automated Testing\\PID.txt'

The unspecified file is in the runForeground function, which has been already removed by cancel function.

There are two scenarios:

1. There is no error if we run the model in the general case, which runForeground function do as expected. But we want to cancel the model.
2. There is error when converting data for gridlabd, which runForeground will be stuck at some point before running gridlabd. And we want to cancel the model.

I suggest to kill(stop) the parent process and kill the subprocess if already created in the cancel function.

[dpinney]

We should probably re-architect the model-running code. Let's discuss.

[cshjin]

Cancel function fixed by commits 0c16acc6d1dedf4613938773dda9da1f57a45ea7 and a867a238e070faf9d275da91ee38d51da0aeaba0. atexit feature could be considered for killing process tree in the future. Closed for now.