search

RLGC-Project / RLGC

An open-source platform for applying Reinforcement Learning for Grid Control (RLGC)
Other
108 stars 30 forks source link

Mismatch between Function names in python and Java #1

Closed amar-iastate closed 4 years ago

amar-iastate commented 5 years ago

Hi, There seems to be a name mismatch between the functions defined in the "IpssPyGateway.java" file and the functions called from the python files as methods of the "ipss_app" object. A few examples

  1. ipss_app.Reset() is called in a few python files while the Java file has the method "reset"
  2. ipss_app.getEnvironmentObversations() is called in a few python files while the Java file has the method ''getEnvObversations"

After I tried to correct them manually in a few python files for running python trainIEEE39LoadSheddingAgent.py, I got the following error:

py4j.protocol.Py4JJavaError: An error occurred while calling t.initStudyCase.
: java.lang.NullPointerException
    at java.util.Hashtable.put(Unknown Source)
    at org.pnnl.gov.pss_gateway.IpssPyGateway.initObsverationSpace(IpssPyGateway.java:657)
    at org.pnnl.gov.pss_gateway.IpssPyGateway.initStudyCase(IpssPyGateway.java:228)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
    at py4j.Gateway.invoke(Gateway.java:282)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.GatewayConnection.run(GatewayConnection.java:214)
    at java.lang.Thread.run(Unknown Source)

Similarly, when I am trying to run python trainKundur2areaGenBrakingAgent.py, the following error is raised:

py4j.protocol.Py4JError: An error occurred while calling t.reset. Trace:
py4j.Py4JException: Method reset([class java.lang.Integer, class java.lang.Integer, class java.lang.Double, class java.lang.Double]) does not exist
    at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
    at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326)
    at py4j.Gateway.invoke(Gateway.java:274)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.GatewayConnection.run(GatewayConnection.java:214)
    at java.lang.Thread.run(Unknown Source)

Process finished with exit code 1

I am not sure how to debug/rectify these errors to as it seems to be deeper in Java and I am not able to debug it from python.

thuang commented 5 years ago

With the latest update https://github.com/RLGC-Project/RLGC/commit/9952d7f22251fdc76cb7610368d7ab1953396c0e, this bug should be fixed.

The environment definition are updated (v3 now) to based on the latest OpenAI Gym. Test codes are updated to show how it should be used properly.

Please update your local branch and try it out. @amar-iastate

amar-iastate commented 5 years ago

Thanks for following this up RLGC Team. This particular error is not appearing anymore. However, the trainIEEE39LoadSheddingAgent.py file does not run straight from the updated repo.

After a bit of debugging, the problem turned out to be in the RL configuration json file ( IEEE39_RL_loadShedding_3motor_2levels.json ). The variable faultBusCandidates was missing. After adding a line in the json file "faultBusCandidates": ["Bus504"], the code works. The command window running the jar file is giving the following output, which seems to suggest that it is ok. 


Initial values of the observed states:
[0.98712668745297, 0.9748013683313521, 0.9731456890757114, 1.0089329691108317, 0.9813836367392125, 0.9712347933931141, 0.9652967656365352, 1.005561904995773]
Bus, initLoad, shedLoadFraction = Bus507, 2.338, -0.2
Bus, initLoad, shedLoadFraction = Bus518, 1.6320000000000001, -0.2
Bus, initLoad, shedLoadFraction = Bus504, 5.0, -0.2
Bus, initLoad, shedLoadFraction = Bus507, 2.338, -0.2
Bus, initLoad, shedLoadFraction = Bus518, 1.6320000000000001, -0.2
Bus, initLoad, shedLoadFraction = Bus504, 5.0, -0.2
Bus, initLoad, shedLoadFraction = Bus504, 5.0, -0.2
Bus, initLoad, shedLoadFraction = Bus507, 2.338, -0.2
Bus, initLoad, shedLoadFraction = Bus518, 1.6320000000000001, -0.2
Bus, initLoad, shedLoadFraction = Bus504, 5.0, -0.2
Bus, initLoad, shedLoadFraction = Bus507, 2.338, -0.2
Bus, initLoad, shedLoadFraction = Bus507, 2.338, -0.2
Bus, initLoad, shedLoadFraction = Bus518, 1.6320000000000001, -0.2
Jul 28, 2019 2:54:06 PM org.pnnl.gov.rl.action.LoadChangeActionProcessor applyAction
SEVERE: change fraction for bus =Bus507 is large than remaining fraction, it has been changed to : -5.551115123125783E-17
Bus, initLoad, shedLoadFraction = Bus504, 5.0, -0.2
Bus, initLoad, shedLoadFraction = Bus507, 2.338, -5.551115123125783E-17
Bus, initLoad, shedLoadFraction = Bus518, 1.6320000000000001, -0.2
Jul 28, 2019 2:54:06 PM org.pnnl.gov.rl.action.LoadChangeActionProcessor applyAction
SEVERE: change fraction for bus =Bus504 is large than remaining fraction, it has been changed to : -5.551115123125783E-17
Jul 28, 2019 2:54:06 PM org.pnnl.gov.rl.action.LoadChangeActionProcessor applyAction
SEVERE: change fraction for bus =Bus518 is large than remaining fraction, it has been changed to : -5.551115123125783E-17
Bus, initLoad, shedLoadFraction = Bus504, 5.0, -5.551115123125783E-17
Bus, initLoad, shedLoadFraction = Bus518, 1.6320000000000001, -5.551115123125783E-17
Jul 28, 2019 2:54:07 PM org.pnnl.gov.rl.action.LoadChangeActionProcessor applyAction
SEVERE: change fraction for bus =Bus504 is large than remaining fraction, it has been changed to : 0.0
Jul 28, 2019 2:54:07 PM org.pnnl.gov.rl.action.LoadChangeActionProcessor applyAction
SEVERE: change fraction for bus =Bus507 is large than remaining fraction, it has been changed to : 0.0
Jul 28, 2019 2:54:07 PM org.pnnl.gov.rl.action.LoadChangeActionProcessor applyAction
SEVERE: change fraction for bus =Bus504 is large than remaining fraction, it has been changed to : 0.0
Jul 28, 2019 2:54:07 PM org.pnnl.gov.rl.action.LoadChangeActionProcessor applyAction
SEVERE: change fraction for bus =Bus507 is large than remaining fraction, it has been changed to : 0.0
Jul 28, 2019 2:54:07 PM org.pnnl.gov.rl.action.LoadChangeActionProcessor applyAction
SEVERE: change fraction for bus =Bus507 is large than remaining fraction, it has been changed to : 0.0
Jul 28, 2019 2:54:07 PM org.pnnl.gov.rl.action.LoadChangeActionProcessor applyAction
SEVERE: change fraction for bus =Bus518 is large than remaining fraction, it has been changed to : 0.0
Jul 28, 2019 2:54:07 PM org.pnnl.gov.rl.action.LoadChangeActionProcessor applyAction
SEVERE: change fraction for bus =Bus504 is large than remaining fraction, it has been changed to : 0.0
Jul 28, 2019 2:54:09 PM org.pnnl.gov.rl.action.LoadChangeActionProcessor applyAction
SEVERE: change fraction for bus =Bus507 is large than remaining fraction, it has been changed to : 0.0

It would be nice if the various entries in the dynamics config and the RL config jsons file are explained briefly in a document. It would make it easy to understand the various options.

Thanks again for taking the time to rectify this.