2 questions

[zhangdahua1]

Sorry, I have two new problems through learning your program.

1. How do you define offline and online.
2. Why does (offline training online -------- training training -------- converting data) need to cycle three times after running the main. py I'm looking forward to your reply.

[yg-smile]

Dear @zhangdahua1 , Thank you for asking..

1. the purpose of offline phase is to 'pre-train' the RL algorithm on some existing historical operational data; and the online phase is to let the algorithm interacting with the environment while learning.. The exact length for the two phases are left out for the users to experiment. They can be configured through main.py->config['offline_split']/config['online_split'].
2. Are you referring to line 54?

   This is to run experiments multiple times, each with a different random seed. Many RL algorithms' performance is sensitive to the random initialization of neural network, exploratory actions etc. In order to assess the consistency and variability of the performance it is almost always desirable to run experiment with random seed controlled and other setup fixed...

Please let me know if that answered your questions..

[zhangdahua1]

Can I understand that offline is a training phase. And online is a testing phase

[yg-smile]

Yes, kind of like that .. But unlike supervised learning, training RL on offline data alone cannot always produce good results.. Therefore continued learning into the 'testing' (online) phase is needed ..

This is especially true for this particular dataset. The VVC device status (action) are generated by the legacy control approach. As a result, the "actions" are highly correlated with the "states". This makes offline training very difficult..

[zhangdahua1]

I understand your answer and thank you for your answer. If I want to view the actions of ltc tap and cap status, can I view them through one parameter in your program . And can I check the voltage change of one node?

[yg-smile]

Sorry, the code is not very well written. It did provide an convenient API.. The voltage can be checked by the info['v'] field and the ltc tap, cap status by info['action'] field. The ltc taps are placed first. The info is a returned parameter of env.step().

envs/env.py:

[zhangdahua1]

It's my problem that I don't have a thorough understanding of your program. My question has been answered.Thank you very much for your reply.

[zhangdahua1]

Hello, dear @yg-smile . I have two new questions.

1. Did you calculate the power flow during the training? If so,How do you implement power flow calculation？
2. Why is there 88 columns of data in the data file of 8500 nodes, not 8500 columns or more. I'm looking forward to your reply.

[yg-smile]

Thank you for your questions.

1. The power flow for all timestamps have been solved and saved to the data/processed folder. In the offline vvc (training), the code simply loads the data from these files. This is to save the computation time.. The power flow calculation is done by OpenDSSDirect, a Pthon package to the OpenDSS. The vvc control logic for generating the training data can be found in _envs/dssxxx where xxx is feeder id. Please let me know if you have additional questions.
2. Thank you for your question. First, not all nodes of the 8500 feeder have loads (they could be a connection point). In practice, only nodes with loads will have meters installed. There are only about 1000+ load nodes.. Second, because 1000+ is still a large number, we aggregate every few meter's data to reduce the dimensionality. If you would like to access all the metered values, please refer to the envs/env.py/def step line 163 onward to see how to get all data from power flows..