eclipse-volttron / volttron-openadr-ven

Apache License 2.0
5 stars 5 forks source link

Try to run open ADR agent #14

Closed bbartling closed 2 years ago

bbartling commented 2 years ago

Okay so trying this out on volttron 8.1.1 on an Ubuntu instance.

activating poetry venv and installing the dependencies: (env) $ poetry install

And running the openadr agent through the poetry venv:

(.venv) bbartling@vm-bbartling2:~/openadr-ven-working$ AGENT_CONFIG='/home/bbartling/openadr-ven-working/volttron_openadr_ven/config.json' poetry run python /home/bbartling/openadr-ven-working/volttron_openadr_ven/agent.py

This will pop up in the console:

2021-11-12 06:46:49,234 root DEBUG: Creating ZMQ Core None
2021-11-12 06:46:49,235 volttron.client.vip.agent.core DEBUG: address: ipc://@/home/bbartling/.volttron/run/vip.socket
2021-11-12 06:46:49,235 volttron.client.vip.agent.core DEBUG: identity: b138eb83-32df-47cd-b69c-919ecc5d3ec8
2021-11-12 06:46:49,236 volttron.client.vip.agent.core DEBUG: agent_uuid: None
2021-11-12 06:46:49,236 volttron.client.vip.agent.core DEBUG: serverkey: None
2021-11-12 06:46:49,237 volttron.utils.file_access DEBUG: missing file /home/bbartling/.volttron/keystores/b138eb83-32df-47cd-b69c-919ecc5d3ec8/keystore.json
2021-11-12 06:46:49,237 volttron.utils.file_access INFO: creating file /home/bbartling/.volttron/keystores/b138eb83-32df-47cd-b69c-919ecc5d3ec8/keystore.json
2021-11-12 06:46:49,238 volttron.utils.keystore DEBUG: calling generate from keystore
2021-11-12 06:46:49,238 volttron.client.vip.agent.core DEBUG: AGENT RUNNING on ZMQ Core b138eb83-32df-47cd-b69c-919ecc5d3ec8
2021-11-12 06:46:49,239 volttron.client.vip.agent.core DEBUG: keys: server: mjVHcEumcdJ_NQ95TzFA6olmycFzbYpxG13-qcfVrig public: 34nXaspteWW1v_Iw3mgmuegHIEqhpdSsb16GGMfpw04, secret: 8sB8Ufe5gngFeXhuxXaKyQVoY6y04cO2EaAAU4eoWOU
2021-11-12 06:46:49,240 __main__ DEBUG: Configuring agent with:
 {'ca_file': None,
 'cert': 'secret/TEST_RSA_VEN_211111203054_cert.pem',
 'debug': True,
 'disable_signature': True,
 'key': 'secret/TEST_RSA_VEN_211111203054_privkey.pem',
 'passphrase': None,
 'show_fingerprint': None,
 'ven_id': None,
 'ven_name': 'slipstream-ven4',
 'vtn_fingerprint': None,
 'vtn_url': 'https://eiss2demo.ipkeys.com/oadr2/OpenADR2/Simple/2.0b'}
2021-11-12 06:46:49,242 __main__ INFO: Creating OpenLeadrVen Client...
2021-11-12 06:46:49,242 asyncio DEBUG: Using selector: GeventSelector
2021-11-12 06:46:49,256 __main__ INFO: Configuration complete.
2021-11-12 06:46:49,257 volttron.client.vip.zmq_connection DEBUG: ZMQ connection b138eb83-32df-47cd-b69c-919ecc5d3ec8
2021-11-12 06:46:49,259 volttron.client.vip.zmq_connection DEBUG: connecting to url ipc://@/home/bbartling/.volttron/run/vip.socket?publickey=34nXaspteWW1v_Iw3mgmuegHIEqhpdSsb16GGMfpw04&secretkey=8sB8Ufe5gngFeXhuxXaKyQVoY6y04cO2EaAAU4eoWOU&serverkey=mjVHcEumcdJ_NQ95TzFA6olmycFzbYpxG13-qcfVrig
2021-11-12 06:46:49,259 volttron.client.vip.zmq_connection DEBUG: url type is <class 'str'>
2021-11-12 06:46:59,271 volttron.client.vip.agent.core ERROR: No response to hello message after 10 seconds.
2021-11-12 06:46:59,271 volttron.client.vip.agent.core ERROR: Type of message bus used zmq
2021-11-12 06:46:59,272 volttron.client.vip.agent.core ERROR: A common reason for this is a conflicting VIP IDENTITY.
2021-11-12 06:46:59,272 volttron.client.vip.agent.core ERROR: Another common reason is not having an auth entry onthe target instance.
2021-11-12 06:46:59,272 volttron.client.vip.agent.core ERROR: Shutting down agent.

In a different tmux session, running volttron the listener agent has this:

'GOOD'
2021-11-12 06:38:03,924 () volttron.platform.auth INFO: AUTH: After authenticate user id: None, b'0b391f6b-8cd2-4ff0-a78d-9f225a9738cd'
2021-11-12 06:38:03,924 () volttron.platform.auth INFO: authentication failure: userid='2999df6e-b956-4436-b2d4-039a1f3ccd73', domain='vip', address='localhost:1000:1000:1724', mechanism='CURVE', credentials=['TqdXj-NZDFLKGu7xz-WAQZ6x289RY4FUZfJ2NWZTOFs']
2021-11-12 06:38:06,887 (listeneragent-3.3 1683) __main__ INFO: Peer: pubsub, Sender: listeneragent-3.3_1:, Bus: , Topic: heartbeat/listeneragent-3.3_1, Headers: {'TimeStamp': '2021-11-12T12:38:06.881742+00:00', 'min_compatible_version': '3.0', 'max_compatible_version': ''}, Message:
'GOOD'
2021-11-12 06:38:11,883 (listeneragent-3.3 1683) __main__ INFO: Peer: pubsub, Sender: listeneragent-3.3_1:, Bus: , Topic: heartbeat/listeneragent-3.3_1, Headers: {'TimeStamp': '2021-11-12T12:38:11.879157+00:00', 'min_compatible_version': '3.0', 'max_compatible_version': ''}, Message:
'GOOD'

Any ideas to try?

bbartling commented 2 years ago

I also notice in the repo I git cloned, step 3 in the README:

(env) $ pip install volttron_utils-0.1.1-py3-none-any.whl volttron_client-0.1.2-py3-none-any.whl

These wheel files dont exist. @bonicim was there a merge that @craig8 was going to do from your repo on this agent? Am I missing this step 3 process?

craig8 commented 2 years ago

pip install volttron-client will work.

bbartling commented 2 years ago

Would you guys know how to troubleshoot this?

Could there be anyway I am not using @bonicim version of openleadr that doesnt have the disable signature flag feature?

2021-11-16 13:49:45,523 volttron.utils.keystore DEBUG: calling generate from keystore
2021-11-16 13:49:45,524 volttron.client.vip.agent.core DEBUG: AGENT RUNNING on ZMQ Core 45fe3364-a008-4041-8ae8-00c740a9efd3
2021-11-16 13:49:45,525 volttron.client.vip.agent.core DEBUG: keys: server: mjVHcEumcdJ_NQ95TzFA6olmycFzbYpxG13-qcfVrig public: U-HHXxpv46LWxjYkditRgaDX8K5_XwE6yGwrXc3iHF0, secret: UhGDB9xBYwUnuMqcpbbS7ctwOxYxMHsKP-ij-1iwPTM
2021-11-16 13:49:45,528 __main__ DEBUG: Configuring agent with:
 {'ca_file': None,
 'cert': 'secret/TEST_RSA_VEN_211111203054_cert.pem',
 'debug': True,
 'disable_signature': True,
 'key': 'secret/TEST_RSA_VEN_211111203054_privkey.pem',
 'passphrase': None,
 'show_fingerprint': None,
 'ven_id': None,
 'ven_name': 'slipstream-ven4',
 'vtn_fingerprint': None,
 'vtn_url': 'https://eiss2demo.ipkeys.com/oadr2/OpenADR2/Simple/2.0b'}
2021-11-16 13:49:45,532 __main__ INFO: Creating OpenLeadrVen Client...
Traceback (most recent call last):
  File "/home/bbartling/openadr-ven-working/volttron_openadr_ven/agent.py", line 370, in <module>
    sys.exit(main())
  File "/home/bbartling/openadr-ven-working/volttron_openadr_ven/agent.py", line 309, in main
    vip_main(ven_agent, version=__version__)
  File "/home/bbartling/volttron-openadr-ven/.venv/lib/python3.8/site-packages/volttron/utils/commands.py", line 167, in vip_main
    agent = agent_class(
  File "/home/bbartling/openadr-ven-working/volttron_openadr_ven/agent.py", line 352, in ven_agent
    return OpenADRVenAgent(
  File "/home/bbartling/openadr-ven-working/volttron_openadr_ven/agent.py", line 148, in __init__
    self.configure_agent(self.default_config)
  File "/home/bbartling/openadr-ven-working/volttron_openadr_ven/agent.py", line 158, in configure_agent
    self.ven_client = OpenADRClient(
TypeError: __init__() got an unexpected keyword argument 'disable_signature'
bonicim commented 2 years ago

I also notice in the repo I git cloned, step 3 in the README:

(env) $ pip install volttron_utils-0.1.1-py3-none-any.whl volttron_client-0.1.2-py3-none-any.whl

These wheel files dont exist. @bonicim was there a merge that @craig8 was going to do from your repo on this agent? Am I missing this step 3 process?

You can ignore that step. It is outdated. I will submit a PR soon to update the README. See https://github.com/VOLTTRON/volttron-openadr-ven/pull/15

bonicim commented 2 years ago

Would you guys know how to troubleshoot this?

Could there be anyway I am not using @bonicim version of openleadr that doesnt have the disable signature flag feature?

2021-11-16 13:49:45,523 volttron.utils.keystore DEBUG: calling generate from keystore
2021-11-16 13:49:45,524 volttron.client.vip.agent.core DEBUG: AGENT RUNNING on ZMQ Core 45fe3364-a008-4041-8ae8-00c740a9efd3
2021-11-16 13:49:45,525 volttron.client.vip.agent.core DEBUG: keys: server: mjVHcEumcdJ_NQ95TzFA6olmycFzbYpxG13-qcfVrig public: U-HHXxpv46LWxjYkditRgaDX8K5_XwE6yGwrXc3iHF0, secret: UhGDB9xBYwUnuMqcpbbS7ctwOxYxMHsKP-ij-1iwPTM
2021-11-16 13:49:45,528 __main__ DEBUG: Configuring agent with:
 {'ca_file': None,
 'cert': 'secret/TEST_RSA_VEN_211111203054_cert.pem',
 'debug': True,
 'disable_signature': True,
 'key': 'secret/TEST_RSA_VEN_211111203054_privkey.pem',
 'passphrase': None,
 'show_fingerprint': None,
 'ven_id': None,
 'ven_name': 'slipstream-ven4',
 'vtn_fingerprint': None,
 'vtn_url': 'https://eiss2demo.ipkeys.com/oadr2/OpenADR2/Simple/2.0b'}
2021-11-16 13:49:45,532 __main__ INFO: Creating OpenLeadrVen Client...
Traceback (most recent call last):
  File "/home/bbartling/openadr-ven-working/volttron_openadr_ven/agent.py", line 370, in <module>
    sys.exit(main())
  File "/home/bbartling/openadr-ven-working/volttron_openadr_ven/agent.py", line 309, in main
    vip_main(ven_agent, version=__version__)
  File "/home/bbartling/volttron-openadr-ven/.venv/lib/python3.8/site-packages/volttron/utils/commands.py", line 167, in vip_main
    agent = agent_class(
  File "/home/bbartling/openadr-ven-working/volttron_openadr_ven/agent.py", line 352, in ven_agent
    return OpenADRVenAgent(
  File "/home/bbartling/openadr-ven-working/volttron_openadr_ven/agent.py", line 148, in __init__
    self.configure_agent(self.default_config)
  File "/home/bbartling/openadr-ven-working/volttron_openadr_ven/agent.py", line 158, in configure_agent
    self.ven_client = OpenADRClient(
TypeError: __init__() got an unexpected keyword argument 'disable_signature'

To ensure that you are using the 'forked' version of OpenLEADR, you need to delete the 'openleadr' folder in your repo and then rerun the git submodule command to get the forked version of OpenLEADR. Do the following in the root level of your repo:

rm -rf openleadr
git submodule update --init --recursive

You can check that the openleadr library was installed by running poetry show.

bonicim commented 2 years ago

Okay so trying this out on volttron 8.1.1 on an Ubuntu instance.

activating poetry venv and installing the dependencies: (env) $ poetry install

And running the openadr agent through the poetry venv:

(.venv) bbartling@vm-bbartling2:~/openadr-ven-working$ AGENT_CONFIG='/home/bbartling/openadr-ven-working/volttron_openadr_ven/config.json' poetry run python /home/bbartling/openadr-ven-working/volttron_openadr_ven/agent.py

This will pop up in the console:

2021-11-12 06:46:49,234 root DEBUG: Creating ZMQ Core None
2021-11-12 06:46:49,235 volttron.client.vip.agent.core DEBUG: address: ipc://@/home/bbartling/.volttron/run/vip.socket
2021-11-12 06:46:49,235 volttron.client.vip.agent.core DEBUG: identity: b138eb83-32df-47cd-b69c-919ecc5d3ec8
2021-11-12 06:46:49,236 volttron.client.vip.agent.core DEBUG: agent_uuid: None
2021-11-12 06:46:49,236 volttron.client.vip.agent.core DEBUG: serverkey: None
2021-11-12 06:46:49,237 volttron.utils.file_access DEBUG: missing file /home/bbartling/.volttron/keystores/b138eb83-32df-47cd-b69c-919ecc5d3ec8/keystore.json
2021-11-12 06:46:49,237 volttron.utils.file_access INFO: creating file /home/bbartling/.volttron/keystores/b138eb83-32df-47cd-b69c-919ecc5d3ec8/keystore.json
2021-11-12 06:46:49,238 volttron.utils.keystore DEBUG: calling generate from keystore
2021-11-12 06:46:49,238 volttron.client.vip.agent.core DEBUG: AGENT RUNNING on ZMQ Core b138eb83-32df-47cd-b69c-919ecc5d3ec8
2021-11-12 06:46:49,239 volttron.client.vip.agent.core DEBUG: keys: server: mjVHcEumcdJ_NQ95TzFA6olmycFzbYpxG13-qcfVrig public: 34nXaspteWW1v_Iw3mgmuegHIEqhpdSsb16GGMfpw04, secret: 8sB8Ufe5gngFeXhuxXaKyQVoY6y04cO2EaAAU4eoWOU
2021-11-12 06:46:49,240 __main__ DEBUG: Configuring agent with:
 {'ca_file': None,
 'cert': 'secret/TEST_RSA_VEN_211111203054_cert.pem',
 'debug': True,
 'disable_signature': True,
 'key': 'secret/TEST_RSA_VEN_211111203054_privkey.pem',
 'passphrase': None,
 'show_fingerprint': None,
 'ven_id': None,
 'ven_name': 'slipstream-ven4',
 'vtn_fingerprint': None,
 'vtn_url': 'https://eiss2demo.ipkeys.com/oadr2/OpenADR2/Simple/2.0b'}
2021-11-12 06:46:49,242 __main__ INFO: Creating OpenLeadrVen Client...
2021-11-12 06:46:49,242 asyncio DEBUG: Using selector: GeventSelector
2021-11-12 06:46:49,256 __main__ INFO: Configuration complete.
2021-11-12 06:46:49,257 volttron.client.vip.zmq_connection DEBUG: ZMQ connection b138eb83-32df-47cd-b69c-919ecc5d3ec8
2021-11-12 06:46:49,259 volttron.client.vip.zmq_connection DEBUG: connecting to url ipc://@/home/bbartling/.volttron/run/vip.socket?publickey=34nXaspteWW1v_Iw3mgmuegHIEqhpdSsb16GGMfpw04&secretkey=8sB8Ufe5gngFeXhuxXaKyQVoY6y04cO2EaAAU4eoWOU&serverkey=mjVHcEumcdJ_NQ95TzFA6olmycFzbYpxG13-qcfVrig
2021-11-12 06:46:49,259 volttron.client.vip.zmq_connection DEBUG: url type is <class 'str'>
2021-11-12 06:46:59,271 volttron.client.vip.agent.core ERROR: No response to hello message after 10 seconds.
2021-11-12 06:46:59,271 volttron.client.vip.agent.core ERROR: Type of message bus used zmq
2021-11-12 06:46:59,272 volttron.client.vip.agent.core ERROR: A common reason for this is a conflicting VIP IDENTITY.
2021-11-12 06:46:59,272 volttron.client.vip.agent.core ERROR: Another common reason is not having an auth entry onthe target instance.
2021-11-12 06:46:59,272 volttron.client.vip.agent.core ERROR: Shutting down agent.

In a different tmux session, running volttron the listener agent has this:

'GOOD'
2021-11-12 06:38:03,924 () volttron.platform.auth INFO: AUTH: After authenticate user id: None, b'0b391f6b-8cd2-4ff0-a78d-9f225a9738cd'
2021-11-12 06:38:03,924 () volttron.platform.auth INFO: authentication failure: userid='2999df6e-b956-4436-b2d4-039a1f3ccd73', domain='vip', address='localhost:1000:1000:1724', mechanism='CURVE', credentials=['TqdXj-NZDFLKGu7xz-WAQZ6x289RY4FUZfJ2NWZTOFs']
2021-11-12 06:38:06,887 (listeneragent-3.3 1683) __main__ INFO: Peer: pubsub, Sender: listeneragent-3.3_1:, Bus: , Topic: heartbeat/listeneragent-3.3_1, Headers: {'TimeStamp': '2021-11-12T12:38:06.881742+00:00', 'min_compatible_version': '3.0', 'max_compatible_version': ''}, Message:
'GOOD'
2021-11-12 06:38:11,883 (listeneragent-3.3 1683) __main__ INFO: Peer: pubsub, Sender: listeneragent-3.3_1:, Bus: , Topic: heartbeat/listeneragent-3.3_1, Headers: {'TimeStamp': '2021-11-12T12:38:11.879157+00:00', 'min_compatible_version': '3.0', 'max_compatible_version': ''}, Message:
'GOOD'

Any ideas to try?

Don't know if this works, but when I tested the OpenADR agent with the Volttron platform, I ran the Volttron platform in a different shell, however, I did not use Tmux. Try running Volttron platform in a different without using Tmux and see if you get different results.

Disregard my last. I was able to recreate your issue. I don't know why this is happening. I ran Volttron on both develop and releases/8.1.1 and saw this error. Will look into this and determine the root cause and get this agent working again.

acedrew commented 2 years ago

@bonicim Ben and I worked through this yesterday, he had a site packages version of openleadr that was taking precedence.

bbartling commented 2 years ago

@bonicim here's a gist for what Andrew and I modified to get it working.

In the main function Andrew had me modify a few things as well as create another .json file called key_store.json which resides in the same directory as config.json:

def main():
    """Main method called to start the agent."""
    if keystore_file := os.environ.get("KEYSTORE_FILE"):
        with open(keystore_file) as f:
            keys = jsonapi.load(f)

    vip_main(ven_agent, version=__version__, publickey=keys["public"],secretkey=keys["secret"],serverkey=keys["server"])

key_store.json

{
  "public": "PY-lRPcHfYhOcyJQ4tgTGWNyS5TD-RgBwOBxanqI7lw",
  "secret": "SORg0ZttwYvrwTBjMR4P-KvyphFRXgS2j73yySoFs_U",
  "server": "mjVHcEumcdJ_NQ95TzFA6olmycFzbYpxG13-qcfVrig"
}

Also for Kryo Certs to work on IP Keys I had to run this below to get the fingerprint in shaw format:

openssl x509 -fingerprint -in GeoTrust_Global_CA_2.pem -noout

And on the IPKeys side for the VEN config, their server took the fingerprint without having to remove any of the : characters

Running this from the Poetry virtual environment, it just worked.

KEYSTORE_FILE='./keystore.json' AGENT_CONFIG='/home/bbartling/openadr-ven-working/volttron_openadr_ven/config.json' poetry run python /home/bbartling/openadr-ven-working/volttron_openadr_ven/agent.py

How exactly does the posi-trac rear end on a Plymouth work?

  • It just does.
  • It just does. -Joe Dirt
craig8 commented 2 years ago

Note the := only works with python 3.8+. Glad you got this working for you however!

bonicim commented 2 years ago

@bbartling Nice! On my PR, #17 , the key store is now part of the agent configuration.