Closed Executioner1939 closed 1 month ago
swcurran
commented
1 month ago @esune @WadeBarnes — could one of help out (or find someone on the BC team) to help out with this question. It sounds like the answer is very close, and we’ve definitely had this going for some time.
esune
commented
1 month ago @Executioner1939 thanks for the very through details, this helps understand where you are at.
The startup scripts you are using (with the added parameters described at the bottom of the issue) seem correct to me, and it looks like you have your endorser agent working correctly.
I suspect your issue is related to two things:
What I would try is the following:
/didexchange/create-request or /out-of-band/create-invitation to create a connection invitation and accept it before you do anything else (after provisioning the endorser and author agents, of course).author end as you're already doing. You may also need to set transaction_their_job="TRANSACTION_ENDORSER" on the author agent side.With the auto flags turned on, you should see the transaction going to the endorser, getting processed and sent back to the author. Depending on what you choose when you set-up the request, the endorser may write the transaction for you OR might send a signed transaction back for your author to write/publish.
I think this might be it. Give it a try and, if it doesn't work, we'll dig deeper.
Executioner1939
commented
1 month ago Hi @esune and @swcurran thank you for the reply! I'll tackle this tomorrow and provide feedback!
Executioner1939
commented
1 month ago Hi @esune and @swcurran sorry for the delay, this is part of my RnD and other tickets came in that required my attention.
This is where I have gotten today.
I am using this docker-compose file:
version: '3'
services:
postgres:
image: postgres:16
environment:
- POSTGRES_USER=admin
- POSTGRES_PASSWORD=admin
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- "5432:5432"
endorser_agent:
image: ghcr.io/hyperledger/aries-cloudagent-python:py3.12-1.0.0
depends_on:
postgres:
condition: service_started
ports:
- "8000:8000"
- "11000:11000"
environment:
- ACAPY_WALLET_STORAGE_CONFIG={"url":"postgres:5432","max_connections":5}
- ACAPY_WALLET_STORAGE_CREDS={"account":"admin","password":"admin","admin_account":"admin","admin_password":"admin"}
command: start
--inbound-transport http 0.0.0.0 8000
--outbound-transport http
--admin 0.0.0.0 11000
--wallet-name <wallet-name>
--wallet-key <wallet-key>
--seed <wallet-seed>
--admin-api-key <admin-api-key>
--webhook-url <webhook-url>
--auto-respond-credential-proposal
--auto-store-credential
--auto-respond-presentation-request
--auto-respond-presentation-proposal
--auto-respond-credential-request
--auto-respond-credential-offer
--auto-accept-requests
--auto-accept-invites
--auto-respond-messages
--debug-connections
--debug-credentials
--endpoint http://endorser_agent:8000
--genesis-url <genesis-url>
--accept-taa on_file 1.0
--log-level error
--emit-new-didcomm-mime-type
--emit-new-didcomm-prefix
--exch-use-unencrypted-tags
--tails-server-base-url <revocation-url>
--auto-provision
--wallet-type askar
--wallet-storage-type postgres_storage
--endorser-protocol-role endorser
--endorser-endorse-with-did <endorser-did>
--auto-endorse-transactions
multitenant_agent:
image: ghcr.io/hyperledger/aries-cloudagent-python:py3.12-1.0.0
depends_on:
postgres:
condition: service_started
ports:
- "8001:8000"
- "11001:11000"
environment:
- ACAPY_WALLET_STORAGE_CONFIG={"url":"postgres:5432","max_connections":5}
- ACAPY_WALLET_STORAGE_CREDS={"account":"admin","password":"admin","admin_account":"admin","admin_password":"admin"}
command: start
--inbound-transport http 0.0.0.0 8000
--outbound-transport http
--admin 0.0.0.0 11000
--seed <wallet-seed>
--admin-api-key <admin-api-key>
--wallet-name <wallet-name>
--wallet-key <wallet-key>
--webhook-url <webhook-url>
--auto-respond-credential-proposal
--auto-store-credential
--auto-respond-presentation-request
--auto-respond-presentation-proposal
--auto-respond-credential-request
--auto-respond-credential-offer
--auto-accept-requests
--auto-accept-invites
--auto-respond-messages
--debug-connections
--debug-credentials
--endpoint http://multitenant_agent:8000
--genesis-url <genesis-url>
--accept-taa on_file 1.0
--log-level error
--emit-new-didcomm-mime-type
--emit-new-didcomm-prefix
--exch-use-unencrypted-tags
--tails-server-base-url <revocation-url>
--auto-provision
--wallet-type askar
--wallet-storage-type postgres_storage
--multitenant
--multitenant-admin
--jwt-secret <secret>
--endorser-protocol-role author
--endorser-public-did <endorser-did>
--auto-request-endorsement
--auto-write-transactions
--auto-create-revocation-transactions
--auto-promote-author-did
volumes:
postgres_data:I think it's better to just show where I am at the moment:
I create a connection like this now:
async def create_endorser_connection(subwallet_client: AcaPyClient, endorser_client: AcaPyClient, wallet_name: str) -> str:
"""Create a DID Exchange invitation with custom roles and establish a connection."""
try:
# Step 1: Create the OOB invitation using the endorser_client
oob_invitation = await endorser_client.out_of_band.create_invitation(
auto_accept=True,
body=InvitationCreateRequest(
accept=[
"didcomm/aip1",
"didcomm/aip2;env=rfc19"
],
alias=f"{wallet_name}-to-endorser",
handshake_protocols=[
"https://didcomm.org/didexchange/1.0"
]
)
)
logger.info(f"OOB Invitation URL: {oob_invitation.invitation_url}")
# Step 2: Subwallet accepts the invitation
logger.info("Subwallet is accepting the invitation...")
accept_response = await subwallet_client.out_of_band.receive_invitation(
body=oob_invitation.invitation,
alias=f"{wallet_name}-from-multitenant",
auto_accept=True # Ensure the endorser agent auto-accepts the invitation
)
# Step 3: Update connection metadata from subwallet's perspective
await subwallet_client.connection.set_metadata(
conn_id=accept_response.connection_id,
body=ConnectionMetadataSetRequest(
metadata={
"my_role": "author",
"their_role": "endorser",
"connection_purpose": "endorsement",
"endorser_info": {
"label": "Endorser",
"endorser_did": "<endorser-did>",
"endpoint": "http://endorser_agent:8000"
},
"transaction_jobs": {
"transaction_my_job": "TRANSACTION_AUTHOR",
"transaction_their_job": "TRANSACTION_ENDORSER"
}
}
)
)
logger.info(f"Connection established successfully. Connection ID: {accept_response.connection_id}")
return accept_response.connection_id
except Exception as e:
logger.error(f"Error in create_did_exchange_invitation: {str(e)}")
raiseThen I try to Create my DID and write it to the ledger:
async def create_and_register_subwallet_did(
subwallet_client: AcaPyClient,
endorser_client: AcaPyClient,
connection_id: str,
name: str
) -> DID:
try:
# Create a new DID in the subwallet
create_did_response = await subwallet_client.wallet.create_did(DIDCreate())
did_info = create_did_response.result
logger.info(f"Created new DID in subwallet: {did_info.did}")
# Register the DID on the ledger using the endorser
register_response = await subwallet_client.ledger.register_nym(
create_transaction_for_endorser=True,
conn_id=connection_id,
did=did_info.did,
verkey=did_info.verkey,
alias=name
)
logger.info(f"Transaction initiated. Response: {register_response}")
# Wait for the transaction to be endorsed by the endorser
max_attempts = 10
attempt = 0
while attempt < max_attempts:
transaction_record = await subwallet_client.endorse_transaction.get_transaction(
register_response.txn.transaction_id
)
logger.info(f"Current transaction state: {transaction_record.state}")
if transaction_record.state == "transaction_acked":
await subwallet_client.endorse_transaction.write_transaction(
tran_id=register_response.txn.transaction_id)
if transaction_record.state == "transaction_endorsed":
logger.info("Transaction has been endorsed successfully.")
break
await asyncio.sleep(2) # Wait before checking again
attempt += 1
if transaction_record.state != "transaction_endorsed":
raise Exception("Transaction endorsement failed or timed out.")
return did_info
except Exception as e:
logger.error(f"Error in create_and_register_subwallet_did: {str(e)}")
raiseI am creating a wallet in this way:
async def create_new_wallet(multitenant_client: AcaPyClient, wallet_name: str) -> Tuple[str, str]:
"""Create a new wallet and return its token and wallet ID."""
response = await multitenant_client.multitenancy.create_wallet(
body=CreateWalletRequest(
wallet_name=wallet_name,
wallet_key=f"{wallet_name}_key",
wallet_type="askar",
label=wallet_name,
wallet_webhook_urls=[
<webhook-url>
]
),
)
logger.info(f"Created subwallet '{wallet_name}' with token: {response.token}")
return response.token, response.wallet_idThese are my logs:
2024-09-30 17:07:14,907 - INFO - Wallet 'Test16' already exists.
2024-09-30 17:07:14,952 - INFO - OOB Invitation URL: <invitation>
2024-09-30 17:07:14,952 - INFO - Subwallet is accepting the invitation...
2024-09-30 17:07:15,048 - INFO - Connection established successfully. Connection ID: 83d21958-8171-48da-b723-64cbe7926308
2024-09-30 17:07:15,113 - INFO - Created new DID in subwallet: 24mRMrwnLH868jtSpSRxNR
2024-09-30 17:07:16,247 - INFO - Transaction initiated. Response: success=False txn=TransactionRecord(type='https://didcomm.org/sign-attachment/1.0/signature-request', connection_id='83d21958-8171-48da-b723-64cbe7926308', created_at='2024-09-30T15:07:16.230785Z', endorser_write_txn=False, formats=[{'attach_id': 'f95e7cba-b644-426e-9bb6-5e94e9bb266d', 'format': 'dif/endorse-transaction/request@v1.0'}], messages_attach=[{'@id': 'f95e7cba-b644-426e-9bb6-5e94e9bb266d', 'mime-type': 'application/json', 'data': {'json': '{"did": "did", "verkey": "verkey", "alias": "Test16", "role": null}'}}], meta_data={'did': '24mRMrwnLH868jtSpSRxNR', 'verkey': 'verkey', 'alias': 'Test16', 'role': None}, signature_request=[{'context': 'did:sov', 'method': 'add-signature', 'signature_type': 'default', 'signer_goal_code': 'aries.transaction.ledger.write_did', 'author_goal_code': 'aries.transaction.register_public_did'}], signature_response=[], state='request_sent', thread_id=None, timing={'expires_time': None}, trace=False, transaction_id='b636bd86-3656-47bf-b6e5-e807e260d1d6', updated_at='2024-09-30T15:07:16.235174Z')
2024-09-30 17:07:16,255 - INFO - Current transaction state: request_sent
2024-09-30 17:07:18,269 - INFO - Current transaction state: transaction_acked
2024-09-30 17:07:20,281 - INFO - Current transaction state: transaction_acked
2024-09-30 17:07:22,292 - INFO - Current transaction state: transaction_ackedThese are the ACA-py logs:
endorser_agent-1 | Created new invitation
multitenant_agent-1 | Created new connection record from invitation
multitenant_agent-1 | Created connection request
multitenant_agent-1 | Sent connection request
endorser_agent-1 | Receiving connection request
endorser_agent-1 | Received connection request from invitation
endorser_agent-1 | Creating connection response
endorser_agent-1 | Created connection response
endorser_agent-1 | Sent connection response
multitenant_agent-1 | Accepted connection response
multitenant_agent-1 | Sent connection complete
endorser_agent-1 | Connection promoted to active
endorser_agent-1 | Received connection completeThen it logs this:
onse status 400, caused by: Bad Request'), <traceback object at 0x7f3ca0463480>); Re-queue failed message ...
multitenant_agent-1 | 2024-09-30 15:07:18,136 aries_cloudagent.wallet.routes ERROR Error promoting to public DID: No endorser connection found
multitenant_agent-1 | Traceback (most recent call last):
multitenant_agent-1 | File "/home/aries/.local/lib/python3.12/site-packages/aries_cloudagent/wallet/routes.py", line 1365, in on_register_nym_event
multitenant_agent-1 | _info, attrib_def = await promote_wallet_public_did(
multitenant_agent-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
multitenant_agent-1 | File "/home/aries/.local/lib/python3.12/site-packages/aries_cloudagent/wallet/routes.py", line 861, in promote_wallet_public_did
multitenant_agent-1 | raise web.HTTPBadRequest(reason="No endorser connection found")I still think I am just missing something small as I am staring at logs all day.
esune
commented
1 month ago @Executioner1939 it looks like you may need to set an endorser-alias in the startup parameters for the agent to be able to pick-up the right connection to the endorser (see here.
Executioner1939
commented
1 month ago @esune @swcurran thank you for the help, it seems I finally have it working.
I disabled --auto-promote-author-did on the Multitenant Agent.
I also enabled --requests-through-public-did on the Endorser Agent.
Then I do the following (below code is very messy):
Create the Connection to the Endorser and when it's active I set the Endorser Role and Information.
async def create_endorser_connection(subwallet_client: AcaPyClient, wallet_name: str) -> str:
"""Create a DID Exchange invitation with custom roles and establish a connection."""
try:
request = await subwallet_client.did_exchange.create_request(
their_public_did="NF8U1Dp9xrRikXisTJuaEs",
alias=f"{wallet_name}-from-multitenant",
my_endpoint="http://multitenant_agent:8000"
)
max_attempts = 10
attempt = 0
while attempt < max_attempts:
conn = await subwallet_client.connection.get_connection(conn_id=request.connection_id)
logger.info(f"Current connection state: {conn.state}")
if conn.state == "active":
await subwallet_client.endorse_transaction.set_endorser_role(conn_id=conn.connection_id, transaction_my_job="TRANSACTION_AUTHOR")
await subwallet_client.endorse_transaction.set_endorser_info(conn_id=conn.connection_id, endorser_did="<endorser-did>")
return conn.connection_id
await asyncio.sleep(2)
attempt += 1
logger.info(f"Connection established successfully. Connection ID: {conn.connection_id}")
return conn.connection_id
except Exception as e:
logger.error(f"Error in create_did_exchange_invitation: {str(e)}")
raiseThen once it's created properly, I create the DID and Write it to the Ledger and set it to public using the Endorser
async def create_and_register_subwallet_did(
subwallet_client: AcaPyClient,
connection_id: str,
name: str
) -> DID:
try:
# Create a new DID in the subwallet
create_did_response = await subwallet_client.wallet.create_did(DIDCreate())
did_info = create_did_response.result
logger.info(f"Created new DID in subwallet: {did_info.did}")
# Register the DID on the ledger using the endorser
register_response = await subwallet_client.ledger.register_nym(
create_transaction_for_endorser=True,
conn_id=connection_id,
did=did_info.did,
verkey=did_info.verkey,
alias=name
)
logger.info(f"Transaction initiated. Response: {register_response}")
max_attempts = 10
attempt = 0
while attempt < max_attempts:
transaction_record = await subwallet_client.endorse_transaction.get_transaction(
register_response.txn.transaction_id
)
logger.info(f"Current transaction state: {transaction_record.state}")
if transaction_record.state == "transaction_acked":
await subwallet_client.wallet.set_public_did(did=did_info.did, conn_id=connection_id, create_transaction_for_endorser=True)
return did_info
await asyncio.sleep(2)
attempt += 1
return did_info
except Exception as e:
logger.error(f"Error in create_and_register_subwallet_did: {str(e)}")
raiseThen using the connection I am able to create Schemas and Credential Definitions.
swcurran
commented
1 month ago That’s great. We need to get an update to the Endorser and/or Multitenancy documentation to capture what is in here.
WadeBarnes
commented
1 month ago Sorry I'm late to the party. Here are a number of references that would likely have helped:
I think @loneil might be able to point at some examples of how the connections are established in Traction too.
Executioner1939
commented
1 month ago @WadeBarnes thanks for the additional information, I actually stumbled across the second link you posted which helped.
A few clarifying questions:
WadeBarnes
commented
1 month ago
- Once I have created this connection, I will need to use it for all transactions for this wallet going forward, including Credential Issuance and so on?
Yes, it's a dedicated (named) connection between the author and endorser.
- Are you using that registerAuthor script in k8s? If were to do the following, with an initContainer would I only need to ensure a single connection is created? Or should it be for each Tenant?
Each tenant requires a connection, I believe. We were using the script initially to setup standalone agents. @loneil would have details of how that's configured and managed in Traction.
You might find it difficult to configure an endorser connection with and initContainer since it requires interaction between the author and endorser including some final configuration on both sides once the connection is established. If you were to do it with an initContainer you'd need to ensure only the single connection is made. If you look at the script is goes out of it's way to ensure there is a single connection and if anything goes wrong, that same connection is detected before the process continues.
Executioner1939
commented
1 month ago @WadeBarnes thanks for the information, that makes sense.
I think we can mark this issue as solved. The information in this document served perfectly: https://github.com/hyperledger/aries-cloudagent-python/tree/main/demo/docker-agent#connecting-to-an-endorser-service
Hi all,
I have some questions regarding the setup of my Multitenancy Node with a separate Endorser Node.
I feel as though I am going in circles and am stuck in a configuration nightmare so to speak and think I have just confused myself to the point that I need to ask for some clarifications on the setup.
I feel I am missing something as I can't get my Sub wallets (hereafter referred to simply as Tenants) to create the Schemas and associated Credential Definitions.
I have already read the following:
1657
983
1460
My takeaway from the above is to use a separate instance of ACA-py to run as an Endorser and do the Endorsement for transactions for my Tenants.
I am using the following version of ACA-py, denoted by this docker image, I am also using
askaras the wallet type.ghcr.io/hyperledger/aries-cloudagent-python:py3.12-1.0.0I have also read the documentation here:
I will detail below what I have done and where I am currently at.
I am using the von-network for testing, I have it running on a VM in GCP. I created a DID and Wallet for it following this guide:
This gives me a Public
DID,Verkey, andSeed. I then registered, via the UI, theDIDas anEndorseron the von-network.I then have one instance of ACA-py running as my "Endorser" with the following startup parameters (db parameters excluded):
Then I have my Multitenant Agent running with these parameters (db parameters excluded):
If someone can help me strip down the arguments to the bare minimum to get this working as well that would help.
Now, I have a bootstrap script, this is used to:
This can be seen below (Please note, I have summarized my code into a small single function to convey the steps I am following)
Questions:
When I run the code, it successfully creates the Tenant, DID, Writes it to the Ledger, creates the connection, sets the metadata and sets the DID as public as be seen from my logs:
But when I try to perform actions against my Subwallet I am running into issues, I shall just show the one method I use to create a Schema, I have a feeling this approach is completely wrong:
This always resulted in a Transaction Not Found error.
NB: Please also note, the Multitenant Agent and Endorser Agent are using separate databases, I am not sure if this should be the case.
I then started looking at the Endorser CLI arguments and amended my
Endorser Agentto include:and my
Multitenant Agentto include:Now when I run my script, it fails with the following error:
I would like some guidance if anyone has some time as I feel I took a wrong turn and wound up in an abandoned mine full of confusion and despair.
I will need to document this entire setup at work and would be very willing to contribute it back to the ACA-py documentation once done.
Please let me know if anything is unclear or if I have simply misunderstood something basic.