search

openwallet-foundation / owl-agent-test-harness

Aries agent test framework, with agent backchannel support
https://aries-interop.info
Apache License 2.0
60 stars 66 forks source link

Mobile Backchannel Headless App Support #834

Open nodlesh opened 5 months ago

nodlesh commented 5 months ago

Goal: Develop a DIDComm-based (DRPC) back-channel communication protocol to facilitate testing for any Aries agent including headless mobile.

Description: We aim to create a communication framework using DIDComm (Decentralized Identity Communication) and DRPC (Decentralized Remote Procedure Call) to enable back-channel interactions in AATH with Aries Agents. This will support the seamless, headless testing of Aries agents, allowing for automated and efficient testing processes without the need for a user interface.

This proposal would impact the Aries Frameworks. There would be a relatively simple protocol with a handful of APIs to allow AATH to control an agent through this protocol as it does now through the backchannel API. Each Aries Framework would have to implement this protocol to participate in the interop test suite. The protocol could even become part of AIP #.# or whatever means we have to determine that protocols should be implemented at a given level.

The backchannel logic for each framework would be moved inside the framework, and developed in the frameworks repo. The hope is that the backchannel logic would be maintained as protocols get added or changed, and that the testing of these protocols with other frameworks will be front and centre to development of these frameworks. The above perceived benefit may not be the case. The Backchannel is a controller and there is no reason this needs to move inside the frameworks. So the Backchannel is probably still external, maybe in its own repo, in a plug-in or stay in AATH?

When working with a mobile agent (mobile wallet) The wallet and AATH would need a connection, then once connected, the mobile agent can go into "headless mode" and allow AATH to send instructions to the wallet to communicate over didcomm to issuers, verifiers, and other wallets/holders.

Some initial draft thoughts on what it might look like: https://hackmd.io/@wrf54Y6BSmWpzuBx9yJg_Q/S1Ih_h_rR

Summary of Questions, Concerns & Comments:

  1. Connection between AATH and the Agent with a DIDComm Connection? before DRPC can happen? How?
  2. AATH would have to initiate the connection.
  3. The Test Protocol could be a plugin?
  4. Is there another use case for having DRPC calls through this protocol besides testing?
  5. The Headless Mobile use case should be enough of a reason to move forward. However, as Daniel Bluhm mentions below, "The main value is mobile agent backchannel and I think this can and should be done independent of the backchannel interfaces of other agent implementations."
  6. AATH using DIDComm V2 for connectionless operations. DID as a toggle and allow-list for DRPC protocol.

Current Proposed:

Screenshot 2024-07-23 at 12 23 41 PM
nodlesh commented 4 months ago

Adding some Good conversation in the chat from ACAPUG on July 23, 2024

11:13:20 From Daniel Bluhm to Everyone: Jotting down some thoughts:

nodlesh commented 4 months ago

Link to the July 23, 2024 ACAPUG recording where this was discussed is here, https://wiki.hyperledger.org/display/ARIES/2024-07-23+Aries+Cloud+Agent+-+Python+Users+Group+Community+Meeting

swcurran commented 3 months ago

I think that establishing the connection would be done something like the following:

Note that this setup does not take into account the current feature of being able to restart a test agent with a different configuration when needed. This is a common thing done in ACA-Py tests — restarting the agents with different startup parameters between tests.

swcurran commented 3 months ago

I think the only thing this gets us is the ability to directly connect to a wallet to play a headless role in the testing. It doesn’t move any of the backchannel features into the frameworks — they remain in the backchannel controller. It’s a pretty big change to the architecture and the existing backchannel for a relatively small addition to the capability. A good addition, but all the changes needed in the existing backchannels don’t result in any benefits — and perhaps some loss of functionality.

What about doing this? Add a “wallet-backchannel” that:

I think we get the same result and have to do the same thing in the wallet, but we don’t change the architecture, and we don’t have to update the ACA-Py, Credo and Aries VCX backchannels.

nodlesh commented 3 months ago

Just wanted to note that doing anything mentioned in this issue doesn't totally eliminate the need to put the wallet into a state to be able to accept a connection. App startup, initialization, and user onboarding may still need to happen using UI automation. This means that to run this fully automated and/or in test/build pipelines would require some of the infrastructure of the Aries Mobile Test Harness/Mobile Wallet Test Harness.

JamesKEbert commented 3 months ago

I haven't been involved in too many of the conversations on this proposed protocol, but I wanted to provide some feedback on this--I think it would be cool, but I have some concerns:

  1. Having our testing framework rely on the technology we're testing could prove potentially troublesome. For instance, if there are issues with connections, then it's possible that testing fails for all tests, when that's not really accurate to the situation. But, I think this could potentially be outweighed by the benefits of enabling testing in mobile contexts.
  2. I am concerned about the usage of DRPC in this context. From my perspective, it would be better to create a new protocol or set of protocols to handle this, rather than create a new DRPC service that gets piped over DIDComm. This could be something that looks like the existing aries-toolbox admin api protocols:
nodlesh commented 3 months ago
  1. Having our testing framework rely on the technology we're testing could prove potentially troublesome. For instance, if there are issues with connections, then it's possible that testing fails for all tests, when that's not really accurate to the situation. But, I think this could potentially be outweighed by the benefits of enabling testing in mobile contexts.

Yes, I agree in principle to this, and I also mentioned something akin to this in the ACAPug meeting when we talked about this solution. However upon further reflection, the testing principle at play here is the fact that we don't want the system under test to do any validation of the system under test(itself). In this case, we may use the system under test to establish a connection to service the actual testing on mobile, so no testing or validation is done unless this connection is established. In other words it is part of the test setup procedure(test fixture) before any validation happens and if this setup procedure fails then we don't run tests or do any validation.

swcurran commented 3 months ago

After the discussion on the call, I don’t think this makes sense as a change for all Test Agents/Backchannels. In listening to the discussion at the ACA-Pug meeting, I don’t think this approach gives much value for other than the mobile agent test case. For that, I think that enhancing the existing mobile backchannel should be considered as the easiest way to do periodic runs. For automated runs, so sort of headless wallet mechanism would be needed.

Or, we could go down this root for a mobile wallet. Create a backchannel for a specific wallet (e.g. Bifold/BC Wallet — that is perhaps general enough for other wallets) that uses DIDComm and DRPC to control the wallet. The backchannel would use the same HTTP interface between the test engine and the backchannel as all backchannels, but the backchannel and the Bifold (or whatever)instance would use DIDComm to tell the instance what to do. Not easy though: