Enhancements to Hyperledger Indy SDK (wallet) and implementation of a large scale claims store

[ccoldwell]

Value: $50,000.00Closes: Wednesday, January 31, 2018Location: Victoria In-person work NOT required

Opportunity Description

This is part of an effort to understand the applicability of emerging technologies to solve some challenge problems in public sector. We are working in the open source community, in particular, the Hyperledger Indy community to explore challenges in the service to business domain. We want to find ways to improve the experience of BC businesses in their interactions with government. This work falls into a new area that is known in the industry as "self-soveriegn identity" or decentralized identity. 

In the self-sovereign identity (SSI) world, “wallet” is the go to term for the digital equivalent to the physical place you keep your data. The type of data held in the wallet depends somewhat on the role of the identity in a self-sovereign ecosystem - a claims issuer, holder (aka prover) or verifier (aka inspector). Any particular identity might take on multiple roles and so have any and all types of SSI data. We expect that for most "typical" use cases for wallets, the major implementations will be provided by 3rd parties in the market. For example, there are software vendors working on mobile wallets targeting consumers, whose primary role is as a holder of claims about themselves (they are the subject of the claims). As well, we understand software vendors are working on "Enterprise Agents" - applications that have wallets for the purpose of primarily operating as either or (more commonly, we think) both claims issuers and claim verifiers.

TheOrgBook is somewhat different from both of those "typical" models. It is primarily a claims holder, but quite a different one from a consumer-type claims holder. It will only be holding public claims about organizations. Notably, it will hold large scale data volumes, and the claims that it holds are about many subjects (organizational entities) - not about itself. As such the requirements of its wallet are quite different from those of a typical consumer wallet.

A claims holder wallet (consumer or TheOrgBook) would be expected to contain the following:

• DIDs (references to decentralized IDs) that make up the wallet owner's identity, and context of those DIDs - e.g. for pair-wise DIDs, the connection to which that DID applies.
• The DIDs of the owner’s connections (e.g. banks, stores, government services and so on)
• Verifiable credentials issued to the owner
• Cryptographic materials - public and private keys associated with your DIDs, and possibly other materials such as in the Hyperledger Indy implementation your Master Secret for requesting claims (to issuers) and providing proofs (to verifiers)

Note that the private keys may or may not (by implementation) go in the wallet - they may deliberately be kept separate to prevent the theft of a wallet giving the thief access to the use and all the information in the wallet.

While we expect the TheOrgBook wallet to hold those same pieces of data to be used for the operations as a consumer wallet, there are several requirements that are quite different:

• The volume of claims will be much higher than a typical consumer holder wallet - on the order of millions.
• The number of claims associated with specific Schema/Issuers will be much higher than a typical consumer wallet. For example, in loading the full history of the BC Registry set of "Certificate of Incorporation" and related claims (annual reports, name changes, amalgamations, etc), there will be millions claims loaded. In a typical consumer/small business scenario there would likely be 1 or at most a small number of "Certificate of Incorporation" claims.
• Claims are about different subjects, with the unique identifier of the claim subject embedded in a field in the claim.

Large Scale Persistence

TheOrgBook currently uses the default Hyperledger Indy wallet implementation based on an encrypted version of SQL-Lite (SQLCipher). To both handle the volume of claims that TheOrgBook will need to support and to provide more robust database administration handling, we want to update the wallet implementation to use (likely) PostgreSQL for persistence. TheOrgBook runs on the BC Government's private Red Hat OpenShift Platform as a Service implementation, and for relational databases, PostgreSQL is the preferred choice. Note that if the developers feel that a noSQL-based wallet solution would be better, we would like go to MongoDB (although Redis is available out of the box with OpenShift).

Getting Claims for Proof Requests

The current Hyperledger Indy wallet interface has a call that given a Proof Request (an array of claim names, each from a possibly different credential associated with one or more schema and/or issuers) returns all of the credentials in the wallet that could satisfy each claim in the Proof Request. Since, as noted above, the wallet of TheOrgBook holds the same claim for (literally) millions of subjects, Proof Requests will return from the default wallet API call millions of credentials - likely causing a significant performance issue. To prevent a performance impact we have proposed that a Proof Request can include an optional filter condition that allows the call to the wallet to filter based on claim values the credentials of interest for a given Proof Request. In case of TheOrgBook, the filter condition will usually simply be the unique identifier of the Organization of interest - the subject of the claim. Our proposal for an update to the Hyperledger Indy Proof Request format to support this functionality can be found in the Hyperledger Indy JIRA system (IS-486).

While we think that change could be used to resolve this issue, we are open to other proposals. In addition, we think (but again, could be wrong) that because of the data volumes in TheOrgBook, the wallet implementation will need to do this credential filtering at as low a level as possible - ideally at the database level - to prevent the manipulation of large volumes of data for each Proof Request. There may be challenges with this depending on how the existing wallet implementation stores the data.

Opportunity

This is an opportunity to work with the Verifiable Organizations Network team to deliver enhancements to both a Hyperledger Project and BC Government projects.

The fixed-price reward is for a potential total of $CDN 50,000* for satisfaction of the Acceptance Criteria below, per this payment schedule:

• Completion of Phase 1 - $10,000
• Completion of Phase 2 - $20,000
• Completion of Phase 3 - $20,000

* The full amount is dependent on an evaluation of outputs at the end of Phase 1, during which Verifiable Organizations Network team will determine whether or not work will continue into phases 2 and 3.

Acceptance Criteria

• Phase 1 - Agree upon database backend technology, technical approach, implementation details, testing plan, and interaction flow diagrams (for all phases). Develop materials necessary to share the implementation proposal with the Hyperledger Indy SDK community, share with the community and incorporate feedback to increase potential for implementation to be accepted as a pull request to Hyperledger Indy SDK. Outcomes of this phase could impact the activities of Phase 2 and 3.
• Phase 2 - Implement a large scale wallet solution using the agreed upon technology in Phase 1. The wallet implementation should have the same features as the existing HL-Indy wallet implementation and include a thorough test suite. Test suite should exercise implementation for load and volume of claims issued and stored. Test scenario should exceed 10 million claims of a variety of claim types to be stored. Implementation must also test proof request/response throughputs. Existing users should be able to easily switch to the new implementation by providing a database connection string or similar. Implementation will conform with coding standards of the Hyperledger Indy SDK to minimize effort of integration into main implementation of Indy-SDK. This phase will not be gated by acceptance into main Indy-SDK GitHub repo but must be merged into a forked copy of the main Indy-SDK repo in the BCGov GitHub Organization.
• Phase 3 - Based on the agreed upon approach in phase 1, implement the ability to filter claims returned by the holder’s wallet at the database level. This should be implemented for the new implementation and the existing implementation. The implemented feature must include a thorough test suite. Test suite should exercise implementation for load and volume of claims proven. More information regarding the requirement is available here: https://jira.hyperledger.org/projects/IS/issues/IS-486. Implementation will conform with coding standards of the Hyperledger Indy SDK to minimize effort of integration into main implementation of Indy-SDK. This phase will not be gated by acceptance into main Indy-SDK GitHub repo but must be merged into a forked copy of the main Indy-SDK repo in the BCGov GitHub Organization.

How to Apply

Go to the Opportunity Page, click the Apply button above and submit your proposal by 16:00 PST on Wednesday, January 31, 2018.

We plan to assign this opportunity by Friday, February 2, 2018 with work to start on Monday, February 5, 2018.

Proposal Evaluation Criteria

1. Your approach to completing the Acceptance Criteria in a short proposal (1-2 pages) which includes evidence to support the criteria outlined in items 2-6 below. Evidence can include GitHub IDs, projects for example. (20 points)
2. Your prior experience contributing to community-based open source projects. (10 points)
3. Your experience in identity, self-sovereign identity. (20 points)
4. Your experience in verifiable claims, decentralized identifiers, distributed ledger/blockchain or other relevant technologies. (30 points)
5. Your expertise in large scale services design and implementation. (20 points)
6. Your ability to satisfy the Acceptance Criteria on or before 31 March 2018 (10 points)

[jljordan42]

Hi Everyone ... happy to answers any questions that arise from this opportunity.

[ianco]

Is there a strict 2-page limit on proposals?

Thanks!

[jljordan42]

That is an order of magnitude suggestion. The idea being not 10 pages .. if its a bit longer that 2 that is fine if the extra space is needed.

[ianco]

Great, thanks for the clarification!

[jljordan42]

Done

On Jan 30, 2018, at 15:16, Ian Costanzo notifications@github.com<mailto:notifications@github.com> wrote:

Great, thanks for the clarification!

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/bcgov/TheOrgBook/issues/164#issuecomment-361767724, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AD-6DFi06msfL2uU5CaKZkR77vSpO8E0ks5tP6LbgaJpZM4RsHQT.

[jljordan42]

Thanks to everyone for their interesting and bids. We are pleased to assign this opportunity to ianco. If you bid and would like a debrief .. please contact me.

[ccoldwell]

This opportunity has been assigned

[swcurran]

This work is now complete.