Create a migration script for AFJ updating an SQLite Indy SDK wallet to an SQLite Aries Askar wallet

[TimoGlastra]

Related to: https://github.com/hyperledger/aries-framework-javascript/issues/1164

Can be based on: andrewwhitehead/acapy-wallet-upgrade@main/bin/askar-upgrade.py (raw)

We need to create a migration script that allows to update an Indy based wallet to an Aries Askar based wallet.

In this issue only focus on the upgrade of an SQLite based Indy SDK to an SQLite based Aries Askar wallet. We'll likely make it a separate addon (TBD) because it requires an sqlite execution environment which is different per platform.

The steps (taken from the acapy upgrade script) are:

1. Run Pre Upgrade script that directly executes SQLite on the encrypted storage
2. Register a new Askar profile using SQLite (which uses the indy key)
3. Loop through all items stored in the indy database and re-encrypt them in the askar structure
4. remove old indy SDK tables
5. Apply post upgrade transformations directly using Askar. See "Updates" for the list of updates

Updates

Once the wallet database has been transformed to an Askar compliant interface there's still records that use the Indy SDK style context structure. All records that aren't created by AFJ, but internal by the Indy SDK (e.g. the private parts of a credential definition) need to be fetched, transformed to an storage record and the re-saved.

Using storage records for all those objects means that from now on the storage storage is completely separated from the rest of the codebase (i.e. you can swap out askar and it will work fine)

The updates that need to happen (may be missing things):

• Fetch all Indy::Key records and store them as Askar key instances
  • We haven't used key metadata in AFJ ever. Does Indy store metadata itself? If yes, we may need to migrate it
  • After migration remove the Indy::Key and Indy::KeyMetadata records
• Fetch all Indy::MasterSecret records and store them in an AnonCredsMasterSecret record (https://github.com/hyperledger/aries-framework-javascript/issues/1164)
  • After migration remove the Indy::MasterSecret record
• Fetch all Indy::Did records, extract the key and store it as an Askar key object.
  • For some dids a DidRecord will exist, but for some there won't this means the case where a public did was created will break. On mobile devices this isn't common, so we'll skip this edge case for now. We may want to instruct users that they need to re-register the did (by storing a did record) for all dids that existed in the indy-sdk and are actually registered on the ledger. We've also used createDid for e.g. peer to peer connections and connectionless exchanges.
  • After migration remove the Indy::Did record
• Fetch all Indy::Schema records and store them in the existing AnonCredsSchemaRecord. the schema structure should be updated to match the new AnonCreds specification
  • Are there any private parts in the schema? I don't think so
  • After migration remove the Indy::Schema record
• Fetch all Indy::Schema records and store them in the existing AnonCredsSchemaRecord. the schema structure should be updated to match the new AnonCreds specification
  • After migration remove the Indy::Schema record
• Fetch all Indy::CredentialDefinition records.
  • There's three indy records related to cred defs. CredentialDefinition, CredentialDefinitionPrivateKey and CredentialDefinitionCorrectnessProof. The correctness proof is maybe for revocation? Question here is, how do we want to split it? I think maybe a public part (for which we already have a record, we need to update it to match anoncreds spec) and a private part that inclues the private and correctness proof (if defined?). We can also store them all in a single record. Thoughts?
  • After migration remove the credential definition indy records. We also need to remove the Indy::SchemaId record I think?
• We don't have to update revocation registry definitions, revocation registry keys, revocation registry states, and revocation registry info I think because we don't support issuer revocation. Maybe there's some records stored as holder? Need to verify with Andrew Whitehead
• Update all Indy::Credential records to a AnonCredsCredentialRecord. The structure should be updated to remove indy-isms and adhere to the anoncreds spec.

[berendsliedrecht]

I think it looks good, not too familiar with the storage internals.

You mentioned Indy::Schema twice where the second one misses the point about the private parts and to my knowledge there are no private parts, as that would mean that you cannot reuse someone else's schema, right?

Re Indy::CredentialDefinition, I think it would be good to split the private parts from the public parts. We will likely fetch the public parts a lot for issuance and such and not directly work with the private parts all that much (I assume). When we fetch the public parts from the wallet, it would be good not to load the private parts into memory.

Re Indy::Credential, I have never heard of an indy-isms, what is that?

[TimoGlastra]

Relevant repo currently being worked on: https://github.com/Indicio-tech/acapy-wallet-upgrade