Issuer uses wrong accumulator values after fixing revocation registry with `PUT /revocation/registry/{rev_reg_id}/fix-revocation-entry-state`

How I break the revocation registry:

So I issue 25 credentials to a holder (from one issuer to one holder) all from the same cred_def.
I then try to break the revocation registry by revoking all issued credentials simultaneously (no batching).
By doing this I am able to get to a state where the ledger complains about current accumulator value being wrong.
When I take a look at the ledger I see that some of the REVOC_REG_ENTRYs of credentials the issuer believes is revoked never ended up on the ledger. (I think the writes reach the ledger out of order hence the ledger complaining about the current accumulator value)

(we are running acapy in multi-tenancy mode with an endorser)

Now I try to fix it:

Then I use the endpoint PUT /revocation/registry/{rev_reg_id}/fix-revocation-entry-state
This seems to work as intended. It finds the diff between the ledger and issuer and sends the new REVOC_REG_ENTRY to the ledger and I can find the entry on the ledger.

BUT when I then try to revoke another credential the accumulator values used are completely wrong.

Take a look at my calls below:

PUT /revocation/registry/KES3KNQbpuq5T84XLnTFQQ%3A4%3AKES3KNQbpuq5T84XLnTFQQ%3A3%3ACL%3A8%3AEpic%3ACL_ACCUM%3Adcf175a9-2732-451f-858c-c13e754917e0/fix-revocation-entry-state?apply_ledger_update=false

{
  "rev_reg_delta": {
    "ver": "1.0",
    "value": {
      "accum": "21 13083F2D51F85F208A0E4EC2622927E3A24C8AFEF37D9F70A5233B32168C56FFB 21 13F31355419C3C9BE9EE9A09266922073F128ED8003C6608FCE5D0D7B25A6067F 6 70FB3DFFBB30036CD74327469206039311BE70FDF74E7E920F3D1ED6D7A42B4E 4 3529428C8377D928E503B38D8A5558C981BD4FA6440B53E381D3F7B841183B2F 6 8B2A4BE5126C79A4CBD6B7E13684A6B580B95F95779B77D77DDD1DCFD38B1A32 4 09C20CE0B8EBEC6885BD4CE90726CB10C07D8602ACD33C5AACA1C3F177458141",
      "issued": [],
      "revoked": [
        1,
        5,
        7,
        14,
        19
      ]
    }
  },
  "accum_calculated": {
    "ver": "1.0",
    "value": {
      "prevAccum": "21 13083F2D51F85F208A0E4EC2622927E3A24C8AFEF37D9F70A5233B32168C56FFB 21 13F31355419C3C9BE9EE9A09266922073F128ED8003C6608FCE5D0D7B25A6067F 6 70FB3DFFBB30036CD74327469206039311BE70FDF74E7E920F3D1ED6D7A42B4E 4 3529428C8377D928E503B38D8A5558C981BD4FA6440B53E381D3F7B841183B2F 6 8B2A4BE5126C79A4CBD6B7E13684A6B580B95F95779B77D77DDD1DCFD38B1A32 4 09C20CE0B8EBEC6885BD4CE90726CB10C07D8602ACD33C5AACA1C3F177458141",
      "accum": "21 13CE554864E756CCCE6506120714A421DA98AA78E605A833FAC2A2503C76A7F4D 21 12152059E7F05B876EDE1F9F279530EB7101DAD585B12EBF329981B3DD1EEF8E7 6 64BA3C768FC560E91ABEEA2B79E24998471EFFFDC5814581D2959B9F0BE8897A 4 31A15893752370A38D7CFBB4E7AA60A1F656E6352004CC57FCA64FCC8B698FBC 6 7F89D9C6FE1C55E0FACBCEF4955A3B558EB76FC6FBE136E82F1C757DD4D58724 4 32E48552BD4BB61F784D8432641EDBD4CFDD75A7713734EC3B8E869BE88DE610",
      "revoked": [
        13,
        15,
        10,
        25
      ]
    }
  },
  "accum_fixed": {}
}

then with true

PUT /revocation/registry/KES3KNQbpuq5T84XLnTFQQ%3A4%3AKES3KNQbpuq5T84XLnTFQQ%3A3%3ACL%3A8%3AEpic%3ACL_ACCUM%3Adcf175a9-2732-451f-858c-c13e754917e0/fix-revocation-entry-state?apply_ledger_update=true

Response:

{
  "rev_reg_delta": {
    "ver": "1.0",
    "value": {
      "accum": "21 13083F2D51F85F208A0E4EC2622927E3A24C8AFEF37D9F70A5233B32168C56FFB 21 13F31355419C3C9BE9EE9A09266922073F128ED8003C6608FCE5D0D7B25A6067F 6 70FB3DFFBB30036CD74327469206039311BE70FDF74E7E920F3D1ED6D7A42B4E 4 3529428C8377D928E503B38D8A5558C981BD4FA6440B53E381D3F7B841183B2F 6 8B2A4BE5126C79A4CBD6B7E13684A6B580B95F95779B77D77DDD1DCFD38B1A32 4 09C20CE0B8EBEC6885BD4CE90726CB10C07D8602ACD33C5AACA1C3F177458141",
      "issued": [],
      "revoked": [
        1,
        5,
        7,
        14,
        19
      ]
    }
  },
  "accum_calculated": {
    "ver": "1.0",
    "value": {
      "prevAccum": "21 13083F2D51F85F208A0E4EC2622927E3A24C8AFEF37D9F70A5233B32168C56FFB 21 13F31355419C3C9BE9EE9A09266922073F128ED8003C6608FCE5D0D7B25A6067F 6 70FB3DFFBB30036CD74327469206039311BE70FDF74E7E920F3D1ED6D7A42B4E 4 3529428C8377D928E503B38D8A5558C981BD4FA6440B53E381D3F7B841183B2F 6 8B2A4BE5126C79A4CBD6B7E13684A6B580B95F95779B77D77DDD1DCFD38B1A32 4 09C20CE0B8EBEC6885BD4CE90726CB10C07D8602ACD33C5AACA1C3F177458141",
      "accum": "21 13CE554864E756CCCE6506120714A421DA98AA78E605A833FAC2A2503C76A7F4D 21 12152059E7F05B876EDE1F9F279530EB7101DAD585B12EBF329981B3DD1EEF8E7 6 64BA3C768FC560E91ABEEA2B79E24998471EFFFDC5814581D2959B9F0BE8897A 4 31A15893752370A38D7CFBB4E7AA60A1F656E6352004CC57FCA64FCC8B698FBC 6 7F89D9C6FE1C55E0FACBCEF4955A3B558EB76FC6FBE136E82F1C757DD4D58724 4 32E48552BD4BB61F784D8432641EDBD4CFDD75A7713734EC3B8E869BE88DE610",
      "revoked": [
        10,
        25,
        13,
        15
      ]
    }
  },
  "accum_fixed": {
    "txn": {
      "type": "114",
      "data": {
        "revocDefType": "CL_ACCUM",
        "revocRegDefId": "KES3KNQbpuq5T84XLnTFQQ:4:KES3KNQbpuq5T84XLnTFQQ:3:CL:8:Epic:CL_ACCUM:dcf175a9-2732-451f-858c-c13e754917e0",
        "value": {
          "accum": "21 13CE554864E756CCCE6506120714A421DA98AA78E605A833FAC2A2503C76A7F4D 21 12152059E7F05B876EDE1F9F279530EB7101DAD585B12EBF329981B3DD1EEF8E7 6 64BA3C768FC560E91ABEEA2B79E24998471EFFFDC5814581D2959B9F0BE8897A 4 31A15893752370A38D7CFBB4E7AA60A1F656E6352004CC57FCA64FCC8B698FBC 6 7F89D9C6FE1C55E0FACBCEF4955A3B558EB76FC6FBE136E82F1C757DD4D58724 4 32E48552BD4BB61F784D8432641EDBD4CFDD75A7713734EC3B8E869BE88DE610",
          "prevAccum": "21 13083F2D51F85F208A0E4EC2622927E3A24C8AFEF37D9F70A5233B32168C56FFB 21 13F31355419C3C9BE9EE9A09266922073F128ED8003C6608FCE5D0D7B25A6067F 6 70FB3DFFBB30036CD74327469206039311BE70FDF74E7E920F3D1ED6D7A42B4E 4 3529428C8377D928E503B38D8A5558C981BD4FA6440B53E381D3F7B841183B2F 6 8B2A4BE5126C79A4CBD6B7E13684A6B580B95F95779B77D77DDD1DCFD38B1A32 4 09C20CE0B8EBEC6885BD4CE90726CB10C07D8602ACD33C5AACA1C3F177458141",
          "revoked": [
            10,
            25,
            13,
            15
          ]
        }
      },
      "protocolVersion": 2,
      "metadata": {
        "from": "KES3KNQbpuq5T84XLnTFQQ",
        "reqId": 1728562024140839400,
        "taaAcceptance": {
          "mechanism": "service_agreement",
          "taaDigest": "0be4d87dec17a7901cb8ba8bb4239ee34d4f6e08906f3dad81d1d052dccc078f",
          "time": 1728518400
        },
        "digest": "9734742736b2de4ee16c81a79490f70830273e4f6c62604761b45906b5f66d53",
        "payloadDigest": "3dda19b98d7176ebb126c4205537a30ca827e70f53ebb8f9f9b8cb56c4d037c1"
      }
    },
    "txnMetadata": {
      "txnId": "5:KES3KNQbpuq5T84XLnTFQQ:4:KES3KNQbpuq5T84XLnTFQQ:3:CL:8:Epic:CL_ACCUM:dcf175a9-2732-451f-858c-c13e754917e0",
      "txnTime": 1728562024,
      "seqNo": 21
    },
    "reqSignature": {
      "type": "ED25519",
      "values": [
        {
          "from": "KES3KNQbpuq5T84XLnTFQQ",
          "value": "4mYhB9CjXCkj7JvMpsW3gN6ygiZvzb7g4MoeN9bxqUGvoqdiyCpdwFWtRUqJ4HykSU923NT9MpqcBb4tfzAYprcH"
        }
      ]
    },
    "ver": "1",
    "rootHash": "2han1X3VHhJJvqx3cFGe2scVDqSbxPBBsrmfgV6X3fEG",
    "auditPath": [
      "Avnr4usgJjzWT3zt5skzUwNzfCuoMryMhdcTpFAA13qe",
      "HunqRHaVSsxqGcwmXzCXntucuwE3rsevMwjrgDVBYzYi"
    ]
  }
}

I then revoke another credential and the endorser receives the following endorsement event (note the accumulator values )

{
    "group_id": null,
    "origin": "Governance",
    "payload": {
        "_type": "https://didcomm.org/sign-attachment/1.0/signature-request",
        "connection_id": "697f8b3f-ec38-4b34-af70-d236139a12c9",
        "created_at": "2024-10-10T12:09:32.000574Z",
        "endorser_write_txn": false,
        "formats": [
            {
                "attach_id": "87e79835-a725-447f-b9e3-fdb85ab1214b",
                "format": "dif/endorse-transaction/request@v1.0"
            }
        ],
        "messages_attach": [
            {
                "@id": "87e79835-a725-447f-b9e3-fdb85ab1214b",
                "data": {
                    "json": {
                        "endorser": "2WkBvUzBd1iFSHXhJNBwXi",
                        "identifier": "KES3KNQbpuq5T84XLnTFQQ",
                        "operation": {
                            "revocDefType": "CL_ACCUM",
                            "revocRegDefId": "KES3KNQbpuq5T84XLnTFQQ:4:KES3KNQbpuq5T84XLnTFQQ:3:CL:8:Epic:CL_ACCUM:dcf175a9-2732-451f-858c-c13e754917e0",
                            "type": "114",
                            "value": {
                                "accum": "21 121CFFB45DB55FCB6F4D3F733CB06C9A83F967F102ACC12B22335980B7DD446D6 21 1366B793EEE53E7EAE10927CAAFE5A461E94FC88D87FF4ECC1F2CCDABF6181D87 6 5E2A92CE61638A19EDB310F374C8215656C351C78D58CFF5AC5E98A9FF16C86A 4 01C449D8D4EA9B9FF9AC6B4F18ECD91AC9124E1FF81C6693D82D842031141105 6 6D8057BEAA0CE71FB39C745367DE2106EB143905144B266B13DAC22A7D303B9B 4 1B1484CAD9BEE12772BF319FFF599F471A4E16013834C3D9BF6119250BB2E471",
                                "prevAccum": "21 1281369D5EF8484AC92C577D505AFAC54AA75DE0D9542CB3C0276FFF1773A4BA7 21 111F585AC0E1F20CD76B5757641DFCC85008BF9530CE6CEAAA54AE40F73799819 6 6D3AFCD25E2EA7D0EFD2FD8F3ADAF878A3CABCC62BEA14BC7F25FB0BC94AADDC 4 278CE5F698CCE332722A94EEC72534B0316B228854EAEFA01B990B82AFF907BE 6 6C297F213A57DA35AEE8EF0C069AC4FE2A70B05B8E63A3024BA43DEF633E564A 4 138A1D46F1CABF8991F8A6A3C472BF6C38D3ACAFE017CF6158411CA024694BC8",
                                "revoked": [
                                    11
                                ]
                            }
                        },
                        "protocolVersion": 2,
                        "reqId": 1728562171951909164,
                        "signature": "29bFhsd5oejVgbYnbCkuSzQaw1LHuPu3zZFDswZUtZe5MXUEjB4bjX7ZXeSMYCFrkoPXHKuAGXMDqCBumBEnViBm",
                        "taaAcceptance": {
                            "mechanism": "service_agreement",
                            "taaDigest": "0be4d87dec17a7901cb8ba8bb4239ee34d4f6e08906f3dad81d1d052dccc078f",
                            "time": 1728518400
                        }
                    }
                },
                "mime-type": "application/json"
            }
        ],
        "meta_data": {
            "context": {},
            "processing": {}
        },
        "signature_request": [
            {
                "author_goal_code": "aries.transaction.ledger.write",
                "context": "did:sov",
                "method": "add-signature",
                "signature_type": "default",
                "signer_goal_code": "aries.transaction.endorse"
            }
        ],
        "signature_response": [],
        "state": "request-received",
        "thread_id": "b23ddea1-3ab9-49f1-b80c-b48cde37cf01",
        "timing": {
            "expires_time": null
        },
        "trace": false,
        "transaction_id": "7a407812-95b1-4dcd-9fb4-d2554007a654",
        "updated_at": "2024-10-10T12:09:32.000574Z"
    },
    "topic": "endorsements",
    "wallet_id": "Governance"
}

I can then again use PUT /revocation/registry/{rev_reg_id}/fix-revocation-entry-state to get that revoked credential's REVOC_REG_ENTRY on the ledger which works, but the accumulator values of issuer stay wrong when revoking further credentials.

Let me know if I am being an idiot or if you need more info.

openwallet-foundation / acapy

Issuer uses wrong accumulator values after fixing revocation registry with `PUT /revocation/registry/{rev_reg_id}/fix-revocation-entry-state` #3281