witness: Internal Server Error when trying to query oobi for delegated multisig AID

[lenkan]

Currently, we are experiencing internal server errors in witnesses for a multisig AID that has been delegated from another multisig AID created in keripy. The error we are experiencing is this:

2024-04-17 12:42:59 [FALCON] [ERROR] GET /oobi/<multisig-prefix>/witness/<witness-prefix> => Traceback (most recent call last):
--
File "falcon/app.py", line 365, in falcon.app.App.__call__
File "/keripy/src/keri/end/ending.py", line 594, in on_get
msgs = hab.replyToOobi(aid=aid, role=role, eids=eids)
File "/keripy/src/keri/app/habbing.py", line 2007, in replyToOobi
return self.replyEndRole(cid=aid, role=role, eids=eids)
File "/keripy/src/keri/app/habbing.py", line 1962, in replyEndRole
msgs.extend(self.replay(cid))
File "/keripy/src/keri/app/habbing.py", line 1546, in replay
for msg in self.db.cloneDelegation(kever=kever):
File "/keripy/src/keri/db/basing.py", line 1307, in cloneDelegation
dkever = self.kevers[kever.delegator]
File "/keripy/src/keri/db/basing.py", line 68, in __getitem__
raise ex  # reraise KeyError
File "/keripy/src/keri/db/basing.py", line 63, in __getitem__
return super(dbdict, self).__getitem__(k)
KeyError: '<delegator-prefix>'

The impliciation of this is that no one can resolve the OOBI of our multisig AID. We have not been able to create a consistent reproduction. There is a related issue in keria that is also having problems with the same AID: https://github.com/WebOfTrust/keria/issues/231.

Does anyone have any clue of what the cause might be?

We are using keripy version: 1.1.10

[pfeairheller]

Yes, your witness has received your delegated multisig AID inception event without receiving the approving delegator's KEL. it is the controller's responsibility to ensure the witnesses have all the artifacts they need to provide a valid KEL.

The fix here is that the witness should check to ensure a delegated AID has been approved before responding to an OOBI request. In this case, the witness should return a 404 for the OOBI of the delegated multisig AID until it receives the anchoring event.

[pfeairheller]

We also need to skip events in cloneEvtMsg (by raising an exception) if a KEL is delegated and the seal does not exist

[2byrds]

Brilliant @pfeairheller thank you for the info and PR! @lenkan i'm looking forward to your update after trying it with the witness having received the delegator's KEL w/ anchoring event.

[lenkan]

Thanks for your reply and PR!

I have not yet been able to try the update. Will report back.

Yes, your witness has received your delegated multisig AID inception event without receiving the approving delegator's KEL. it is the controller's responsibility to ensure the witnesses have all the artifacts they need to provide a valid KEL.

When you say it is the controllers responsibility to ensure the witnesses have all the data. In a Signify controller + Keria agent setup, would it be the signify controller or keria agent responsibility?

Also, I recall you did something during a session we had where you did resent the KELs to the witnesses. Is this possible currently through a Keria endpoint?

[2byrds]

From our KERI dev meeting: delegators KEL must be sent to witness so that they are aware of the delegation event. KERIA hopefully has the functionality, this needs to be confirmed. On main of KERIA the ordering has changed, this was tested with 0.1.2 (latest) In that version Receiter from KERIpy... more details to follow. Can this be repaired? KERIA probably not. From command line witness submit could catch it up. Possibly 'witness submit' in KERIA could be a workaround.