Signature format

[arj03]

Background, I was looking for how to encode buttwoo signatures.

I was looking over the signature spec and currently it specifies that 0x0400 is a classic signature. But we are also using this for bendy butt. Technically those 2 can be seen as separate, so could have their own numbers. I'm wondering if we should change the definition in this spec for the signature to be just a ed25519 signature? It still needs to have the same HMAC signing capability.

[staltz]

Good find. I see what you say, that the two options are either (1) changing the semantics of the signature to be ed25519 signatures, not "classic SSB" signatures or (2) adding a new signature format for bendy butt and a new signature for buttwoo.

Option (1) is good because the bytes encoded are literally just "ed25519 signatures" and saves us from wasting our limited slots (maximum 8 formats!). Option (2) is good because it keeps the consistency with the other cases, e.g. 0x0000 "classic" 0x0003 "bendy butt" 0x0004 "buttwoo" are all ed25519, but they have their own slots. It would make sense to keep this consistency, that formats are split by semantics, not cryptographic algorithm.

Honestly, I don't know which one to pick. It's a good discussion you raised up. I'm kind of preferring (2) because I'm a consistency geek. If we fill up all 8 slots for formats, then we can come up with "bfe2" or something like that which is more extensible.

[arj03]

Also flagging this, because if we are choosing option 2, then we need to update the bendy butt implementations and specs and that would be good to do before we put it into production :-)

I'm also a bit torn on what option to choose. I do agree that for consistancy we should probably aim for option 2. Hmm (thinking out loud here), maybe feeds and messages are special. Compared to signatures, they exist in "vacuum", so when you point to a feed or message you don't have any other information and the format is important. On the other hand signatures almost always exist within the context of a message. So the format is in a sense implied. So it's more like algorithm, rather than specification.

[staltz]

maybe feeds and messages are special. Compared to signatures, they exist in "vacuum", so when you point to a feed or message you don't have any other information and the format is important. On the other hand signatures almost always exist within the context of a message. So the format is in a sense implied. So it's more like algorithm, rather than specification.

I agree with this!

Could we get more opinions? @mycognosist @mixmix ?

[mycognosist]

I also agree that contextual information encoded in the format is more important for feeds and messages.

Overall, I think that consistency (and clarity, by extension) should be our primary motivation here. I think that saves future developers the work of having to reason about categorical differences; they simply apply the same standard TF approach regardless of type. Maybe it's easier in the long run to create consistency than to justify and explain inconsistency?

[staltz]

I also agree with glyph! Overall I still feel like (2) would be better. Maybe there's an unknown use case for having all of these things consistent, and we may miss out on that if we take a shorter path this time.

[arj03]

I'm mostly in favor of removing it. It doesn't convey very much information unless the signature is found separate from the feed message. If you look at bamboo there is no yamf encoding of the signature because it is given by the key used for the feed. So it can't be changed unlike the hash function used for content hashing.

[staltz]

Removing what, like the whole Signatures table? Would we replace with 0x0603 (any-bytes)? Or what?

[arj03]

Yes removing the whole 4 section. The reason would be: 1) it's not used in classic, classic is not a binary format. 2) it is confusing for other formats, can we use this or not? What does it really mean? 3) most formats don't need this kind of BFE encoding of signatures anyway.

That is my "do what is most right" argument. I know that it is a big thing to change bendy butt to something else, for this reason a compromise could be to define it as a ed25519 signature instead of classic. I feel quite bad now having raised this point, because I don't think I have capacity to fix it myself. In the process of formulating the arguments here, I did find what I need to do in buttwoo. So we could also pretend this never happened ;-)

As for the specific of how we would encode it for bendy butt if we remove the whole section, then the spec currently says: "SSB Binary Field Encodings (SSB-BFE) is used to encode all non-list non-dictionary non-integer values contained in the payload and signature". So yeah that leaves any-bytes for the job.

[mycognosist]

Shall we have a call to discuss the work involved with making the change (if that's the option we go with)?

I'm on holiday at the moment but should have access to a computer from Monday or Tuesday.

[mixmix]

haven't read whole thread, just coming here to say - DO NOT confuse the algorithm (ed25519) with the format. The format is meant to tell you not just HOW something was signed, but WHAT that thing was - like if it's classic, then you took the JSON, stringified it, then signed everything in that. The same will not be true for every format... even if you do use ed25519.

When in doubt, mint another format.

Can't remember if this type is used by anything yet. If it's not in use it should never have been put in this spec :grumpy_face:

[staltz]

@mycognosist Sorry for not responding to your request for a call. I can't speak for arj but I can say that we (mix, arj, me) are very busy and this particular issue may have fallen out of our mental priorities.

I'm working on an ssb-db2 refactor, and I'll also touch some of the buttwoo implementation and maybe specs, so it could be that this issue becomes relevant again.

[mycognosist]

No trouble at all, @staltz. I only just got my laptop working again (have been enjoying a computer-less holiday). Give me a ping any time in the future if you want some help.