Keep in mind: ANY change is retro incompatible. Even just including a field without changing anything else. The purpose of this document is to provide alternatives for making the next modifications less impacting.
Today
Claim is a protobuf message. It can be either a stream or a certificate. Certificate holds the public key, while the claim holds metadata.
Inside that metadata, we have a signature. To validate that we:
Get the content claim.
Figure out which certificate signed it, fetch the certificate.
Extract the signature from the protobuf
Clear the signature field
4.1. Now we have the original protobuf, used for signing. This requires the whole operation to be deterministic. If one bit gets flipped, the signature won't match what is being signed.
Take the hash of that re-encoded protobuf binary serialization, check that the signature was made for this hash. Done.
The main issue
We need to be able to re-encode it deterministically in order to validate.
Issues
We depend on Google by making re-encoding deterministic
This can easily lock us on a protobuf version if serialization changes
Even being able to parse a protobuf, deterministically being able to recode it requires knowledge on the previous schema, at a minimum.
We need to know how a 10 years old claim was serialized, 10 years ago
Depending on how far ago it was signed, we would need technical knowledge on how a claim was encoded 10 years ago.
We can't verify claims from a future version, as we have no idea how that version serialized it. Even being able to parse the fields we care about.
Proposal 1 - Having the signature decoupled from the claim binary.
Instead of being a protobuf field, the signature would be appended to the protobuf binary, like:
Claim content := <protobuf binary><signature>
How to verify:
Parse the protobuf and check if has a signatures. Options for that:
1.1. check the certificate id from the protobuf. If it has a certificate id, assume it was signed and proceed to verification. If not, stop.
1.2. If not signed, add trailing zeroes. (thats not good, but I want to make sure I told all options :P)
Get the trailing X bytes. The signature is a fixed length representation.
Verify the original payload against the appended verification.
Upsides
Little modification.
Forward compatible. (You just need to parse, not to re-encode)
Downsides
Still needs to parse, as before.
lbrycrd can't do anything without knowledge of the protobuf, as before.
Proposal 2 - No on-chain claim metadata, with better off chain publishing support
How it would work:
Publish your channel certificate as usual. If you really want the name, bid it for real. Or just get your channel permanent URI.
Two scenarios:
2.1. if you want your published content to have a LBRY name or a traditional LBRY URI (with a claim id, for instance) and prove that it's on the blockchain
-- Publish the hash of the metadata + signature (on chain publishing with off chain metadata)
2.2. if only you want your published content to be cryptographically proven legit (authentic LBRY channel publish) over any protocol
-- Don't use the blockchain, just for channels (off-chain publishing)
The new on-chain format:
Many options are possible, but the main idea is that the content of the claim wouldn't be directly used, so the claim content would be:
<SHA384 hash of the claim metadata><certificate claim id><signature>
On chain publishing:
Publisher:
Publish a certificate for a channel
Publish some signed content for that channel under a new name.
Claim the blob hash and signature under a name
Client:
Fetch the certificate
Resolve an URI that gives the new format back
Get the "metadata blob" using the first hash.
Verify it
Off chain publishing (after you have a channel):
Publisher:
Use that certificate to sign new content anywhere you want
Client:
Fetch a certificate for a channel
Get new content from anywhere, use the certificate to check content signature (authenticity).
Rationale:
LBRY Blockchain provides decentralized cryptography proofs of name claims ownership under an auction.
That proof can include a content
If we put a public key under that content, we are free to extend trust. As we do for certificate claims for channels. Where you can prove cryptographically that you are the authority behind a channel.
That's more than enough! We don't need the full metadata to be signed. Just a hash, if we really want to bid another name.
We don't loose description information due claim size limits
We can use the "summary/metadata blob" to point to even richer custom metadata, which would make the content more rich in details and context. Like a movie synopsis, an image thumbnail, a full quality album cover or even a band interview from that album or a movie review sponsored by the publisher.
It could hold an optional list of mirrors, like: ipfs, http, bittorrent, lbry. Without breaking anything (if you know how to use the "mirrors" field, use it, or ignore it).
Extending trust can allow the metadata to be served from anywhere, without any other required level of trust as only the original author (private key owner) could "lie".
Content also can come from anywhere, no need to specify the sources on the metadata protobuf. If you know it can come from X, use X. If you want to advertise X supports it, add this to the metadata. Or just download from whoever gave you that metadata using the same protocol.
You can extend the LBRY Blockchain security to provide authenticity over anything. From Tweets to emails.
Security can even be extended for third party services, like transcoding (to be explained in anotehr section)
Downsides:
LBRY URI is currently tied to LBRY Blockchain.
We could add support for blobs URIs, like: lbry://@channel/<sha384 blob hash + signature>
A name could be added optionally, after publishing! Just claim my_cool_title with the claim metadata signature in it.
If we keep serving the blobs exclusively over our p2p, it will be slower. However there is no need for that as it can be served from anywhere since we have a way to verify authenticity and integrity.
My counter point on this: current claim data fits any Raspberry model RAM and the smallest instances on cloud providers. It's around 80-100mb. Could fit even some router models memory.
Yes, that's the whole claim metadata of every single valid claim.
Having the signed hashsum, anyone can serve claim information without trust.
100x is still 10gb. Still fits DigitalOcean smallest instance SSD.
If we compress, ...
It's a big change, but can also be made easier as we already work with blobs and mirrors currently anyway.
I'm biased. There should be more downsides.
Off chain services examples:
Basically, by making the metadata just like any other binary content and caring only for certificates, we are back to a model of public/private key, which is the most used model today for centralized services security. Except that now NO UNKNOWN AUTHORITY is necessary (like the root certificates or government issued signatures). Only the channel/person you already trusted in a decentralized manner over LBRY Blockchain. This allows whoever is serving the claim metadata blob to serve an infinite number of services.
Descriptions hosters
If we add a "description source" field to the certificate claim, you can edit descriptions at will and sign the edits under that source. Could be HTTP, for instance. Since integrity is not verified (just authenticity), you can provide up-to-date alternative descriptions if you want to. (hey, this is already possible"-ish". Maybe include it to the description editing issue?)
Live streams
As myself: I can claim that I will stream something at something something protocol. I can also sign the stream so you can be sure its me.
As my poor famous self celebrity: I can let my fans further broadcast my stream, as it's signed and I don't need to trust them.
As my own fan: I can sign the recorded signed stream and make a publish to watch myself over and over again, being cryptographically sure its me.
Transcoding
As the publisher:
I can transcode my videos to any format you pick, so you can watch directly at perfect quality/bandwidth rate, with subtitles, and sign the stream. So you can watch live in any device that supports video streams and I can prove I'm the author.
As a sevice:
The publisher trusts me. He signs my transcoding key and I can serve his videos in any format you want. If I misbehave, the publisher can revoke my signature. Or even clients, if we find a way to get authentic reports.
research: there may be a way to do deterministic transcoding. So the publisher can sign "transcoded blobs hashes" and nobody can lie.
CDNs/centralized services/current tech
I can serve my blobs from anywhere and anyone can serve them for me as in scalable fast centralized services. Except they can't lie/tamper/pretend its missing and don't need to be trusted.
"Optional options" for reducing the new format to 384 bits:
Claim the blob hash of the metadata blob content concatenated with the signature (proposal 1)
Claim the blob hash of a blob with signature+cert_id+metadata_blob_hash
2.1 adds a layer of indirection, so I prefer 1, but maybe reducing to 384 isn't a gain at all. Something for when we hit severe fee issues, maybe.
Proposal 3 - New transaction type
It was discussed something about creating a new type of transaction with the signature attached. It could be a push data or a whole new type of transaction. I will list why I don't like it, but I'm not a good fit to defend it as I think it's a bad idea.
How it would work (from what I understood):
Two new transaction fields: certificate_id and signature
How to verify:
extract the payload
extract the signature
extract the certificate id
get the certificate and verify
Upsides
lbrycrd/lbryumx can parse it even having no clue on what a protobuf is
it would be as simple as parsing transaction signatures
Downsides
Requires deep lbrycrd modifications and is very delicate/hard to do as it requires a new transaction type
Requires that to be reimplemented on the wallet and wallet server
May require a hard fork (if old nodes reject the new format)
Proposal 4 - Having the claim protobuf versioned
We already have a version for each part of the protobuf. Extend that so we know which version generated the signature.
I can't comment on that as I don't know how it would solve the problem. I believe we would be able to tell that we can't verify the signature because we don't know that schema, instead of just failing. However it would still have the main issue of forward compatibility.
lyoshenka
commented
5 years ago
Claim signatures
Keep in mind: ANY change is retro incompatible. Even just including a field without changing anything else. The purpose of this document is to provide alternatives for making the next modifications less impacting.
Today
Claimis a protobuf message. It can be either a stream or a certificate. Certificate holds the public key, while the claim holds metadata. Inside that metadata, we have a signature. To validate that we:The main issue
We need to be able to re-encode it deterministically in order to validate.
Issues
Proposal 1 - Having the signature decoupled from the claim binary.
Instead of being a protobuf field, the signature would be appended to the protobuf binary, like:
Claim content := <protobuf binary><signature>How to verify:
Upsides
Downsides
Proposal 2 - No on-chain claim metadata, with better off chain publishing support
How it would work:
The new on-chain format:
Many options are possible, but the main idea is that the content of the claim wouldn't be directly used, so the claim content would be:
<SHA384 hash of the claim metadata><certificate claim id><signature>On chain publishing:
Publisher:
Client:
Off chain publishing (after you have a channel):
Publisher:
Rationale:
Upsides:
Downsides:
lbry://@channel/<sha384 blob hash + signature>my_cool_titlewith the claim metadata signature in it.Off chain services examples:
Basically, by making the metadata just like any other binary content and caring only for certificates, we are back to a model of public/private key, which is the most used model today for centralized services security. Except that now NO UNKNOWN AUTHORITY is necessary (like the root certificates or government issued signatures). Only the channel/person you already trusted in a decentralized manner over LBRY Blockchain. This allows whoever is serving the claim metadata blob to serve an infinite number of services.
Descriptions hosters
If we add a "description source" field to the certificate claim, you can edit descriptions at will and sign the edits under that source. Could be HTTP, for instance. Since integrity is not verified (just authenticity), you can provide up-to-date alternative descriptions if you want to. (hey, this is already possible"-ish". Maybe include it to the description editing issue?)
Live streams
As myself: I can claim that I will stream something at something something protocol. I can also sign the stream so you can be sure its me. As my poor famous self celebrity: I can let my fans further broadcast my stream, as it's signed and I don't need to trust them. As my own fan: I can sign the recorded signed stream and make a publish to watch myself over and over again, being cryptographically sure its me.
Transcoding
As the publisher:
CDNs/centralized services/current tech
I can serve my blobs from anywhere and anyone can serve them for me as in scalable fast centralized services. Except they can't lie/tamper/pretend its missing and don't need to be trusted.
"Optional options" for reducing the new format to 384 bits:
Claim the blob hash of a blob with signature+cert_id+metadata_blob_hash2.1 adds a layer of indirection, so I prefer 1, but maybe reducing to 384 isn't a gain at all. Something for when we hit severe fee issues, maybe.Proposal 3 - New transaction type
It was discussed something about creating a new type of transaction with the signature attached. It could be a push data or a whole new type of transaction. I will list why I don't like it, but I'm not a good fit to defend it as I think it's a bad idea.
How it would work (from what I understood): Two new transaction fields:
certificate_idandsignatureHow to verify:
Upsides
Downsides
Proposal 4 - Having the claim protobuf versioned
We already have a version for each part of the protobuf. Extend that so we know which version generated the signature.
I can't comment on that as I don't know how it would solve the problem. I believe we would be able to tell that we can't verify the signature because we don't know that schema, instead of just failing. However it would still have the main issue of forward compatibility.