[WIP] proto3 proposal

[lyoshenka]

My proposed schema changes. this is very much WIP. I just wanted to see what it might look like and get your opinions on it.

Goals:

• upgrade to proto3
• drop Version from all protobuf definitions. One of the core ideas of protobufs is that versioning is not necessary
• rename and rearrange some of the data to better match our current terminology and understanding of the protocol
• add new fields that we know we need (releaseTime, file fields, tags, etc)

Keep in mind that in proto3, every field is optional. Also keep in mind that I'm trying to match this stuff to https://spec.lbry.io/#specification (e.g. Stream.hash vs Stream.sd_hash)

When you look at the field numbers, anything numbered 1-15 means we think that field will be set in most claims. Numbers 16+ means we think that field will not be set in most claims. There's no functional difference other than a small space savings if we predict it correctly.

I think the biggest questions are about the structure of the stream.proto file.

Closes #3, closes #16, closes #8, closes #13, closes #6, closes lbryio/internal-issues#205.

[kauffj]

Is there anything else I should do to test or run this? We also have a semi-long list of fields to add. Would it make sense to add those before merging in case some should go in the top 15?

[lyoshenka]

putting this here so i don't forget: https://github.com/lbryio/types/issues/8

[lyoshenka]

related: https://github.com/lbryio/types/issues/15

[lyoshenka]

@kauffj yes, link any fields that need to be added here. i found a few PRs but not sure i got em all

[shyba]

great, maybe make a backup of the current one like /proto_legacy/? The current proto2 one is still needed for decoding claims.

[lyoshenka]

Questions re: channel metadata

• what does Name mean? don't channels already have a name? is this like the difference between username and display name?
• should cover and thumbnail be web URLs or LBRY URLs or claim IDs?

[lyoshenka]

Question for reviewers:

• are any fields still missing?
• are any fields unnecessary?
• should the fields be structured differently?

[neb-b]

What about a website/contact field? Most sites have something for you to add your email address or link to a website. I'm not sure if both would be valuable, or if you could just merge it into one field.

[kauffj]

:+1: on adding both a website and an email field to channels @lyoshenka

[eukreign]

Question Responses:

1. I think name is the display name. Slightly less ambigious (not sure by how much) might be to use title insead of name, but I think name is fine too.
2. cover and thumbnail should be claim IDs

Feedback:

• thumbnail_url -> thumbnail
• cover_url -> cover
• using float for the Fee is a bad idea. It should be an integer and then the precision is 2 decimals for fiat currencies (USD) and 8 decimals for crypto currencies (LBC/BTC). ($1USD would be 100 and 1LBC would be 100000000).

New Questions:

• Does Channel make sense for certificates if we're planning to also use them for a users identity? I think the old Certificate is more generic when applying to both channels and identities.
• Can they all go in one file? They are pretty short and it's easier to get the full picture with just one file open.
• Stream vs. File is confusing. Are there Streams which aren't Files?
• Insead of having Claim container type with Channel and Stream inside of it, can we just have Channel and Stream as root containers (and get rid of Claim)? (And perhaps add other root objects like Comment in the future?)

[tzarebczan]

I don't think I should have to download thumbnails/covers over DHT, unless the long term plan is to allow that for claim thumnails as well (https://github.com/lbryio/lbry/issues/1842)

[neb-b]

I agree. I do not think thumbnail/cover should be a claim id.

I can see a case where it is a url or a claim id (or something else), but since thumbnail is already a url for a claim, this should follow the same behavior.

[kauffj]

@seanyesmunt note that as long as we encourage usage of spee.ch for images, we are encouraging thumbnail/cover to be a claim id just with extra steps ;)

[kauffj]

Another feature that just struck me as missing is creator identity verification (where verification equals proving I own a certain Twitter account, YouTube account, domain name, or ??). You would prove you own relevant property by issuing a tweet, publishing a video, adding a DNS entry / uploading a file, etc.

This is more complicated than basically everything else we're proposing to add, so I don't want to block on this, but if we can come up with a generic format or nail down a few of them specifically, it'd be good to include.

[kauffj]

@lyoshenka is anything blocking this?

[lyoshenka]

i'm dropping the ability to list payments in BTC. we're just keeping usd and lbc.