Verification Workflow

[ericxtang]

Verification Workflow

The verification module assumes the following workflow:

source

Transcoder

The transcoder is a GPU running on a machine. We assume the verification module is shipped as a part of the code running on the GPU. The verification module is invoked for every transcoded segment, resulting in an array of values for validation and a single value for ranking. This list of values is published with the transcode result.

Note: We should make sure this assumption is viable. What if the machine running the GPU is a stripped down crypto mining rig that may not have the ability to run the verification module?

Verifier

In the ideal world, the verifier would be a Truebit module running on the Ethereum chain. There is a prototype built with a modified version of ffprobe, but we haven't ran it in production yet. In the mean time, we can assume it's a docker instance running on a cloud hosted instance with access to Ethereum.

The verifier invokes the quality measurement module, and compares the result with the input validation values (created by the transcoder). The verification reports fail if the values are different.

Protocol

Smart contracts running on the Ethereum blockchain. The transcoder needs to monitor the challenge event in order to invoke the verification request.

Object Store

We can assume it's a S3 bucket with public access.

[j0sh]

The transcoder is a GPU running on a machine. We assume the verification module is shipped as a part of the code running on the GPU.

I was under the assumption that verification would be a decode-time process by checking the output from the transcoder (which may be running on a GPU). Here, it appears that verification becomes an encode-time process that's run on a transcoder.

Can that be elaborated as far as the expected additional workload, how to handle trust, etc?

Personally, I think it'd be much preferable if verification was a post-decode process. This is both faster since it allows us to check more data at less cost (decoding much easier than encoding), and is agnostic to the choice of encoder, decoder or codec.

This quality score is published with the transcode result.

Perhaps I'm misunderstanding here. I assume the broadcaster already has a certain level of quality it expects, which has been agreed on as part of the job. I'd expect the broadcaster (or any non-transcoder verifier) to be able to independently derive this quality score given the transcoded result and the original source video. If the derived score is less than the expected score, then the segment is a candidate to be challenged on-chain (or slashed under the current protocol). So I'm not sure what publishing the score actually gets us, if the score should be derived independently given the transcode results.

[lu-zero]

The Quality Measurement module produces two kind of information:

• per-frame quality measurement
• per-sequence ranking value

The module needs to decode the encoded data and compare by a number of different metrics the source (reference) frame and the decoded frame matching it from the just encoded segment.

Perhaps I'm misunderstanding here. I assume the broadcaster already has a certain level of quality it expects, which has been agreed on as part of the job. I'd expect the broadcaster (or any non-transcoder verifier) to be able to independently derive this quality score given the transcoded result and the original source video.

The information indeed can be derived by the encoded sequence and the source sequence and it costs a decode + the quality measurement computation.

[luctrudeau]

@ericxtang looking at the diagram, the Verifier cannot obtain the original sequence from the Transcoder. A malicious Transcoder could send an altered original sequence.

The Verifier must obtain the original sequence from a trusted authority. For example:

source What is the point of entry of the original sequence in your system?

[j0sh]

@luctrudeau The source is signed by the broadcaster, and the verifier can check the signature. The source hash/signature is signaled as part of the verification request. So the source content can come from anywhere, including an object store.

Currently, the source is made available via IPFS from the transcoder. However, that still presents issues with data availability and withholding. Namely, under the current scheme, verifiers are powerless if the data is not available, which would be an issue no matter where the source or results are hosted.

The flow Eric posted is a sketch of how the system currently works as-is. It's certainly imperfect, and we're working to fill the gaps, including incentives to make data available for verification. Object store will definitely have a role in that.