Open dob opened 6 years ago
Our transcoder and segmenter already have single entry points in C, so wrapping those up for use by the Truebit VM would hopefully be as straightforward as it gets, and likely the same amount of work for a 'simpler' program with ffmpeg and friends.
Without knowing too much about Truebit, the present challenge seems to be in getting ffmpeg built for wasm and other external considerations such as I/O. There are some other folks that have attempted the feat, which we can look to: https://github.com/bgrins/videoconverter.js/issues/43
After some discussion with the Truebit, we ended up with the following goals for an initial proof of concept:
ffprobe -i <file>
command that outputs the video/audio metadata for a given input video segment. The metadata would include fields such as bitrate, framerate, resolution,etc. The binary would then return a video profile identifier that matches the computed metadataThe workflow of the proof of concept would look something like the following for video segment 1:
codeHash
in the JobsManager contractverify()
in the JobsManager contract with the IPFS hash for video segment 1 and also sends a payment to fund the Truebit taskcodeHash
, the IPFS hash for video segment 1 and the hardcoded video profile. Step 3 and this step constitutes a user acting as a task giver in the Truebit systemcodeHash
, runs it using https://github.com/TrueBitFoundation/ocaml-offchain and the data pulled from IPFS using the IPFS hash for video segment 1receiveVerification()
function with the video profile identifier output generated from the WASM binaryWhen the proof of concept is completed, it will ideally follow the above flow, but as a first step in that direction we can set up the contract scaffold in a repo so that the Truebit team can understand what our contract interface looks like and how we can include the Truebit contracts into that repo. Separately, we can create a wrapped WASM binary that can accept data input when run through the off-chain interpreter.
Here's a diagram showing the high-level flow.
We have a new dispute resolution layer interface that can help decouple some of these pieces. Specifically the receiveVerification callback in Step 7 isn't great. It is more of a relic from scrypt-interactive.
interface IDisputeResolutionLayer {
function status(bytes32 id) public view returns (uint); //returns State enum
function commitChallenge(address solver, address verifier, bytes32 spec) public returns (bytes32 gameId);
}
It seems the Livepeer contract is acting as what we would call the IncentiveLayer, but I guess in future versions this would have its own box at some point. Essentially, the IncentiveLayer can expect to interact with a dispute resolution layer that implements the above interface. Ideally, even this detail will be abstracted away by Truebit, and the Livepeer contract can simply interact with it.
Instead of the callback, the dispute resolution layer provides a status code for the verification game which is denoted by an Enum
enum State { Uninitialized, Challenged, Unresolved, SolverWon, ChallengerWon }
Then your "Incentive Layer" acts upon (slash or don't slash) based on the status code.
commitChallenge is fairly straightforward it states who is playing the verification game, and a hash of the parameters of the verification game, and then it returns a gameId (commitChallenge is called in Solidity).
This way a Livepeer transcoder is simply running a process to track all of the tasks it has started, and deal with them in a concurrent fashion.
@hswick Thanks for the update! I like this new interface.
In the case where the Livepeer contracts are used as a custom incentive layer, the transcoder could play the role Solver and if the verification game resolves to SolverWon
at the dispute resolution layer nothing would happen within the Livepeer protocol, but if the verification game resolves to ChallengerWon
at the dispute resolution layer, another actor (likely the challenging verifier that won the game) would trigger the slashing function in the Livepeer contracts that would check the status of the game and the transcoder would be slashed within the Livepeer protocol.
@yondonfu that is a good understanding of how the interface works. I'm not sure if not slashing the challenger is a good idea though. There would be no consequences to people spamming verification games. The typical Truebit incentive layer does slash the verifier if they lost the game.
Edit: Didn't realize Livepeer had its own slashing as part of its own protocol. So it looks like you all wouldn't be using the IDisputeResolutionLayer interface, but rather a IIncentiveLayer interface that we need to work on.
I wanted to kick off this thread so we can plan what we'd like to accomplish with the Truebit team on the Livepeer/Truebit prototype work on February 21st and 22nd. (Related to #1)
Goal: Get a working prototype of verification of transcoding work using a TruebitVerifier working.
Challenges
I think that for 3+4 to be accurate and production ready is far beyond the scope of the 2-3 days that we spend together.
My initial instinct would be to consider the prototype successful if we can do verification of work on some reduced Job binary, which has something to do with the underlying libraries that we use for transcoding. So for example, we write a simple C program that links with the underlying library that does some sort of much more trivial transcode job, like just rescaling the video from the input profile to one single target profile.
If we can compile this for the Truebit VM and run a version of our protocol that accepts this option as the only transcode profile, and then verifies it correctly, that would be a HUGE win, and good starting point for the collaboration going forward.
Open to other ideas and thoughts as to what we need to do to plan for this. @j0sh what do you think about this goal, and potentially preparing a much more simple "program" for us to use as the sample transcoding job as we try the first integration?