Process video in TEE

[otytlandsvik]

We want to process video in the tee so that we can attest it.

• Transfer video data from the optee client to the ta
  • We can use some sort of library to read video frames into a nice struct
• Do an arbitrary operation in the ta on the video
  • Maybe grayscale, maybe just incrementing pixel values by one for now
• Send video back to the client
• Client must save video back into a file

[asbjaare]

Suggestions for video processing.

1. Video Frame Extraction: Implement a system that reads video files and extracts specific frames at regular intervals. This task involves reading and decoding video data, which can be computationally expensive and show clear distinctions in performance between CPU and GPU processing.
2. Greyscale Conversion: Convert the color video frames to greyscale. This is a straightforward operation where you convert each pixel's RGB values to a single brightness value. This operation can be easily parallelized, making it suitable for GPU acceleration, and you can compare this to CPU processing within a TEE.
3. Gaussian Blur: Apply a Gaussian blur to video frames. This involves a convolution operation, which is more computationally intensive than simple frame extraction or color conversion. It's a good test case for demonstrating GPU strengths in handling parallel computations.
4. Edge Detection: Implement a simple edge detection algorithm like the Sobel operator. This process involves gradient calculations on the image pixels and is another good candidate for GPU optimization. It's more complex than greyscale conversion but still manageable in C.
5. Motion Detection: Simple motion detection between video frames can be implemented by comparing pixel differences or using more sophisticated techniques like background subtraction. This could be an interesting way to explore how the two processing environments handle more complex logic.
6. Video Resizing (Scaling): Implement a basic video scaling algorithm that changes the resolution of the video frames. This could involve nearest-neighbor or bilinear interpolation. Scaling is a common task in video processing that could benefit from GPU acceleration.
7. Frame Rate Conversion: Change the frame rate of a video by interpolating frames or skipping them. This involves temporal interpolation and could show how well each processing unit handles time-based video processing tasks.