Uploading images on iOS

[adamtow]

I'm trying to upload a PNG image to the Media Pool on iOS but am getting stuck with what I believe to be the correct value to set for the uncompressedSize parameter to uploadStill. See my code below:

The resizeImage function just resizes the image to fit within a 1920x1080 box. The same problem occurs if I try to send the original UIImage.

Looking at your TitleGenerator example, you're setting the size of the uncompressedData to be 1920x1080x4. When I do this, or use the actual byte count from the PNG data, the ATEM gets stuck and unresponsive to future uploads.

Any thoughts on what might be happening?

func uploadStill(at index: Int?, image: UIImage) {
    guard let slot = index,
        let resizedImage = self.resizeImage(image: image, targetSize: CGSize(width: 1920, height: 1080)),
        let pngData = resizedImage.pngData()
    else { return }

    let imageSize: CGSize = resizedImage.size
    let width = imageSize.width
    let height = imageSize.height

    let numBytes = UInt32(pngData.count)

    let yuvData = Media.encodeRunLength(rgbData: pngData)
    let numYuvDataBytes = yuvData.count

    let size: UInt32 = UInt32(width * height * 4 * resizedImage.scale)

    // controller should be defined outside of this method. Neither of these two methods work.

    controller.uploadStill(slot: UInt16(slot), data: yuvData, uncompressedSize: numBytes)

//  controller.uploadStill(slot: UInt16(slot), data: yuvData, uncompressedSize: size)
}

[Dev1an]

Hey Adam, I have no ATEM at my disposal anymore but maybe I can think along and help you.

Uncompressed size should be the number of bytes of a full resolution (depends on the resolution of your ATEM) uncompressed YUV image. (That is: Alpha1 Blue Lum1 Alpha2 Red Lum2, with 10bits for each color channel). This is the only format that the ATEM accepted at the time I wrote the code. Hence the size should always be targetResolution * 4.

I see that you are using pngData as a source to feed encodeRunLength(rgbData:). I don’t know what pngData exactly is but I suspect that it is the compressed PNG data, which is the wrong format. That function requires raw 8bit RGBA data. That is a long string of 32 bit values representing pixels, the pixels are encoded as 4 consecutive 8bit premultiplied color values: red, green, blue, alpha.

[adamtow]

Thanks. This will point me in the right direction.

Follow up question. What is your plan for the code base now that you don’t have an Atem anymore? Aside from implementing more of the commands, were there sections that remained unfinished and in need of improvement?

[Dev1an]

I'm a software engineering student and I have close contacts with a broadcasting company. From time to time I get some equipment from them to take home and develop some software. This is all done in my spare time. And it just happened so that for the moment I have no equipment at home. But I still intend to expand this library and to continue to improve it.

The broadcasting company is using a lot of automation software that I wrote in javascript and runs in a nodeJS engine. This project is the start of a transition that I would like to make: from javascript to Swift.

were there sections that remained unfinished and in need of improvement?

These are some things that remain undone but that I would like to finish:

• Merge your pull request ;)
• Update the documentation
• Add Codable support for the ATEM commands to make it more easy to add new ones.
• Use native buffers from SwiftNIO instead of [UInt8] arrays
• Do some testing:
  • Add endurance tests using real hardware
  • Automated testing: test my simulator against the provided libraries from black magic design.

[Dev1an]

What are you using this library for? Do you already have something in production?

[adamtow]

Been working on a paid iOS app to control Atem devices. Getting close (I hope) to release, but there’s a lot of features to support in these devices so the scope keeps increasing. I only have an Atem Mini Pro at the moment. The simulator has been useful to at least see how other features might be represented in my app.

[adamtow]

I got this to work thanks to your pointers. The key part was making sure that the image is resized to fit and drawn into a 1920x1080 rectangle (or whatever the switcher is outputting). This image is then converted to RGBA data object which is then sent to the Media.encodeRunLength function.

Using this code to do the resizing:

func resizeImage(image: UIImage, targetSize: CGSize) -> UIImage? {
    let targetRect = CGRect(x: 0, y:0, width: targetSize.width, height: targetSize.height)
    let drawRect = AVMakeRect(aspectRatio: image.size, insideRect: targetRect)
    let format = UIGraphicsImageRendererFormat()

    format.opaque = false
    format.scale = 1.0
    format.preferredRange = .standard

    let renderer = UIGraphicsImageRenderer(size: targetSize, format: format)

    return renderer.image { (context) in
        image.draw(in: drawRect)
    }
}

and this UIImage extension to extract the pixel information:

extension UIImage {
    func pixelData() -> [UInt8]? {
        let size = self.size
        let dataSize = size.width * size.height * 4
        var pixelData = [UInt8](repeating: 0, count: Int(dataSize))
        let colorSpace = CGColorSpaceCreateDeviceRGB()
        let context = CGContext(data: &pixelData,
                            width: Int(size.width),
                            height: Int(size.height),
                            bitsPerComponent: 8,
                            bytesPerRow: 4 * Int(size.width),
                            space: colorSpace,
                            bitmapInfo: CGImageAlphaInfo.noneSkipLast.rawValue)
        guard let cgImage = self.cgImage else { return nil }
        context?.draw(cgImage, in: CGRect(x: 0, y: 0, width: size.width, height: size.height))

        return pixelData
    }
}