OpenSL Capture: First 0.5 Seconds Samples Seems Corrupted

[MikuAuahDark]

Recently, I received more complains about audio recording being corrupted in LÖVE for Android. I can reproduce the audio corruption in my phone too and it seems happen on first 0.5 seconds of the capture.

Sadly I can't reproduce this with alrecord example program (it just writes empty captures for some reason). I also can't reproduce this in my laptop running Windows, so this is probably Android-specific. However LÖVE code to start and stop audio capture is just thin wrapper around OpenAL's alcCapture* functions.

To reproduce, install the debug APK (please rename the extension) with test mode flag using ADB. Run LÖVE for Android once, which should gives you the balloon screen (and optionally may prompt your for external storage access which you can deny).

Afterwards, copy these files to Internal Storage/Android/data/org.love2d.android/files/save/lovegame. Create the folder if it doesn't exist:

conf.lua

function love.conf(t)
    t.audio.mic = true
    t.window.fullscreen = true
end

main.lua

local love = require("love")
local recDev = nil
local isRecording = -1
local audioSource = nil
local devicesString
local recordingFreq, recordingChan, recordingBitDepth

-- Possible combination testing
local sampleFmts = {48000, 44100, 32000, 22050, 16000, 8000}
local chanStereo = {2, 1}
local bitDepths = {16, 8}

function love.load()
    local devices = love.audio.getRecordingDevices()
    assert(#devices > 0, "no recording devices found")

    recDev = devices[1]

    local devStr = {}
    for i, v in ipairs(devices) do
        devStr[#devStr + 1] = string.format("%d. %s", i, v:getName())
    end
    devicesString = table.concat(devStr, "\n")
end

function love.update(dt)
    if isRecording > 0 then
        isRecording = isRecording - dt

        if isRecording <= 0 then
            -- Stop recording
            local soundData = recDev:getData()
            isRecording = -math.huge
            audioSource = love.audio.newSource(soundData)
            recDev:stop()
        end
    end
end

function love.mousereleased()
    if isRecording == -1 then
        -- start recording
        local success = false
        -- Test all possible combination
        for _, sampleFmt in ipairs(sampleFmts) do
            for _, bitDepth in ipairs(bitDepths) do
                for _, stereo in ipairs(chanStereo) do
                    success = recDev:start(math.floor(sampleFmt * 5.1), sampleFmt, bitDepth, stereo)

                    if success then
                        recordingFreq = sampleFmt
                        recordingBitDepth = bitDepth
                        recordingChan = stereo
                        isRecording = 5
                        print("Recording", sampleFmt, bitDepth, stereo)
                        return
                    end

                    print("Record parameter failed", sampleFmt, bitDepth, stereo)
                end
            end
        end

        assert(success, "cannot start capture")
    elseif isRecording == -math.huge and audioSource then
        if audioSource:isPlaying() then
            audioSource:pause()
        else
            audioSource:play()
        end
    end
end

function love.keyreleased(k)
    if k == "escape" then
        love.event.quit()
    end
end

function love.draw()
    if isRecording == -1 then
        love.graphics.print("Recording ready", 2, 2)
    elseif isRecording == -math.huge then
        love.graphics.print("Replay "..audioSource:tell(), 2, 2)
    else
        love.graphics.print("Recording at "..recordingFreq..", "..recordingBitDepth..", "..recordingChan..": "..isRecording, 2, 2)
    end

    love.graphics.print(devicesString, 2, 16)
end

After 2 of those files copied, start LÖVE for Android again and it should ask for microphone permission. Allow it. At this point, tap the screen once to start recording then wait for 5 seconds. Afterwards, tap the screen again to replay the captured samples.

If you have any problems reproducing it, please let me know. I performed my tests on commit 8c4adfd752aa25a4d6d49db870f88287858bc5fc.

[kcat]

I can't see what would cause this. However, I made a recent change with commit 2fb7538f6860086a8b6dbfbcf983ed2dffb71edd to initially clear the buffers in case OpenSL wasn't properly clearing them when starting capture. Also, commit a42fe862c54bcaa4cfe39cfcf4de8a4c0f3eab14 should bring the Oboe backend to OpenAL API conformance, so you can try a build with the Oboe backend instead of the OpenSL backend and see if that improves the issue.

[MikuAuahDark]

Alright. I'll try in next week.

[MikuAuahDark]

Hello, sorry for taking long time to test.

The Oboe capture solves the issue, but it seems to be very picky about the buffer size. In my phone, anything beyond 2048 buffer size will be rejected for some reason and I see these.

2022-05-28 17:59:02.736 19048-19189/org.love2d.android W/openal: [ALSOFT] (WW) Failed to open capture device: Failed to set large enough buffer size (16384 > 16320)
2022-05-28 17:59:02.736 19048-19189/org.love2d.android W/openal: [ALSOFT] (WW) Error generated on device 0x0, code 0xa004
2022-05-28 17:59:07.947 19048-19189/org.love2d.android W/openal: [ALSOFT] (WW) Failed to open capture device: Failed to set large enough buffer size (8192 > 8160)
2022-05-28 17:59:07.947 19048-19189/org.love2d.android W/openal: [ALSOFT] (WW) Error generated on device 0x0, code 0xa004
2022-05-28 17:59:10.703 19048-19189/org.love2d.android W/openal: [ALSOFT] (WW) Failed to open capture device: Failed to set large enough buffer size (4096 > 4032)
2022-05-28 17:59:10.703 19048-19189/org.love2d.android W/openal: [ALSOFT] (WW) Error generated on device 0x0, code 0xa004

Looks like the buffer size reported for anything beyond 2048 is slightly just below the requested buffer size. My recording test app is set to test 2^n buffersize where n = 14 to n = 10.

OpenSL capture on the other hand, seems broken. I'm getting spammed with this

2022-05-28 18:03:56.707 19816-19887/org.love2d.android E/openal: [ALSOFT] (EE) bufferQueue->Enqueue: Buffer insufficient
2022-05-28 18:03:56.755 19816-19887/org.love2d.android W/libOpenSLES: Leaving BufferQueue::Enqueue (SL_RESULT_BUFFER_INSUFFICIENT)

And no audio is recorded.

Tested using version 1.22.0

PS: Also the find_package(Oboe) doesn't work at all. AAR-compiled Oboe doesn't get detected unless I add CONFIG. Furthermore the detection of if(OBOE_FOUND) doesn't work for some reason and I have to use if(TARGET oboe::oboe) instead.

[kcat]

OpenSL capture should finally hopefully work now with commit d7367aeee04764ad7595ef3f5de8e01b7a6d93ad. Oboe capture should also work with larger sizes now with commit 7f1d9725b2df68664e44fbd9944241ab3097a270.

find_package(Oboe) not working I'm not too surprised about, since it doesn't seem Android has some kind of system for installing packages that your other projects can pick up and use. Although I don't know why if(OBOE_FOUND) wouldn't work when it is found, given that the find module only defines the oboe::oboe target when OBOE_FOUND is true. Unless the CONFIG is making it use something else that defines the target without OBOE_FOUND.

[MikuAuahDark]

Looking at the dumped oboeConfig.cmake, it's indeed didn't set the OBOE_FOUND variable, thus I think the correct way to check for its existence is by doing.

find_package(Oboe CONFIG)
if(NOT TARGET oboe::oboe)
    find_package(Oboe)
endif()
if(TARGET oboe::oboe)
    set(OBOE_TARGET "oboe::oboe")
endif()

Anyway thanks, I'll try again ASAP.

[MikuAuahDark]

Thank you. Both patches solves the recording issue both in OpenSL and Oboe.