No audio after 1st loop

[bouncyball-git]

While playing back the clip with loop (never ending playback) turned on audio plays only during 1st loop then there is a silence :) Is that expected behavior?

[masc4ii]

No, this is not expected. For me it (mostly) works. But sometimes strange things happen with audio. And I also know why... it is very bad implemented :-D I have plans how to make it better, but I need some time to do so! Thanks for remind me! ;-)

[masc4ii]

Okay... I was wrong. Does not work for me too. I'll debug it. Quick fix does not work yet :(

[masc4ii]

Good. Or not good. I have to reimplement it when I want to solve it :-( It is not possible to set the stream again to pos=0. So this all has to be very different to now. But it was thought as proof of concept anyway...

[masc4ii]

Please try it out. Audio playback completely rewritten.

[bouncyball-git]

Now audio works but is out of sync (delayed) after 1st loop. Sometimes it catches up and gets in sync on some loops, very strange :), but most of the time is delayed though. I'm talking about Linux here.

[masc4ii]

Hehe, okay, I'll look for that! Thx.

[masc4ii]

Yes, I think I found the bug. See latest commit. Hopefully the delay is better (or even away).

[bouncyball-git]

Maybe it's on Linux only but sync is off and now some loops are silent again. If i press the back to 1st frame button slowly it is working if I press it quickly then silence kicks in. Maybe just set some time out before starting the playback again and it sorts things out?

[masc4ii]

Hm... the "out-of-sync-problem" is strange. Ok - exactly correct it can nearly never be, because audio and video is realized independent. And the slower the frame rate, the bigger the sync delay. But it should never be silent. That is not good. I added some lines, please try again. And pressing 1st frame button was not implmented in reference to audio playback (did that now). Thx again!

[masc4ii]

New idea: audio is now exactly synced when drawing the frame. Negative site: audio playback starts on first frame redraw, not when pressing play button. Hope that is okay. But this should also be good for slow computers. Checked in into repos.

[bouncyball-git]

Can not compile: [qrc_ressources.cpp] Error 1 Am I missing something?

[masc4ii]

Sry. Now you can.

[bouncyball-git]

• RetinaIMG/Actions-color-picker-grey-icon.png
• RetinaIMG/Actions-color-picker-icon.png

files missing

[bouncyball-git]

Yes compiled: but silence almost all the time :-/ on some loops enabled from the beginning 3-4 loops silence.

Very Very strange behavior

[masc4ii]

Okay... I can't understand that... :-( I think I need a Linux now.

[bouncyball-git]

Does it work on mac as expected? The bad thing is that now if I play without loop enabled there is no sound at all :(

[masc4ii]

Yapp. 100%. Can loop it 20x and there is always audio.

[bouncyball-git]

The bad thing is that now if I play without loop enabled there is no sound at all :(

[masc4ii]

Maybe (if you like) try to debug it. The AudioPlayback class is very easy. It can load and unload the audio, start, stop and jump to position (in dependency to the frame number). Unload is automized, but you could also do it manually. So for playing audio: load, jump to position and then play should bring playback. You could also do lines like: qDebug() << "some text" << variableX << ... ; for easy debugging, and you see that in the Application output window of creator.

Edit: to fullfil a jump you'll need a stop() before (at least on my computer). Everything is called from MainWindow.cpp.

Edit2: MainWindow.cpp has in line 166: //qApp->processEvents(); Does it work if you coment that in? Playback should be slower, but maybe you'll have audio?!

Edit3: Adding in AudioPlayback.cpp: qDebug() << "AudioState:" << m_pAudioOutput->state(); will give you what the audio engine is doing in the QtCreators app output window.

[bouncyball-git]

Thank you I'll try this today :)

[masc4ii]

Checked on Windows7Pro: no problem.

[bouncyball-git]

After commit 04e0ad640e4872a73336e49dd03a118bee980947 audio is not working on Linux. Also m_pAudioPlayback->jumpToPos( ui->horizontalSliderPosition->value() ) never worked properly. No matter which commit I'd roll back.

QAudioDeviceInfo info(QAudioDeviceInfo::defaultOutputDevice());
    if (!info.isFormatSupported(format)) {
        qWarning() << "Default format not supported - trying to use nearest";
        format = info.nearestFormat(format);
    }
    foreach (const QAudioDeviceInfo &deviceInfo, QAudioDeviceInfo::availableDevices(QAudio::AudioOutput))
        qDebug() << "Device name: " << deviceInfo.deviceName();

Used this code to check supported mode of devices and their names. Seems all is ok and QT uses pulseaudio plugin on linux.

I'm lost. Even when audio worked before this commit - actually it kinda worked, loosing sync, muting when jump to pos and muting for no reason randomly.

[masc4ii]

Really sorry. Is that a Qt bug on Linux? That works so good on Windows & OSX... In the end, I don't understand why anything worked before, because the function calls are nearly identical.

[bouncyball-git]

Yes I know you switched to m_tryToSyncAudio variable and sync is used only in drawFrame. But as I already said the audio never worked under Linux as expected. Now it's not working at all and this is unfortunately expected ;). There is one project on github which adds support of jackaudio to QT. Maybe I'll try it someday. JACK Audio is for pro audio low latency stuff.

[masc4ii]

That it did not work when you opened the bug was expected. I loaded and started audio when pressing "play" and stopped audio when pressing "stop". So it was expected, if you use a loop, that there would be no audio. That's why I decided to implement an audio class, which has audio loaded the whole time and can jump on any time to any position. I am sad that this does not work on Linux, what is strange, because the used Qt methods are the same.

[bouncyball-git]

I liked new audio class. It's implemented the way it should be. I think the big mess in audio field of Linux is the reason (many sound systems). However, if you'll have time just try it on Linux, maybe you're gonna find something what I missed.

[seescho]

@bouncyball-git

Succeded to give it a short try.

Here in Linux openSUSE I cannon hear sound on mlv-app playback, too. MLV App playback sends a pulseoudia-stream, which is recognized by my system, but I cannot hear anyhing. Screenshot:

https://seescho.files.wordpress.com/2017/10/pulseaudio.jpg

Exporting to avi gives a avi-file with sound.

[masc4ii]

Thanks! Very interesting! Did you enable "Export Audio" in Preferences? If yes, this sounds like a problem with loading the audio from the mlv file. Did you recognise a .wav file when exporting at the path where the .avi should be generated? (the wav is deleted when ready) Try to copy this wave and hear if it is okay, please! Tried exporting a avi with sound on OSX - no problem.

[seescho]

Export to avi is OK with sound. No problem. The problem is on playback. I will try to nail down the commit, which causes this issue.

[masc4ii]

Ah sry. I read "avi-file without sound". My fault. I know which commit brings the problem. I rewrote the whole audio handling. The used Qt methods are still the same as before, so this very strange for me. And it is a Linux-only-issue.

[seescho]

It´s the commit "better audio video sync for playback", right?. That´s the first one without sound on playback for me.

[masc4ii]

Before you really get audio??? Okaaaay... I thought this commit is the last working for Linux.

[seescho]

On commit "added wbPicker icons and action..." I have audio on playback.

[masc4ii]

The only logic difference I see is, I call one stop() to much, if audio is not running yet and should be started. Made a fix for that. I don't think it will help, but please try out. https://github.com/ilia3101/MLV-App/commit/04e0ad640e4872a73336e49dd03a118bee980947 Old line 2328, there is no stop. In the new version, always stop is called first. Maybe linux counts how often I stop and play audio and then nothing comes out... hahahaha.

[masc4ii]

If that does not help, please try to add this to the AudioPlayback::play() function:

void QAudioOutput::resume() Resumes processing audio data after a suspend().

So, m_pAudioOutput->resume() maybe changes anything...

[seescho]

No success with the new commit. I will try your other tip later

[masc4ii]

I did another commit. Can you please check, if the functions in AudioPlayback class are called and some data is loaded?

[seescho]

No audio on playback with the other commit Now it begins to be complicated to me. Not shure, what I must check now. I set a stoppoint on line 98 in AudioPlayback .cpp. Look at my screenshot, maybe this help? https://seescho.files.wordpress.com/2017/10/another-fix.jpg

[masc4ii]

Hm... all looks good to me. Very identical to my debug output. At last line in play() I added qDebug() << m_pAudioOutput->state(); It tells me ActiveState after pressing the play button. Same for you?

[seescho]

Yes, says "ActiveState"

[masc4ii]

A little summary:

• Audio is loaded
• Functions are called
• Linux audio settings shows the connection to Qt via Pulseaudio
• Audio module is in ActiveState
• revisions (older than this), where the audio functions were called from actions were working (with bad sync)
• revisions (this and later), where the audio functions were called from functions, which were called from the timer were not working
• Problems only on linux. No problems on Win&OSX.

[seescho]

MainWindow.cpp I changed lines 376 ff like this for a test

{
        m_tryToSyncAudio = false;
        m_pAudioPlayback->stop();
        m_pAudioPlayback->jumpToPos( 0 );
        m_pAudioPlayback->play();
    }

With this change, audio playback is running.

[masc4ii]

What? You just changed ui->horizontalSliderPosition->value() to 0 and you hear audio? Does the position calculation in jumpToPos not work on linux? Or the seek? But what is strange: when starting audio after loading, the slider value is 0. So where is the difference? I still don't understand. Would be interesting to see what qDebug() << frame; brings in jumpToPos().

[seescho]

Doesn´t compile:

m_tryToSyncAudio = false;
m_pAudioPlayback->stop();
m_pAudioPlayback->jumpToPos( ui->horizontalSliderPosition->value() );
//m_pAudioPlayback->jumpToPos( 0 );
qDebug() << frame;
m_pAudioPlayback->play();

Error:

/home/edgar/MLV App/audiobug/MLV-App-master/platform/qt/MainWindow.cpp:381: Fehler: ‘frame’ was not declared in this scope

[masc4ii]

No sry... add it inside the jumpToPos() function in file AudioPlayback.cpp, please. Maybe also the value of position and audiostreamsize would be interesting. So the function could look like that:

//Jump to frame
void AudioPlayback::jumpToPos( int frame )
{
    if( !doesMlvHaveAudio( m_pMlvObject ) ) return;

    qint64 position = 4 * (qint64)( frame * getMlvSampleRate( m_pMlvObject ) / getMlvFramerate( m_pMlvObject ) );
    m_pAudioStream->device()->seek( position );
    qDebug() << frame << position << m_pAudioStream->device()->size();
}

[bouncyball-git]

as I mentioned above jumpToPos() never worked for linux.

[bouncyball-git]

This is the debug output when loop is enabled (qDebug() << frame << position << m_pAudioStream->device()->size()) When loop is going from the end to the start sometimes 2 lines printed sometimes one. Sometimes frame number is 0 and sometimes 1 :) don't you think it's extremely weird?

1 8008 422400 0 0 422400 0 0 422400 1 8008 422400 0 0 422400 1 8008 422400 0 0 422400 1 8008 422400 0 0 422400 1 8008 422400 0 0 422400 0 0 422400 1 8008 422400 0 0 422400 0 0 422400 1 8008 422400 0 0 422400 0 0 422400 0 0 422400 0 0 422400 1 8008 422400 0 0 422400 1 8008 422400 0 0 422400 1 8008 422400 0 0 422400 1 8008 422400 0 0 422400 1 8008 422400 0 0 422400 1 8008 422400

[masc4ii]

The debug log looks good, no error so far. And also when you got this output there was no audio? So the very last line is the one with seek(). But why does it work, when we hard code a 0, and it does not work when the algorithm writes a calculated 0 on seek? I still don't understand...

No, I don't think it is weird when it starts with 0 or 1. It depends how much time is gone since the last frame is drawn. Important is, that the calculation fits and seek() gets the right value. If the value is bigger than the size this would cause a problem. That there are comming 2 lines at once... hm... I can have a look for that, because I did not plan that. But it is also no problem if that happens.

[masc4ii]

The Qt documentation says:

bool QIODevice::seek(qint64 pos) For random-access devices, this function sets the current position to pos, returning true on success, or false if an error occurred. For sequential devices, the default behavior is to produce a warning and return false.

So what does seek return on linux? On OSX it is always true. qDebug() << "Seek:" << m_pAudioStream->device()->seek( position );

Please try with jumpToPos(0) and jumpToPos(...slider->value()).

[bouncyball-git]

device()->seek( position ) always returns true