search

wang-bin / QtAV

A cross-platform multimedia framework based on Qt and FFmpeg. 基于Qt和FFmpeg的跨平台高性能音视频播放框架. Recommand to use new sdk https://github.com/wang-bin/mdk-sdk
http://qtav.org
3.98k stars 1.5k forks source link

Android and iOS support question. #153

Closed adauvalter closed 8 years ago

adauvalter commented 10 years ago

Since 5.2 iOS and Android supported by Qt. Have you a plan to support QtAV on Android and iOS platform?

wang-bin commented 10 years ago

Yes.

For android, the main problem is audio. I plan to add OpenSL support to work on android. GLWidgetRenderer seems not work now, easy to fix. I have tested player and QMLPlayer on android, video playing works fine. Another problem is deployment. I use QtCreator to install player to android, some libraries must manually push to android.

I have an iOS device, but no development enviroment now.

adauvalter commented 10 years ago

Super! Thank you! I'm already build ffmpeg for iOS, will write here about the results on an iOS devices.

Ligverd commented 10 years ago

I have a problem to deploy on Android with QtCreator.

E/AndroidRuntime( 577): java.lang.UnsatisfiedLinkError: Cannot load library: link_image[1936]: 36 could not load needed library 'libQtAV.so' for 'libplayer.so' (load_library[1091]: Library 'libQtAV.so' not found)

The library libQtAV.so is added to apk lib/armeabi-v7a but the result is the same, where he looking it?

wang-bin commented 10 years ago

i don't know how qt's tool make the apk. libQtAV.so is not in apk. i put it in /system/lib

Ligverd commented 10 years ago

linking QtAV statically?

wang-bin commented 10 years ago

can be a solution

Ligverd commented 10 years ago

libQtAV.so copied to /system/lib demanded libavcodec.so.55 and libavutil.so.52 after these swears on log2f and I again deadlocked :(

incidentally in the generated APK have the following files /assets/--Added-by-androiddeployqt--/lib libavcodec.so.55 libavformat.so.55 libavutil.so.52 libswscale.so.2

adauvalter commented 10 years ago

Normally all so libraries should be inside an apk. Look please to the side of NDK. Here you can find some good documentation. I would like to make also an Android build,but have not enough time for that. 15.04.2014 10:44 пользователь "Ligverd" notifications@github.com написал:

libQtAV.so copied to /system/lib demanded libavcodec.so.55 and libavutil.so.52 after these swears on log2f and I again deadlocked :(

— Reply to this email directly or view it on GitHubhttps://github.com/wang-bin/QtAV/issues/153#issuecomment-40458358 .

Ligverd commented 10 years ago

I happen build normal APK with dynamic linking. Once I did the full integration QtAV in QT5 ie QtAV looks like any other component of the QT itself. I can send you step by step instructions If interested.

wang-bin commented 10 years ago

@Ligverd sounds great! Can you write a wiki?

Ligverd commented 10 years ago

Yes I can, wiki format github?

wang-bin commented 10 years ago

@Ligverd click new page button in https://github.com/wang-bin/QtAV/wiki

github uses markdown format. it's easy. plain text is ok too.

I think it's time to add audio for android : )

Ligverd commented 10 years ago

Yes, with sound problem ...: (

Ligverd commented 10 years ago

While written in Russian, translated by my friend later.

https://github.com/wang-bin/QtAV/wiki/Make-app-for-Android-on-QT5-QtAV-ffmpeg

wang-bin commented 10 years ago

@Ligverd I see. Thanks.Very good tutorial

Now i know how to add a library abc to apk, just write LIB += -labc in pro, am i right? I will have a try.

Ligverd commented 10 years ago

Yes, but it somehow pulls qt dependencies, without explicit instructions, with this I did not understand yet.

Ligverd commented 10 years ago

Build for Android OpenAL and recompiled ffmpeg --enable-openal, QtAV. Sound appeared only very stuttering, disassemble nothing. :(

Build portaudio for Android and rebuild QtAV with portaudio on android device:

QtAV/src/AudioOutputPortAudio.cpp:68 (QtAV::AudioOutputPortAudioPrivate::AudioOutputPortAudioPrivate()): PortAudio get device error!

How do I take sound to Android while generally not supported?

wang-bin commented 10 years ago

@Ligverd OpenAL support in QtAV has some problems now. BTW, --enable-openal is not required, that's for audio capturing.

Android has portaudio?

I plan to use opensl, but it's a little difficult.

adauvalter commented 10 years ago

I saw some PortAudio fork for Android. https://bitbucket.org/karanikk/portaudio-android/src Could it be usefull?

2014-05-24 13:29 GMT+02:00 WangBin notifications@github.com:

@Ligverd https://github.com/Ligverd OpenAL support in QtAV has some problems now. BTW, --enable-openal is not required, that's for audio capturing.

Android has portaudio?

I plan to use opensl, but it's a little difficult.

— Reply to this email directly or view it on GitHubhttps://github.com/wang-bin/QtAV/issues/153#issuecomment-44084589 .

Ligverd commented 10 years ago

Tried this fork. :(

QtAV/src/AudioOutputPortAudio.cpp:68 (QtAV::AudioOutputPortAudioPrivate::AudioOutputPortAudioPrivate()): PortAudio get device error!

I will understand

wang-bin commented 10 years ago

@Ligverd does not work for me. apk still has no those libraries.

Ligverd commented 10 years ago

libQmlAV and libQtAV not packed in APK or you about the sound? Appositely thanks for the script sdk_installs but it does not fully integrate module. If you open in qtcreator designer QML the module is not there. Need to copy the files back in qt5/qml/QtAV

libQmlAV.so plugins.qmltypes qmldir Video.qml

cp  build-QtAV-ARCH-Release/bin/QtAV  qt5/qml
qmlplugindump -nonrelocatable QtAV 1.3

At what file generated plugins.qmltypes qmlplugindump-nonrelocatable QtAV 1.3 I think you can just put his source code and install just copy.

I can make a patch but do not know how to deal with plugins.qmltypes

wang-bin commented 10 years ago

@Ligverd I add some libs like LIBS += -lavcodec, then avcodec is still not in apk.

I forgot qml things, i will try your method.

Ligverd commented 10 years ago

ffmpeg should be assembled in the form of libavcodec-55.so LIBS+=-lavcodec-55 library or in any form misses the APK?

wang-bin commented 10 years ago

i don't know whether plugins.qmptypes depends on Qt version. If yes, we have to use qmlplugindump.

Ligverd commented 10 years ago

I installed Qt 4.8 All gathered without QmlAV

The file QtAV.pro

...
greaterThan(QT_MAJOR_VERSION, 4) {
  # qtHaveModule does not exist in Qt5.0
  isEqual(QT_MINOR_VERSION, 0)|qtHaveModule(quick) {
    SUBDIRS += libqmlav
    libqmlav.file = qml/libQmlAV.pro
    libqmlav.depends += libqtav
    examples.depends += libqmlav
  }
}
...

ie QmlAV not build to Qt below 5 version

Accordingly, try to create plugins.yamptypes not possible.

Ligverd commented 10 years ago

Regarding the sound. And why not use QAudioOutput?

QAudioOutput present in Qt4.6+

wang-bin commented 10 years ago

qt quick is quite different in Qt4. so no qml support for Qt4

Ligverd commented 10 years ago

Spending a lot of time on portaudio finished this occupation does not work. :( For what happened with OpenAL

Tried to direct different sources, focused on

http://repo.or.cz/r/openal-soft/android.git

Had not much to tweak the source code because it requires OpenSLES 1.1 In Android-NDK r9d not fully compatible with 1.1

Video stop twitching as received normal synchronization. The sound is played but a bit "ripples" :(

here is the conclusion of ADB

...

D/dalvikvm(  676): Trying to load lib /data/data/org.test.video/lib/libopenal.so 0x412a5208
D/dalvikvm(  676): Added shared lib /data/data/org.test.video/lib/libopenal.so 0x412a5208

...

D/Qt      (  676): /home/kid/projects/QtAV/src/AVPlayer.cpp:589 (bool QtAV::AVPlayer::load(bool)): auto select clock: audio > external
D/Qt      (  676): /home/kid/projects/QtAV/src/AVPlayer.cpp:594 (bool QtAV::AVPlayer::load(bool)): Using AudioClock
D/Qt      (  676): /home/kid/projects/QtAV/src/AVPlayer.cpp:1239 (bool QtAV::AVPlayer::setupAudioThread()): has audio
D/Qt      (  676): /home/kid/projects/QtAV/src/AudioOutputOpenAL.cpp:108 (ALenum QtAV::audioFormatToAL(const QtAV::AudioFormat&)): OpenAL audio format: 0x10011 ch:2, sample format: flt
D/Qt      (  676): /home/kid/projects/QtAV/src/AudioOutputOpenAL.cpp:108 (ALenum QtAV::audioFormatToAL(const QtAV::AudioFormat&)): OpenAL audio format: 0x10011 ch:2, sample format: flt
D/Qt      (  676): /home/kid/projects/QtAV/src/AudioOutputOpenAL.cpp:181 (virtual bool QtAV::AudioOutputOpenAL::open()): OpenAL devices available: 1
D/Qt      (  676): /home/kid/projects/QtAV/src/AudioOutputOpenAL.cpp:183 (virtual bool QtAV::AudioOutputOpenAL::open()): device 0: OpenSL
D/Qt      (  676): /home/kid/projects/QtAV/src/AudioOutputOpenAL.cpp:186 (virtual bool QtAV::AudioOutputOpenAL::open()): AudioOutputOpenAL Opening default device: OpenSL
D/Qt      (  676): /home/kid/projects/QtAV/src/AudioOutputOpenAL.cpp:192 (virtual bool QtAV::AudioOutputOpenAL::open()): AudioOutputOpenAL creating context...
D/Qt      (  676): /home/kid/projects/QtAV/src/AudioOutputOpenAL.cpp:108 (ALenum QtAV::audioFormatToAL(const QtAV::AudioFormat&)): OpenAL audio format: 0x10011 ch:2, sample format: flt
D/Qt      (  676): /home/kid/projects/QtAV/src/AudioOutputOpenAL.cpp:229 (virtual bool QtAV::AudioOutputOpenAL::open()): AudioOutputOpenAL open ok...
D/Qt      (  676): /home/kid/projects/QtAV/src/AudioResamplerFF.cpp:156 (virtual bool QtAV::AudioResamplerFF::prepare()): swr speed=1.00
D/Qt      (  676): /home/kid/projects/QtAV/src/AudioResamplerFF.cpp:178 (virtual bool QtAV::AudioResamplerFF::prepare()): out: {cl: 3, fmt: flt, freq: 48000}

...

I will continue to investigate.

Ligverd commented 10 years ago

I written a simple APP on Qt5 using library OpenAL, which patching for OpenSLES 1.0.1 and 1.1

APP able to play WAV files mono / stereo 16bit PCM.

Everything works fine!

When compiling with QtAV support OpenAL sound much twitches, ripples. I'm now trying to sort out a bunch of ffmpeg -> QtAV-> OpenAL, to no avail.

Can download the project on GitHub but there's nothing new you will not see for yourself.

wang-bin commented 10 years ago

@Ligverd Greate! The design of audio playback in QtAV has some limitations and openal is not well supported. I have to rewrite the audio part to fit opensl and openal like api. How can i try your code?

Ligverd commented 10 years ago

http://77.241.16.66/pub/openal-test.tar.bz2

read dir tools

openal source + patch + build script and qtcreator sample project poor quality of the code part taken from an Internet honor had to be rewritten, but it is not important.

wang-bin commented 10 years ago

@Ligverd your code only works for a whole file. But what we have to do is play a small buffer. I have find out a good way to do this. And I think it can works for all audio apis, such as opensles, openal, portaudio. I will push it soon.

Ligverd commented 10 years ago

I wanted to test the functionality of OpenAL. For streaming audio of course is necessary to allocate a few short buffers. But for this test was not necessary. I hope it will give a good motivation for the implementation of sound on Android. I did not want to break your code but try to find the problem points.

wang-bin commented 10 years ago

@Ligverd you can test now. It works fine for me on ubuntu 1404, win8. time error is less than 1s in 30min. Some apis and code will change.

Ligverd commented 10 years ago

Everything worked!

You can increase the buffer for the video, and then on WiFi stream 4Mbit twitches / crumbles?

wang-bin commented 10 years ago

Great! Did test android? I plan to directly support OpenSLES soon.

Ligverd commented 10 years ago

Yes on Android. Directly OpenSLES is good!

I figured out the problem of assembling the latest version, I'm an idiot. I compiled a new version without removing the old one, and he took the old set headers. (

Good work! Android submitted. :)

We still get a good playback via WiFi, MXPlayer have no problems with these

Prompt video crop feature planned?

wang-bin commented 10 years ago

@Ligverd I compiled openal for android. no sound. can you send me your libopenal.so?

Ligverd commented 10 years ago

http://77.241.16.66/pub/OpenAL_android_ndk_r9d.tar.bz2 http://77.241.16.66/pub/QMLPlayer.apk

const int kBufferSize = 1024*4;
const int kBufferCount = 8;

You now 8 buffers 4096 bytes 44100Hz 16bit stereo sound, 1 seconds 176400 bytes

Do not small buffers?

wang-bin commented 10 years ago

great! i reboot my phone and sound appears! your QMLPlayer.apk works fine too.

wang-bin commented 10 years ago

@Ligverd i think small buffer is ok. in 1s there are many audio frames. each frame is not large. even if it's large, we can fill the buffers more times. does it brings you problems?

I want to share your QMLPlayer.apk on sourceforge : )

Ligverd commented 10 years ago

Okay, I'll try with buffers different options would be good to be able to control the size of the buffer of the application.

If you want what can I make some applications for Android and send APK

wang-bin commented 10 years ago

there's no api to control the buffer size now. if you want to change, change the value int chunk = qMin(decodedSize, 1024*4); in AudioThread.cpp to the same size as const int kBufferSize

Ligverd commented 10 years ago

Ok try.

wang-bin commented 10 years ago

@Ligverd I have a problem about building apk. I link to ffmpeg libraries with the name as you suggested, for example, libavcodec-55.so, and i run readelf -d lib_linux_arm/libQtAV.so it shows it depends libavcodec-55.so. But if i build an apk, run readelf -d android-build/libs/armeabi-v7a/libQtAV.so and it shows libavcodec.so.55. what's wrong?

wang-bin commented 10 years ago

@Ligverd my mistake. I forgot to update libQtAV.so in Qt dir. run sdk_install.sh works

Ligverd commented 10 years ago

Just rename the library a little. inside it has SONAME attribute that specifies the name of the present library.

here's an example output

$objdump -x ./libavcodec.so | more

./libavcodec.so:     file format elf32-i386
./libavcodec.so
architecture: i386, flags 0x00000150:
...
  NEEDED               libdl.so.2
  NEEDED               libc.so.6
  SONAME               libavcodec.so.52
...

For this we need to build the library in a special way. Sonam pass linker flag I laid out in issues configure fmpeg patch to which it does

--- ./configure.orig    2014-03-24 02:08:02.000000000 +0400
+++ ./configure 2014-05-19 13:48:16.000000000 +0400
@@ -2516,10 +2516,19 @@
 SLIBSUF=".so"
 SLIBNAME='$(SLIBPREF)$(FULLNAME)$(SLIBSUF)'
 SLIBNAME_WITH_VERSION='$(SLIBNAME).$(LIBVERSION)'
-SLIBNAME_WITH_MAJOR='$(SLIBNAME).$(LIBMAJOR)'
+
+# ORIG
+#SLIBNAME_WITH_MAJOR='$(SLIBNAME).$(LIBMAJOR)'
+#LIB_INSTALL_EXTRA_CMD='$$(RANLIB) "$(LIBDIR)/$(LIBNAME)"'
+#SLIB_INSTALL_NAME='$(SLIBNAME_WITH_VERSION)'
+#SLIB_INSTALL_LINKS='$(SLIBNAME_WITH_MAJOR) $(SLIBNAME)'
+
+# NEW
+SLIBNAME_WITH_MAJOR='$(SLIBPREF)$(FULLNAME)-$(LIBMAJOR)$(SLIBSUF)'
 LIB_INSTALL_EXTRA_CMD='$$(RANLIB) "$(LIBDIR)/$(LIBNAME)"'
-SLIB_INSTALL_NAME='$(SLIBNAME_WITH_VERSION)'
-SLIB_INSTALL_LINKS='$(SLIBNAME_WITH_MAJOR) $(SLIBNAME)'
+SLIB_INSTALL_NAME='$(SLIBNAME_WITH_MAJOR)'
+SLIB_INSTALL_LINKS='$(SLIBNAME)'
+

 asflags_filter=echo
 cflags_filter=echo
wang-bin commented 10 years ago

@Ligverd Thanks. I have successfully built apk. You are right. Seems that qtcreator only find the libraries in Qt's lib dir, but not ndk dir. So sdk_install.sh must copy libQmlAV.so to QtDir/lib too. I think we have to simplify the deployment steps.

I change the icon size a little. works perfect on N7. try: http://sourceforge.net/projects/qtav/files/release/1.3.4/QtAV1.3.4-QMLPlayer-android-arm-20140610.apk/download

Ligverd commented 10 years ago

Ok try these assemblies.

Every time I copied libQmlAV.so hands, thought there is some tricky.