search

ionorg / ion

Real-Distributed RTC System by pure Go and Flutter
MIT License
3.83k stars 519 forks source link

Flutter sdk for android app does not receive video stream from peers #101

Closed blackshehz closed 4 years ago

blackshehz commented 4 years ago

Your environment.

What did you do?

Run container exposing port 8080, connect to bash environment

 docker run -it --rm -p 8080:8080 -v $(pwd):/root/ion android-flutter /bin/bash

Create flutter project using instructions:

cd ~/ion/sdk/flutter/example

./scripts/project_tools.sh create

Create flutter project: name=ion_flutter_example, org=com.github.pion.ion ...
Setting "enable-macos-desktop" value to "true".

You may need to restart any open editors for them to read new settings.
Setting "enable-web" value to "true".

You may need to restart any open editors for them to read new settings.
Recreating project ....
  android/ion_flutter_example_android.iml (created)
  android/app/build.gradle (created)
  android/app/src/main/java/com/github/pion/ion/ion_flutter_example/MainActivity.java (created)
  android/build.gradle (created)
  .idea/workspace.xml (created)
  .idea/runConfigurations/main_dart.xml (created)
  .idea/libraries/KotlinJavaRuntime.xml (created)
  .idea/libraries/Dart_SDK.xml (created)
  .idea/libraries/Flutter_for_Android.xml (created)
  .idea/modules.xml (created)
  ion_flutter_example.iml (created)
  .metadata (created)
  ios/Runner.xcodeproj/xcshareddata/xcschemes/Runner.xcscheme (created)
  ios/Runner.xcodeproj/project.xcworkspace/contents.xcworkspacedata (created)
  ios/Runner.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist (created)
  ios/Runner.xcodeproj/project.xcworkspace/xcshareddata/WorkspaceSettings.xcsettings (created)
  ios/Runner/Info.plist (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-60x60@2x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-20x20@1x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-76x76@2x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-29x29@2x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-20x20@2x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-29x29@3x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Contents.json (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-83.5x83.5@2x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-20x20@3x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-29x29@1x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-40x40@2x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-40x40@1x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-76x76@1x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-40x40@3x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-60x60@3x.png (created)
  ios/Runner/Assets.xcassets/AppIcon.appiconset/Icon-App-1024x1024@1x.png (created)
  ios/Runner/Assets.xcassets/LaunchImage.imageset/LaunchImage.png (created)
  ios/Runner/Assets.xcassets/LaunchImage.imageset/LaunchImage@3x.png (created)
  ios/Runner/Assets.xcassets/LaunchImage.imageset/README.md (created)
  ios/Runner/Assets.xcassets/LaunchImage.imageset/Contents.json (created)
  ios/Runner/Assets.xcassets/LaunchImage.imageset/LaunchImage@2x.png (created)
  ios/Runner/Base.lproj/LaunchScreen.storyboard (created)
  ios/Runner/Base.lproj/Main.storyboard (created)
  ios/Flutter/AppFrameworkInfo.plist (created)
  ios/Flutter/Debug.xcconfig (created)
  ios/Flutter/Release.xcconfig (created)
  ios/Runner.xcworkspace/contents.xcworkspacedata (created)
  ios/Runner.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist (created)
  ios/Runner.xcworkspace/xcshareddata/WorkspaceSettings.xcsettings (created)
  ios/.gitignore (created)
  ios/Runner.xcodeproj/project.pbxproj (created)
  ios/Runner/AppDelegate.m (created)
  ios/Runner/AppDelegate.h (created)
  ios/Runner/main.m (created)
  android/settings.gradle (created)
  android/app/src/profile/AndroidManifest.xml (created)
  android/app/src/debug/AndroidManifest.xml (created)
  android/app/src/main/AndroidManifest.xml (created)
  android/app/src/main/res/mipmap-xhdpi/ic_launcher.png (created)
  android/app/src/main/res/mipmap-hdpi/ic_launcher.png (created)
  android/app/src/main/res/mipmap-xxhdpi/ic_launcher.png (created)
  android/app/src/main/res/drawable/launch_background.xml (created)
  android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png (created)
  android/app/src/main/res/values/styles.xml (created)
  android/app/src/main/res/mipmap-mdpi/ic_launcher.png (created)
  android/gradle.properties (created)
  android/gradle/wrapper/gradle-wrapper.properties (created)
  android/.gitignore (created)
  macos/Runner.xcodeproj/xcshareddata/xcschemes/Runner.xcscheme (created)
  macos/Runner.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist (created)
  macos/Runner.xcodeproj/project.pbxproj (created)
  macos/Runner/Configs/AppInfo.xcconfig (created)
  macos/Runner/Configs/Debug.xcconfig (created)
  macos/Runner/Configs/Warnings.xcconfig (created)
  macos/Runner/Configs/Release.xcconfig (created)
  macos/Runner/AppDelegate.swift (created)
  macos/Runner/DebugProfile.entitlements (created)
  macos/Runner/Assets.xcassets/AppIcon.appiconset/app_icon_16.png (created)
  macos/Runner/Assets.xcassets/AppIcon.appiconset/app_icon_1024.png (created)
  macos/Runner/Assets.xcassets/AppIcon.appiconset/app_icon_256.png (created)
  macos/Runner/Assets.xcassets/AppIcon.appiconset/app_icon_512.png (created)
  macos/Runner/Assets.xcassets/AppIcon.appiconset/app_icon_64.png (created)
  macos/Runner/Assets.xcassets/AppIcon.appiconset/app_icon_128.png (created)
  macos/Runner/Assets.xcassets/AppIcon.appiconset/Contents.json (created)
  macos/Runner/Assets.xcassets/AppIcon.appiconset/app_icon_32.png (created)
  macos/Runner/Release.entitlements (created)
  macos/Runner/Info.plist (created)
  macos/Runner/MainFlutterWindow.swift (created)
  macos/Runner/Base.lproj/MainMenu.xib (created)
  macos/Flutter/Flutter-Debug.xcconfig (created)
  macos/Flutter/Flutter-Release.xcconfig (created)
  macos/Runner.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist (created)
  macos/Runner.xcworkspace/contents.xcworkspacedata (created)
  macos/.gitignore (created)
  web/index.html (created)
  web/icons/Icon-512.png (created)
  web/icons/Icon-192.png (created)
  web/favicon.png (created)
  web/manifest.json (created)
  test/widget_test.dart (created)
Running "flutter pub get" in example...                             3.6s
Wrote 100 files.

All done!
[✓] Flutter: is fully installed. (Channel dev, v1.16.4-pre.33, on Linux, locale en_US)
[✓] Android toolchain - develop for Android devices: is fully installed. (Android SDK version
    29.0.0)
[✗] Chrome - develop for the web: is not installed. (Cannot find chrome executable at
    google-chrome)
[!] Android Studio: is not available. (not installed)
[✓] Connected device: is fully installed. (1 available)

Run "flutter doctor" for information about installing additional components.

In order to run your application, type:

  $ cd .
  $ flutter run

Your application code is in ./lib/main.dart.

Add permission labels to iOS.

Insert  <key>NSCameraUsageDescription</key> to line 25
Insert  <string>$(PRODUCT_NAME) Camera Usage!</string> to line 26
Insert  <key>NSMicrophoneUsageDescription</key> to line 27
Insert  <string>$(PRODUCT_NAME) Microphone Usage!</string> to line 28
Traceback (most recent call last):
  File "add-line.py", line 75, in <module>
    main(sys.argv[1:])
  File "add-line.py", line 69, in main
    replaceText(inputfile, string, text)
  File "add-line.py", line 36, in replaceText
    with open(filename, 'r') as fp:
FileNotFoundError: [Errno 2] No such file or directory: '../ios/Podfile'

Add permission labels to AndroidManifest.xml.

Replace minSdkVersion 16 to minSdkVersion 18
Insert     <uses-permission android:name="android.permission.CAMERA" /> to line 8
Insert     <uses-permission android:name="android.permission.RECORD_AUDIO" /> to line 9
Insert     <uses-permission android:name="android.permission.WAKE_LOCK" /> to line 10
Insert     <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" /> to line 11
Insert     <uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" /> to line 12
Insert     <uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" /> to line 13
Insert     <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /> to line 14

Add permission labels to macOS.

Insert  <key>NSCameraUsageDescription</key> to line 19
Insert  <string>$(PRODUCT_NAME) Camera Usage!</string> to line 20
Insert  <key>NSMicrophoneUsageDescription</key> to line 21
Insert  <string>$(PRODUCT_NAME) Microphone Usage!</string> to line 22
Insert        <key>com.apple.security.device.camera</key> to line 11
Insert        <true/> to line 12
Insert        <key>com.apple.security.device.microphone</key> to line 13
Insert        <true/> to line 14
Insert        <key>com.apple.security.network.client</key> to line 15
Insert        <true/> to line 16
Insert        <key>com.apple.security.device.camera</key> to line 7
Insert        <true/> to line 8
Insert        <key>com.apple.security.device.microphone</key> to line 9
Insert        <true/> to line 10
Insert        <key>com.apple.security.network.client</key> to line 11
Insert        <true/> to line 12

Flutter for iOS may have changed, and hence this error FileNotFoundError: [Errno 2] No such file or directory: '../ios/Podfile' Ignored this error as it is not required for Android

Build Release and Debug apk for arm64 (easiest one to test for me):

flutter build apk --target-platform=android-arm64 --release
Running Gradle task 'assembleRelease'...                                
Running Gradle task 'assembleRelease'... Done                      43.7s
✓ Built build/app/outputs/apk/release/app-release.apk (18.6MB).
flutter build apk --target-platform=android-arm64 --debug
Note: Some input files use or override a deprecated API.                
Note: Recompile with -Xlint:deprecation for details.                    
Note: Some input files use unchecked or unsafe operations.              
Note: Recompile with -Xlint:unchecked for details.                      
Running Gradle task 'assembleDebug'...                                  

Running Gradle task 'assembleDebug'... Done                       141.0s (!)
✓ Built build/app/outputs/apk/debug/app-debug.apk.

Pleasantly surprised everything worked so well! Thank you for that.

What did you expect?

Android app to work like in screenshot:

What happened?

  1. Release apk crashes when trying to open (on device Pixel 3XL and using https://appetize.io) - link to release apk

  2. Debug apk opens, connects to room, but does not receive data from any peer. I see bottom banners when other peers connect and disconnet, video/audio stream from phone is received by other peers. link to debug apk. You can test directly here too - https://appetize.io/app/58am4rrxc8w41fmcbhy191njn0?device=pixel4&scale=75&orientation=portrait&osVersion=10.0&deviceColor=black

Web peer on Chrome for macOS:

Screenshot 2020-04-01 at 9 24 33 PM

Android App - at the same time:

Screenshot 2020-04-01 at 9 24 43 PM

Other images form android app - app receives connection information from peer - but no video/audio streams: 2020-04-01 21-25-20 Screenshot 2020-04-01 21-25-06 Screenshot 2020-04-01 21-24-52 Screenshot

  1. Flutter Debug vs Release - https://github.com/flutter/flutter/issues/15100 - tried all solutions from here, but could not fix this behavior. Release always crashes on startup. This issue has better logging - https://github.com/flutter/flutter/issues/48015 Unfortunately, I still could not fix this. My experience with android development or Flutter is quite limited - I am trying to grasp at whatever straws I can hold. I am still figuring out how to get USB pass through to Docker so that I can try adb trace for the app.

  2. Web App does not seem to port forward I tried to build a release version of web-app in Flutter to see if can find any information

    
flutter run -d web --web-port 8080        
Launching lib/main.dart on Web Server in debug mode...
Syncing files to device Web Server...                                   
13,604ms (!)                                       
lib/main.dart is being served at http://localhost:8080

Warning: Flutter's support for web development is not stable yet and hasn't been thoroughly tested in production environments. For more information see https://flutter.dev/web

🔥 To hot restart changes while running, press "r". To hot restart (and refresh the browser), press "R". For a more detailed help message, press "h". To quit, press "q".

I have mapped port 8080 from docker to my machine, but:
<img width="1395" alt="Screenshot 2020-04-01 at 9 41 03 PM" src="https://user-images.githubusercontent.com/51226813/78160427-8cbbf780-7461-11ea-9843-033280c2011a.png">

Although:
```bash
docker exec -it 12abac /bin/bash
root@12abac874d58:/# curl http://localhost:8080
<!DOCTYPE html>
<html>
<head>
  <meta charset="UTF-8">
  <meta content="IE=Edge" http-equiv="X-UA-Compatible">
  <meta name="description" content="A new Flutter project.">

  <!-- iOS meta tags & icons -->
  <meta name="apple-mobile-web-app-capable" content="yes">
  <meta name="apple-mobile-web-app-status-bar-style" content="black">
  <meta name="apple-mobile-web-app-title" content="ion_flutter_example">
  <link rel="apple-touch-icon" href="icons/Icon-192.png">

  <!-- Favicon -->
  <link rel="shortcut icon" type="image/png" href="favicon.png"/>

  <title>ion_flutter_example</title>
  <link rel="manifest" href="manifest.json">
</head>
<body>
  <!-- This script installs service_worker.js to provide PWA functionality to
       application. For more information, see:
       https://developers.google.com/web/fundamentals/primers/service-workers -->
  <script>
    if ('serviceWorker' in navigator) {
      window.addEventListener('load', function () {
        navigator.serviceWorker.register('flutter_service_worker.js');
      });
    }
  </script>
  <script src="main.dart.js" type="application/javascript"></script>
</body>
</html>

At first, I thoght it was a docker port forwarding issue, so I tried a simple server:

docker run -P -d nginxdemos/hello
Screenshot 2020-04-01 at 9 45 49 PM Screenshot 2020-04-01 at 9 46 09 PM

So docker port forwarding works and now I am utterly confused. Any help with this issue will be greatly appreciated! (Although not necessary) Dockerfile for reference - https://gist.github.com/blackshehz/c8cc0d28bf2ddc3b7fb7cb63de24ba12

Please keep up the great work! Do let me know if you would like to separate some of these issues. I am happy to help in any way I can.

crossle commented 4 years ago

Same problem。

biz_1    | 2020-04-13T09:02:27Z INF Send keepalive !!!
biz_1    | 2020-04-13T09:02:27Z INF Send data: {}
biz_1    | 2020-04-13 09:02:30.69 INF
biz_1    | ----------------signal-----------------
biz_1    | room: 1234
biz_1    | peers: 2
adwpc commented 4 years ago

Flutter sdk moved to new repo If this problem still exist, please make issue there

157