search

jackyglony / imsdroid

Automatically exported from code.google.com/p/imsdroid
0 stars 0 forks source link

IMSDroid video not working on HTC EVO #21

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
http://groups.google.com/group/doubango/browse_thread/thread/723f5a39c50ba9bf

Original issue reported on code.google.com by boss...@yahoo.fr on 5 Aug 2010 at 9:16

GoogleCodeExporter commented 9 years ago

Original comment by boss...@yahoo.fr on 6 Aug 2010 at 3:11

Attachments:

GoogleCodeExporter commented 9 years ago 
The Wireshark traces are attached for H263, H263+ and H264 calls.

Original comment by whg...@gmail.com on 6 Aug 2010 at 3:29

Attachments:

GoogleCodeExporter commented 9 years ago 
I have started to modify IMSDroid to support CIF video size. Perhaps I don't 
have all information about your network architecture because all responses sent 
from your client don't respect the specifications (RFC 3264 and 
draft-ietf-sipping-sip-offeranswer-12).

1. H263
  - IMSDroid offers H.263 (34) with QCIF=2
  - The response sent from your contains H.263(34) with QCIF=1;CIF=1, ... and H.263+ (96).
  ==> MUST not add new codec in it's offer
2. H.263+
  - IMSDroid offers H.263+(103) with CIF=2;QCIF=2
  - The response sent from your client contains H263+(96) and H263(34)
  ==> Two errors: MUST not add H263 and the payload type of H.263+ is incorrect
3. H264
  - IMSDroid offers H264 Base Profile 1.0 (104)
  - The response sent from your client contains H264 (96) with a profile-level-id=42000d
  ==> Two errors: The payload type is incorrect and the level does not match.

Do you have any idea why all responses are incorrect?

Original comment by imsframe...@gmail.com on 6 Aug 2010 at 9:36

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
The latest version (1.0.234) should allow decoding any video size. Tests have 
been done with Eyebeam. 
For encoding size, you can now choose the preferred size. For more info: 
http://code.google.com/p/imsdroid/wiki/Quick_Start#Video_Quality.

Original comment by boss...@yahoo.fr on 9 Aug 2010 at 5:21

GoogleCodeExporter commented 9 years ago 
Thanks for the update. Will try it.
In the meantime, can you help figure out why the webcam not working on my HTC 
EVO? I looked at logs and it complains about "Invalid picture size requested: 
352x288".

Original comment by whg...@gmail.com on 9 Aug 2010 at 5:01

Attachments:

GoogleCodeExporter commented 9 years ago 
This is very strange as the previous version only support QCIF (176x144) 
encoding (Bandwidth level was useless). As you have eclipse the best way is to 
run the application in debug mode and set a breakpoint in the prepare() 
function (VideoProducer.java) to see which size is requested by the native 
function. You can also change the bandwidth level (low<->medium) to switch from 
QCIF<->CIF to see if the problem persist.
Have you managed to get it to work with another SIP phone?

Original comment by boss...@yahoo.fr on 9 Aug 2010 at 5:31

GoogleCodeExporter commented 9 years ago 
No sorry I confused you. I was using the latest version of your code to test 
with the bandwidth level set to high.
I also ran test calls to Bria and IMSDroid. The logs say it is failing at the 
setParameters() method when it is trying to open the default webcam.

08-09 14:24:21.783: DEBUG/QualcommCameraHardware(65): requested preview size 
768 x 432
08-09 14:24:21.783: DEBUG/QualcommCameraHardware(65): requested picture size 
352 x 288
08-09 14:24:21.783: ERROR/QualcommCameraHardware(65): Invalid picture size 
requested: 352x288
08-09 14:24:21.783: ERROR/QualcommCameraHardware(65): virtual android::status_t 
android::QualcommCameraHardware::set
Parameters(const android::CameraParameters&): 2945, rc = -22

I changed the setPictureSize() method in VideoProducer.java to 1024 x 768, then 
it was able to open the webcam and got video to going. But the call would only 
last for 10 secs or so and IMSDroid then crashed.

Original comment by whg...@gmail.com on 9 Aug 2010 at 6:39

GoogleCodeExporter commented 9 years ago 
It's normal to crash. You should also change the frame allocation size.
From: float capacity = (float)(width*height)*1.5f/* (3/2) */;
To: float capacity = (float)(1024*768)*1.5f/* (3/2) */;

Original comment by boss...@yahoo.fr on 9 Aug 2010 at 8:53

GoogleCodeExporter commented 9 years ago 
That fixed the 30 secs in-call crash. Now it is crashing during call hang up.
Also, heavy delay is occurring on the inbound video shown on the screen. Looks 
like 1024x768 is a bit too much? Do you know how to set the picture size to CIF 
without the Android platform complaining about invalid picture size?

Original comment by whg...@gmail.com on 9 Aug 2010 at 10:01

GoogleCodeExporter commented 9 years ago 
I confirm that 1024x768 is Toooo much for a mobile. You have the delay because 
the CPU overhead. You should only have delay when you start sending video. Is 
it right?
I'm looking into android source code to understand why this fail. According to 
your log the camera supports CIF (see line 243).
You can also use adb tool to see the "CPU usage".

Original comment by boss...@yahoo.fr on 9 Aug 2010 at 10:12

GoogleCodeExporter commented 9 years ago 
I printed the webcam parameters out:
preview-size-values=1280x720,800x480,768x432,720x480,640x480,576x432,480x320,400
x240,384x288,352x288,320x240,272x272,240x240,240x160,176x144;
picture-size-values=3264x2448,3264x1952,2592x1952,2592x1552,2048x1536,2048x1216,
1600x1200,1280x960,1280x768,1024x768,640x480,640x384,512x384,400x400,272x272;
Now I am testing some of these combinations.

I tested with Bria and my initial finding is that everytime there will be some 
delay no matter what. I used setPreviewSize(176, 144), setPictureSize(272 , 
272) and bandwidth set to low for my testing.

Which SIP client you have tested with that gave you the least delay and best 
performance?

Original comment by whg...@gmail.com on 10 Aug 2010 at 5:28

GoogleCodeExporter commented 9 years ago 
Very strange. I have done my tests with linphone, eyeabeam, xlite and ekiga.
The delay is on the remote or local video?
Which codec are you using? Do you have the same issue with all codecs?
What is the average CPU usage (with and without outgoing video)?

Original comment by boss...@yahoo.fr on 10 Aug 2010 at 5:44

GoogleCodeExporter commented 9 years ago 
Its happening on the local video every time I did the "Send Video". CPU stats 
below:
User 62%, System 17%, IOW 0%, IRQ 1%
User 197 + Nice 0 + Sys 55 + Idle 56 + IOW 0 + IRQ 0 + SIRQ 6 = 314

  PID CPU% S  #THR     VSS     RSS PCY UID      Name
 3728  63% S    23 195164K  29560K  fg app_8    org.doubango.imsdroid
  113  12% S    67 288736K  52964K  fg system   system_server
   65   1% S    13  43832K   5708K  fg media    /system/bin/mediaserver

Original comment by whg...@gmail.com on 10 Aug 2010 at 7:02

GoogleCodeExporter commented 9 years ago 
I tested with H.263.

Original comment by whg...@gmail.com on 10 Aug 2010 at 7:03

GoogleCodeExporter commented 9 years ago 
Without outgoing video, the CPU stats:
User 43%, System 40%, IOW 0%, IRQ 0%
User 136 + Nice 0 + Sys 125 + Idle 48 + IOW 0 + IRQ 0 + SIRQ 3 = 312

  PID CPU% S  #THR     VSS     RSS PCY UID      Name
 3728  39% S    23 196300K  33248K  fg app_8    org.doubango.imsdroid
  113  28% S    67 288736K  52780K  fg system   system_server
  288   9% S     1      0K      0K  fg root     dhd_dpc
 3788   2% R     1    940K    416K  fg shell    top
   65   1% S    13  43832K   5708K  fg media    /system/bin/mediaserver
   76   1% S     8  10468K   1428K  fg root     /system/bin/iqd

Original comment by whg...@gmail.com on 10 Aug 2010 at 7:05

GoogleCodeExporter commented 9 years ago 
Which audio codec is associated to the session?

Original comment by boss...@yahoo.fr on 10 Aug 2010 at 7:10

GoogleCodeExporter commented 9 years ago 
Could you also disable fullscreen mode (Options->General) and give me CPU info 
(With and without encoding).

Original comment by boss...@yahoo.fr on 10 Aug 2010 at 7:14

GoogleCodeExporter commented 9 years ago 
Without encoding:
User 46%, System 24%, IOW 0%, IRQ 1%
User 145 + Nice 0 + Sys 76 + Idle 86 + IOW 0 + IRQ 0 + SIRQ 5 = 312

  PID CPU% S  #THR     VSS     RSS PCY UID      Name
 4533  41% S    24 196836K  30488K  fg app_8    org.doubango.imsdroid
  113  12% S    67 288544K  51752K  fg system   system_server
  288  12% S     1      0K      0K  fg root     dhd_dpc
   65   3% S    13  43832K   5772K  fg media    /system/bin/mediaserver

With encoding:
User 62%, System 33%, IOW 0%, IRQ 3%
User 279 + Nice 0 + Sys 148 + Idle 0 + IOW 0 + IRQ 0 + SIRQ 17 = 444

  PID CPU% S  #THR     VSS     RSS PCY UID      Name
 4533  56% S    24 200804K  33400K  fg app_8    org.doubango.imsdroid
   65  11% S    17  52584K   6220K  fg media    /system/bin/mediaserver
  113   9% S    67 289568K  51756K  fg system   system_server

The application crashed everytime I used "send Video". You can take a look at 
the attached crash.log.

Original comment by whg...@gmail.com on 10 Aug 2010 at 7:34

Attachments:

GoogleCodeExporter commented 9 years ago 
The crash happened during call release.

Original comment by whg...@gmail.com on 10 Aug 2010 at 7:35

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
This is with fullscreen disabled?
On a HTC Hero (528MHz) I have respectively 27% and 48%. I don't know how you 
can have 41% and 50% on EVO(1GHz).
FYI: http://code.google.com/p/imsdroid/wiki/Video_Performance_Issues

Original comment by boss...@yahoo.fr on 10 Aug 2010 at 7:41

GoogleCodeExporter commented 9 years ago 
For the crash try to comment the body of "surfaceDestroyed(SurfaceHolder 
holder)" function to see if it continue to happen.

Original comment by boss...@yahoo.fr on 10 Aug 2010 at 7:52

GoogleCodeExporter commented 9 years ago 
I also tested it on HTC Hero
Full-screen mode: disabled
bandwidth: Low
audio codec: G.711u
video codec: H.263
setPictureSize(272, 272)
Test call made to: CounterPath Bria 2.5

Without encoding
User 44%, System 46%, IOW 0%, IRQ 0%
  PID CPU% S  #THR     VSS     RSS PCY UID      Name
 2408  38% S    21 145180K  24384K  fg app_74   org.doubango.imsdroid

With encoding
User 70%, System 28%, IOW 0%, IRQ 0%
  PID CPU% S  #THR     VSS     RSS PCY UID      Name
 2408  66% S    21 149132K  26008K  fg app_74   org.doubango.imsdroid

Perhaps you can tell me the setup you got when you tested it on your HTC HERO 
with EyeBeam? Did you have to change the code to set the 272x272 picture size? 
I used a Sprint HTC HERO.

Original comment by whg...@gmail.com on 10 Aug 2010 at 8:39

GoogleCodeExporter commented 9 years ago 
Which surfaceDestroyed() function you want me to comment out, VideoProducer or 
ScreenPresence?

Original comment by whg...@gmail.com on 10 Aug 2010 at 8:43

GoogleCodeExporter commented 9 years ago 
you don't need to change the picture size. QCIF is already supported by Hero.
Try the test with QCIF and tell me what is the result?
According to Android source code, all devices should support QCIF and CIF. The 
problem with EVO is very strange. Video resizing (FFMpeg swscale()) is one of 
the function which use most CPU cycles.

The function to comment is from VideoProducer.java.

Original comment by boss...@yahoo.fr on 10 Aug 2010 at 8:48

GoogleCodeExporter commented 9 years ago 
I tried imsdroid-1.0.234.apk on the HTC Hero. I set bandwidth to low, which 
means QCIF. I also disabled full-screen. When I tested it with EyeBeam, I got 
only one-way video because it complained about "Invalid picture size requested: 
176x144" when I did "Send Video". How did you get it to work without changing 
the picture size? I am lost here.

Original comment by whg...@gmail.com on 10 Aug 2010 at 9:20

GoogleCodeExporter commented 9 years ago 
No idea. I'm using HTC Hero with Android 1.5.

Original comment by boss...@yahoo.fr on 10 Aug 2010 at 9:31

GoogleCodeExporter commented 9 years ago 
Mine is Android-2.1. HTC EVO is Android-2.2. So that might be it?

Original comment by whg...@gmail.com on 10 Aug 2010 at 9:38

GoogleCodeExporter commented 9 years ago 
In the comments 22 and 26: I said "HTC Hero" and not "HTC EVO". 
It was about CPU usage. You cannot consume more CPU on EVO(1GHz) than 
Hero(528Hz). Even on an old G1 H.263 call does take 66%. Only H.263 and Theora 
should take more than 50%.

Original comment by boss...@yahoo.fr on 10 Aug 2010 at 9:45

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
Correction: Even on an old G1, H.263 call doesn't take 66%. Only H.264 and 
Theora should take more than 50%.

Original comment by boss...@yahoo.fr on 10 Aug 2010 at 9:48

GoogleCodeExporter commented 9 years ago 
I also encounter this problem of web cam grab on my HTC EVO with Android 2.2. 
Did you finally solve this problem ?

Original comment by didier.c...@gmail.com on 10 Aug 2010 at 9:52

GoogleCodeExporter commented 9 years ago 
@didier.chabanol
No but whglee is on the right direction. He has found that the problem come 
from the requested picture sizes (QCIF or CIF) which are not supported by HTC 
EVO. The problem could be resolved with getSupportedPictureSizes() function 
(http://developer.android.com/reference/android/hardware/Camera.Parameters.html#
getSupportedPictureSizes%28%29) available starting API level 5.

Original comment by boss...@yahoo.fr on 10 Aug 2010 at 10:09

GoogleCodeExporter commented 9 years ago 
I am playing around in eclipse as well on an EVO running 2.1 -- not much 
programming experience though.  Whglee how are you creating those log files?

Original comment by amwalt...@gmail.com on 11 Aug 2010 at 2:45

GoogleCodeExporter commented 9 years ago 
amwalters,
you need Android SDK tools (adb, ddms) installed on your development machine to 
capture those logs from the EVO.

Original comment by whg...@gmail.com on 11 Aug 2010 at 3:48

GoogleCodeExporter commented 9 years ago 
Yep, I have them both and have everything running.  It just looked much more 
imsdroid specific than the logcat output filtered by pid.

Original comment by amwalt...@gmail.com on 11 Aug 2010 at 9:25

GoogleCodeExporter commented 9 years ago 
bossiel,
do you mind to run a logcat from your Hero when making a video call? I would 
like to compare the webcam params on Android 1.5 against my Android 2.1 Hero.
thanks.

Original comment by whg...@gmail.com on 11 Aug 2010 at 4:01

GoogleCodeExporter commented 9 years ago 
See attached.
As I said in comment 34. If you are targeting Android 2.0 or later you can use 
getSupportedPictureSizes() to get supported sizes. Another possible 
optimization is to check if your camera support YUV40SP. If yes, this will 
avoid to convert the picture from NV21 before encoding ().
On Android 2.0 and later you have many possibilities (AMR Cortex, 
CallbackBuffer, Picture Format, ...) to optimize the encoding process.
In all cases, H.263 should not take 66%.

Original comment by boss...@yahoo.fr on 11 Aug 2010 at 6:46

Attachments:

GoogleCodeExporter commented 9 years ago 
I found out why there is a delay on the inbound video everytime I added the 
outgoing video leg. It was the build target. IMSDroid built on API level 3 
(Android 1.5) will not have the delay problem. If it is built on API level 7 or 
8 (Android 2.1 or 2.2), I will see delay. Looks like those optimization you 
mentioned are needed for Android 2.2 and later.

Also, if I set the bandwidth to high, I will see heavy delay on the inbound 
video when outbound video is on.

Regarding picture size, getSupportedPictureSizes() must be used to determine 
the supported size for a given device and IMSDroid can set it accordingly. The 
HTC Hero (2.1) and EVO (2.2) I have don't offer standard QCIF or CIF picture 
size. But, I can use whatever available size returned from the get size method 
and it works for me.

Original comment by whg...@gmail.com on 11 Aug 2010 at 9:24

GoogleCodeExporter commented 9 years ago 
The problem is that getSupportedPictureSizes() is only supported starting API 
level 5 while we target API Level 3 and later.
Have you tried to use 
addCallBackBuffer(http://developer.android.com/reference/android/hardware/Camera
.html#addCallbackBuffer%28byte[]%29) instead of onPreviewFrame()?

Original comment by boss...@yahoo.fr on 11 Aug 2010 at 9:30

GoogleCodeExporter commented 9 years ago 
Have you resolved the crash(when hangup)?

Original comment by boss...@yahoo.fr on 12 Aug 2010 at 4:26

GoogleCodeExporter commented 9 years ago 
I haven't looked at the crash during call release just yet. I am still trying 
to figure out how to use addCallBackBuffer(). I am looking at 
http://code.google.com/p/android/issues/detail?id=2794 comment 22 on how that 
is being implemented. If you have a chance could you give me a hand, or some 
tips? You know better than I do on the code. Then I can spend time to look at 
the crash.

Original comment by whg...@gmail.com on 12 Aug 2010 at 7:20

GoogleCodeExporter commented 9 years ago 
Should I implement addCallBackBuffer() on both VideoProducer and VideoConsumer, 
or more places?

Original comment by whg...@gmail.com on 12 Aug 2010 at 7:47

GoogleCodeExporter commented 9 years ago 
For the crash, I have pushed a patch here: 
http://code.google.com/p/imsdroid/issues/detail?id=31
Could you tell if it works for you?

Original comment by boss...@yahoo.fr on 12 Aug 2010 at 7:47

GoogleCodeExporter commented 9 years ago 
No you only need it for the Producer. It will allow you to use a single buffer 
instead of a buffer-per-frame. The problem with onPreviewFrame is that it will 
create, marshal and destroy a buffer for each _data[] callback. If you look at 
the log you will see a lot of messages about the GC.
addCallbackBuffer is used in conjunction with setPreviewCallbackWithBuffer. 
Each time you provide the buffer the callback should be called when the buffer 
is full.

Original comment by boss...@yahoo.fr on 12 Aug 2010 at 7:58

GoogleCodeExporter commented 9 years ago 
Your patch does fix the crash.

Original comment by whg...@gmail.com on 12 Aug 2010 at 8:41

GoogleCodeExporter commented 9 years ago 
bossiel,
can you explain what onPreviewFrame() does for VideoProducer? Looks like it is 
doing some buffer management.
If I use addCallbackBuffer and setPreviewCallbackWithBuffer, whatever code in 
onPreviewFrame I should not need to keep, am I right? Also how many callback 
buffers would you recommend?

Original comment by whg...@gmail.com on 13 Aug 2010 at 5:52

GoogleCodeExporter commented 9 years ago 
I have implemented addCallbackBuffer and setPreviewCallbackWithBuffer to 
VideoProducer but video call is failing. No video is sending out.

Original comment by whg...@gmail.com on 13 Aug 2010 at 7:24

Attachments:

GoogleCodeExporter commented 9 years ago 
I got the addCallbackBuffer and setPreviewCallbackWithBuffer to work on calls 
with video working both ways. But the delay on the inbound video stream using 
CIF is still horrible. QCIF works great. I am testing it with Bria and only 
H.263 codec enabled. Please take a look at the attached file and let me know if 
I am doing it correctly. I just can't believe HTC EVO, with all that CPU speed, 
cannot support 2 way CIF video calls.

Original comment by whg...@gmail.com on 16 Aug 2010 at 6:19

Attachments: