search

axemclion / browser-perf

Performance Metrics for Web Browsers
BSD 2-Clause "Simplified" License
937 stars 61 forks source link

Can't get it to work on Android + Cordova #50

Closed ghost closed 8 years ago

ghost commented 8 years ago

First of all, great tool! However, I'm stuck. I'm following (this guide)[https://github.com/axemclion/browser-perf/wiki/Setup-Instructions#testing-mobile-cordova-webviewhybrid-applications] and it does not work. I've started Chromedriver 2.20 in one console, and running browser-perf in another window. The app is starting, but then stops immediately. In the chromedriver log I see this:

49.607][DEBUG]: Sending adb command: host:transport:emulator-5554|shell:am force-stop nzpost.provoke.mobile
[50.745][DEBUG]: Received adb response:
[50.746][SEVERE]: unknown error: session deleted because error encountered when notifying listeners of 'GetLog' command
from unknown error: unexpected command response
  (Session info: webview=39.0.0.0)
[50.750][DEBUG]: Log type 'driver' lost 1 entries on destruction
[50.752][DEBUG]: Log type 'performance' lost 240 entries on destruction
[50.754][DEBUG]: Log type 'browser' lost 1 entries on destruction

And, in browser-perf the following response:

Error: [log("performance")] Error response status: 13, , UnknownError - An unknown server-side error occurred while processing the command. Selenium error: unknown error: session deleted because error encountered when notifying listeners of 'GetLog' command
from unknown error: unexpected command response

I've also created tests with Protractor, and it works fine with Appium. I've also tried to run Appium (instead of Chromedriver) and change the URL to point to Appium, but that does not seem to work either; I'm not sure what to put in the configuration.

My config file is:

{
  "selenium": "localhost:9515",
  "browsers": [
    {
      "browserName": "android",
      "chromeOptions": {
        "androidActivity": "io.cordova.mobile.MainActivity",
        "androidPackage": "io.cordova.mobile"
      }
    }
  ]
}

Tested on an Android 4.4.4 device, and an Android 5.1.1 emulator.

Please help me out, thanks!

axemclion commented 8 years ago

@rolandoldengarm Are you still having a problem with the latest release ? If yes, I plan to look at it during this weekend.

ghost commented 8 years ago

@axemclion I'll follow up next week, not working on the project right now.

axemclion commented 8 years ago

@rolandoldengarm Thanks - and sorry for the delay in getting back to you.

blefebvre commented 8 years ago

Hi @axemclion, I'm running into a similar issue with the following environment:

To reproduce:

Result:

[146.617][INFO]: COMMAND Quit {

}
[146.617][DEBUG]: Sending adb command: host:transport:03f3cc21437ccb7d|shell:am force-stop com.brucelefebvre.PGAppPerf
[147.316][DEBUG]: Received adb response:
[147.316][INFO]: RESPONSE Quit
[147.316][DEBUG]: Log type 'driver' lost 0 entries on destruction
[147.316][DEBUG]: Log type 'performance' lost 0 entries on destruction
[147.316][DEBUG]: Log type 'browser' lost 0 entries on destruction

BTW: I love this tool. Thanks for all the work you've put into it!

blefebvre commented 8 years ago

In looking a bit closer at the logs it appears that mine is in fact a different issue. If you'd prefer that I file a separate issue for it @axemclion, just let me know.

axemclion commented 8 years ago

@blefebvre Sure - I was planning to investigate the entire Android issue sometime this week.

blefebvre commented 8 years ago

Thanks @axemclion, I've logged https://github.com/axemclion/browser-perf/issues/61.

axemclion commented 8 years ago

@rolandoldengarm I fixed this https://github.com/axemclion/browser-perf/commit/60477662c950913855460a6d1879920afa703d21. Closing this issue. If you still have this issue, please reopen.

palucdev commented 8 years ago

I got exactly the same problem as @rolandoldengarm . Running on browser-perf 1.4.11. 

C:\Users\Me\Desktop>browser-perf --config-file=configfile.json
Error: [log("performance")] Error response status: 13, , UnknownError - An unkno
wn server-side error occurred while processing the command. Selenium error: unkn
own error: session deleted because error encountered when notifying listeners of
 'GetLog' command
from unknown error: unexpected command response
  (Session info: webview=39.0.0.0)
  (Driver info: chromedriver=2.22.397933 (1cab651507b88dec79b2b2a22d1943c01833cc
1b),platform=Windows NT 6.3.9600 x86_64)...
axemclion commented 8 years ago

@Darkoon Are you still getting this error ? Can you paste in the full log ? What site are you running it against ?

palucdev commented 8 years ago

I'm still getting it for this config file: https://github.com/axemclion/browser-perf/blob/master/test/res/android-hybrid.config.json (with modified androidActivity and androidPackage ofc). I start my AVD and run chromedriver.exe (ver. 2.22, starts listening on port 9515). Then I go to platform-tools in android sdk and go with command: adb server-start. At last I use: browser-perf --config-file=configfile.json. After that I have errors:

Error: [log("performance")] Error response status: 13, , UnknownError - An unknown server-side error occurred while processing the command. Selenium error: unknown error: session deleted because error encountered when notifying listeners of 'GetLog' command
from unknown error: unexpected command response 
(Session info: webview=39.0.0.0)
  (Driver info: chromedriver=2.22.397933 (1cab651507b88dec79b2b2a22d1943c01833cc
1b),platform=Windows NT 6.3.9600 x86_64)
at exports.newError (C:\Users\Me\AppData\Roaming\npm\node_modules\browser-perf\node_modules\wd\lib\utils.js:139:13)
at C:\Users\Me\AppData\Roaming\npm\node_modules\browser-perf\node_modules\wd\lib\callbacks.js:75:19
at C:\Users\Me\AppData\Roaming\npm\node_modules\browser-perf\node_modules\wd\lib\webdriver.js:174:5
at Request._callback (C:\Users\Me\AppData\Roaming\npm\node_modules\browser-perf\node_modules\wd\lib\http-utils.js:87:7)
at Request.self.callback (C:\Users\Me\AppData\Roaming\npm\node_modules\browser-perf\node_modules\wd\node_modules\request\request.js:368:22)
at emitTwo (events.js:87:13)
at Request.emit (events.js:172:7)
at Request.<anonymous> (C:\Users\Me\AppData\Roaming\npm\node_modules\browser-perf\node_modules\wd\node_modules\request\request.js:1219:14)
at emitOne (events.js:82:20)
at Request.emit (events.js:169:7)

ChromeDriver error: 

[805.385][SEVERE]: unknown error: session deleted because error encountered when
 notifying listeners of 'GetLog' command
from unknown error: unexpected command response
  (Session info: webview=39.0.0.0)

My application in AVD is starting but crashes after a second. I tried to modify configfile.json and one configuration worked for me: 

{
    "selenium": "localhost:9515",
    "browsers": [{
        "browserName": "cordova",
        "chromeOptions": {
            "androidActivity": "com.myapps.Cordova2.MainActivity",
            "androidPackage": "com.myapps.Cordova2"
        }
    }]
}

But the log that is generated consist of few metrics (cannot find few of them in /wiki/Metrics), many of them are zeros. EDIT: I tried to use perfjankie with selenium but without success due to the error above. Using browser-perf with selenium I have this error: 

  (Session info: webview=39.0.0.0)
  (Driver info: chromedriver=2.22.397933 (1cab651507b88dec79b2b2a22d1943c01833cc
1b),platform=Windows NT 6.3.9600 x86_64) (WARNING: The server did not provide an
y stacktrace information)
Command duration or timeout: 330 milliseconds
Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
System info: host: 'PC', ip: '10.132.222.132', os.name: 'Windows 8.1', os.arch:
'amd64', os.version: '6.3', java.version: '1.8.0_92'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities [{applicationCacheEnabled=false, rotatable=false, mobileEmulationEn
abled=false, chrome={chromedriverVersion=2.22.397933 (1cab651507b88dec79b2b2a22d
1943c01833cc1b)}, takesHeapSnapshot=true, databaseEnabled=false, handlesAlerts=t
rue, hasTouchScreen=false, version=39.0.0.0, platform=ANDROID, browserConnection
Enabled=false, nativeEvents=true, acceptSslCerts=true, locationContextEnabled=tr
ue, webStorageEnabled=true, browserName=chrome, takesScreenshot=true, javascript
Enabled=true, cssSelectorsEnabled=true}]
Session ID: ed5ba86d08ffc8d9ae3fc0a0c74d22d4
11:24:38.480 INFO - Executing: [delete session: 99098571-20a7-4d08-931a-bcd9faa3
7df4])
11:24:39.195 INFO - Done: [delete session: 99098571-20a7-4d08-931a-bcd9faa37df4]

11:25:38.758 INFO - Executing: [new session: Capabilities [{loggingPrefs=org.ope
nqa.selenium.logging.LoggingPreferences@29c92c5b, browserName=chrome, javascript
Enabled=true, chromeOptions={args=[--enable-gpu-benchmarking, --enable-thread-co
mposting], perfLoggingPrefs={traceCategories=,blink.console,devtools.timeline,di
sabled-by-default-devtools.timeline,toplevel,disabled-by-default-devtools.timeli
ne.frame,benchmark}, androidPackage=com.coolappz.AECordova2, androidActivity=com
.coolappz.AECordova2.MainActivity}, version=5.1.1, platform=ANDROID}]])
11:25:38.760 INFO - Creating a new session for Capabilities [{loggingPrefs=org.o
penqa.selenium.logging.LoggingPreferences@29c92c5b, browserName=chrome, javascri
ptEnabled=true, chromeOptions={args=[--enable-gpu-benchmarking, --enable-thread-
composting], perfLoggingPrefs={traceCategories=,blink.console,devtools.timeline,
disabled-by-default-devtools.timeline,toplevel,disabled-by-default-devtools.time
line.frame,benchmark}, androidPackage=com.coolappz.AECordova2, androidActivity=c
om.coolappz.AECordova2.MainActivity}, version=5.1.1, platform=ANDROID}]
Starting ChromeDriver 2.22.397933 (1cab651507b88dec79b2b2a22d1943c01833cc1b) on
port 12742
Only local connections are allowed.
11:25:43.454 INFO - Done: [new session: Capabilities [{loggingPrefs=org.openqa.s
elenium.logging.LoggingPreferences@29c92c5b, browserName=chrome, javascriptEnabl
ed=true, chromeOptions={args=[--enable-gpu-benchmarking, --enable-thread-compost
ing], perfLoggingPrefs={traceCategories=,blink.console,devtools.timeline,disable
d-by-default-devtools.timeline,toplevel,disabled-by-default-devtools.timeline.fr
ame,benchmark}, androidPackage=com.coolappz.AECordova2, androidActivity=com.cool
appz.AECordova2.MainActivity}, version=5.1.1, platform=ANDROID}]]
11:25:43.468 INFO - Executing: [fetching available log types])
11:25:43.470 INFO - Done: [fetching available log types]
11:25:43.478 INFO - Executing: [fetching logs for: performance])
[4.980][SEVERE]: unknown error: session deleted because error encountered when n
otifying listeners of 'GetLog' command
from unknown error: unexpected command response
  (Session info: webview=39.0.0.0)
11:25:43.816 WARN - Exception thrown
org.openqa.selenium.WebDriverException: unknown error: session deleted because e
rror encountered when notifying listeners of 'GetLog' command
from unknown error: unexpected command response
  (Session info: webview=39.0.0.0)
  (Driver info: chromedriver=2.22.397933 (1cab651507b88dec79b2b2a22d1943c01833cc
1b),platform=Windows NT 6.3.9600 x86_64) (WARNING: The server did not provide an
y stacktrace information)
Command duration or timeout: 337 milliseconds
Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
System info: host: 'PC', ip: '10.132.222.132', os.name: 'Windows 8.1', os.arch:
'amd64', os.version: '6.3', java.version: '1.8.0_92'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities [{applicationCacheEnabled=false, rotatable=false, mobileEmulationEn
abled=false, chrome={chromedriverVersion=2.22.397933 (1cab651507b88dec79b2b2a22d
1943c01833cc1b)}, takesHeapSnapshot=true, databaseEnabled=false, handlesAlerts=t
rue, hasTouchScreen=false, version=39.0.0.0, platform=ANDROID, browserConnection
Enabled=false, nativeEvents=true, acceptSslCerts=true, locationContextEnabled=tr
ue, webStorageEnabled=true, browserName=chrome, takesScreenshot=true, javascript
Enabled=true, cssSelectorsEnabled=true}]
Session ID: d46d41d55a3102e3eaeda5fa812d9948
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Sou
rce)
        at java.lang.reflect.Constructor.newInstance(Unknown Source)
        at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.
java:206)
        at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHa
ndler.java:158)
        at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.ja
va:678)
        at org.openqa.selenium.remote.RemoteExecuteMethod.execute(RemoteExecuteM
ethod.java:35)
        at org.openqa.selenium.remote.RemoteLogs.getRemoteEntries(RemoteLogs.jav
a:81)
        at org.openqa.selenium.remote.RemoteLogs.get(RemoteLogs.java:77)
        at org.openqa.selenium.remote.server.handler.GetLogHandler.call(GetLogHa
ndler.java:43)
        at org.openqa.selenium.remote.server.handler.GetLogHandler.call(GetLogHa
ndler.java:1)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at org.openqa.selenium.remote.server.DefaultSession$1.run(DefaultSession
.java:176)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
11:25:43.820 WARN - Exception: unknown error: session deleted because error enco
untered when notifying listeners of 'GetLog' command
from unknown error: unexpected command response
  (Session info: webview=39.0.0.0)
  (Driver info: chromedriver=2.22.397933 (1cab651507b88dec79b2b2a22d1943c01833cc
1b),platform=Windows NT 6.3.9600 x86_64) (WARNING: The server did not provide an
y stacktrace information)
Command duration or timeout: 337 milliseconds
Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
System info: host: 'PC', ip: '10.132.222.132', os.name: 'Windows 8.1', os.arch:
'amd64', os.version: '6.3', java.version: '1.8.0_92'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities [{applicationCacheEnabled=false, rotatable=false, mobileEmulationEn
abled=false, chrome={chromedriverVersion=2.22.397933 (1cab651507b88dec79b2b2a22d
1943c01833cc1b)}, takesHeapSnapshot=true, databaseEnabled=false, handlesAlerts=t
rue, hasTouchScreen=false, version=39.0.0.0, platform=ANDROID, browserConnection
Enabled=false, nativeEvents=true, acceptSslCerts=true, locationContextEnabled=tr
ue, webStorageEnabled=true, browserName=chrome, takesScreenshot=true, javascript
Enabled=true, cssSelectorsEnabled=true}]
Session ID: d46d41d55a3102e3eaeda5fa812d9948
11:25:43.838 INFO - Executing: [delete session: 8cf68c19-795c-4fb8-a301-bbebbdc9
cae2])
11:25:44.553 INFO - Done: [delete session: 8cf68c19-795c-4fb8-a301-bbebbdc9cae2]

With this config:

{
    "selenium": "localhost:4444/wd/hub",
    "browsers": [{
        "browserName": "chrome",
        "platform": "android",
        "version": "5.1.1",
        "chromeOptions": {
            "androidActivity": "com.coolappz.AECordova2.MainActivity",
            "androidPackage": "com.coolappz.AECordova2"
        }
    }]
}