search

mozilla / pdf.js

PDF Reader in JavaScript
https://mozilla.github.io/pdf.js/
Apache License 2.0
48.71k stars 10.02k forks source link

Use WebDriver BiDi for Chrome #17961

Open timvandermeij opened 7 months ago

timvandermeij commented 7 months ago

I have been looking at the integration test code and noticed the following code: https://github.com/mozilla/pdf.js/blob/master/test/test.mjs#L908-L910

Some time ago I tried using WebDriver BiDi for Chrome too and I recall it failed pretty soon back then. However, I tried again today with the current Puppeteer version and the results are much better now: only one test failure remains! Hence I figured now might be a good time to re-investigate WebDriver BiDi compatibility for Chrome, since it would be great if we could move away from CDP for all browsers we support.

I tried debugging this test, FreeText Editor Freetext must stay focused after having been moved must keep the focus, but I can't figure out why this works with CDP and fails with WebDriver BiDi. The traceback of the failing test is:

Failures:
1) FreeText Editor Freetext must stay focused after having been moved must keep the focus
  Message:
    ProtocolError: Protocol error (script.callFunction): unknown error Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed. ProtocolError: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at Connection._rawSend (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:77:26)
        at CdpCDPSession.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
        at CDPClientAdapter.sendCommand (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/bidi/BidiOverCdp.js:106:39)
        at WindowRealm.callFunction (/home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/chromium-bidi/lib/cjs/bidiMapper/modules/script/Realm.js:243:58)
        at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
        at async WindowRealm.callFunction (/home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/chromium-bidi/lib/cjs/bidiMapper/modules/script/WindowRealm.js:136:16)
        at async ScriptProcessor.callFunction (/home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/chromium-bidi/lib/cjs/bidiMapper/modules/script/ScriptProcessor.js:59:16)
  Stack:
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at BidiConnection.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/bidi/Connection.js:51:32)
        at Session.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/bidi/core/Session.js:172:42)
        at Session.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/util/decorators.js:94:27)
        at WindowRealm.callFunction (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/bidi/core/Realm.js:92:51)
        at WindowRealm.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/util/decorators.js:94:27)
        at #evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/bidi/Realm.js:126:42)
        at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Interestingly I found that the test passes with WebDriver BiDi if I comment out this: https://github.com/mozilla/pdf.js/blob/7290faf840ab5a3828d12bbbb460bd8ad49f6ffa/test/integration/freetext_editor_spec.mjs#L2362-L2366

I have enabled verbose logging using the Puppeteer debugging instructions from https://github.com/puppeteer/puppeteer/blob/ef8c4c808aa9dddd4d2501c8fc160cfcd0c4b9d1/docs/guides/debugging.md?plain=1#L131 but I can't really interpret the output because I don't know much about the WebDriver BiDi protocol and Puppeteer internals.

To reproduce the issue:

  1. Clone the repository.
  2. Run npm install.
  3. Change cdp to webDriverBiDi in https://github.com/mozilla/pdf.js/blob/7290faf840ab5a3828d12bbbb460bd8ad49f6ffa/test/test.mjs#L881
  4. Change it to fit in https://github.com/mozilla/pdf.js/blob/7290faf840ab5a3828d12bbbb460bd8ad49f6ffa/test/integration/freetext_editor_spec.mjs#L2315 to only run this test.
  5. Run npx gulp integrationtest and notice that the test hangs. If the protocol is changed to CDP the test passes. The verbose debugging logging can be obtained with env DEBUG="puppeteer:*" npx gulp integrationtest, for which it helps to change https://github.com/mozilla/pdf.js/blob/master/test/test.mjs#L961 to only run Chrome (otherwise Firefox also spams the logs).

@whimboo Is this perhaps something you could help out with? It could have surfaced a bug in the WebDriver BiDi implementation, but because I'm stuck on debugging this I also can't create a reduced test case to report to upstream...

OrKoN commented 3 months ago

Also, the use of timeout: 0 is not recommended as it will never throw.

whimboo commented 3 months ago

@timvandermeij would it be possible to run a script before and after the tests are done that would print out all the still running Chrome processes? That way we probably could see if a given test job leaves processes behind. Also maybe a cleanup could happen after the tests when all remaining browser processes are killed. Given that you re-use the same machine and don't reset state in-between runs piling up Chrome processes might explain the OOM.

timvandermeij commented 3 months ago

Thank you both the feedback! I think we should try this again with the most recent Puppeteer version in place, and if it still happens we should indeed gather more debug information since I do realize it's not a lot of information to go by right now. I'm planning to update #18590 soon so we can upgrade to Puppeteer 23 first.

whimboo commented 2 months ago

Hi @timvandermeij. Do you have an update for this issue?

timvandermeij commented 2 months ago

We finished the Puppeteer 23 upgrade, so we should be able to try this again now. We waited a bit due to holidays in which we didn't want to potentially break the bots. I currently don't have a lot of time to work on this, so if anyone wants to make a PR to enable BiDi for Chrome again and if possible add some useful debugging tools, please do (otherwise I'll see what I can do later).