search

mozilla / pdf.js

PDF Reader in JavaScript
https://mozilla.github.io/pdf.js/
Apache License 2.0
47.14k stars 9.82k forks source link

Enable dumpio when running the tests in order to have some useful debug data #18260

Closed calixteman closed 2 weeks ago

calixteman commented 2 weeks ago

It could help to figure out what's wrong in #18259.

calixteman commented 2 weeks ago

/botio test

moz-tools-bot commented 2 weeks ago

From: Bot.io (Linux m4)

Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/34fcd127eefeaf9/output.txt

moz-tools-bot commented 2 weeks ago

From: Bot.io (Windows)

Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/cd69d4693491265/output.txt

moz-tools-bot commented 2 weeks ago

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/34fcd127eefeaf9/output.txt

Total script time: 28.57 mins

  different ref/snapshot: 12
  different first/second rendering: 2

Image differences available at: http://54.241.84.105:8877/34fcd127eefeaf9/reftest-analyzer.html#web=eq.log

moz-tools-bot commented 2 weeks ago

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/cd69d4693491265/output.txt

Total script time: 44.23 mins

  different ref/snapshot: 5

Image differences available at: http://54.193.163.58:8877/cd69d4693491265/reftest-analyzer.html#web=eq.log

timvandermeij commented 2 weeks ago

Hm, I'm a bit on the fence about this. On the one hand it did already provide useful information and surfaced issues such as #18246 and the logs below, but on the other hand it also generates a lot of log spam for especially the integration tests which makes navigating the logs harder.

I'd feel better about this if we could tidy the logs up a bit first:

What do you think?

calixteman commented 2 weeks ago

Yes there's some noise :(, especially those exceptions we've because of DecompressionStream (I must file a bug about that). We should try to fix the createBundleFallback one. That said in considering that it helped to fix one bug (in integration tests) and it's the best idea I've for the moment to be able to potentially catch something from this intermittent failure, I'd say we should do it.

timvandermeij commented 2 weeks ago

It could help to figure out what's wrong in https://github.com/mozilla/pdf.js/issues/18259.

Unfortunately I found out that this patch won't help with that particular issue; see https://github.com/mozilla/pdf.js/issues/18259#issuecomment-2173874135.

I'm still somewhat in favor of doing this, but ideally once the integration test logs are cleaned up at least a little bit, to avoid the logs becoming unwieldy to read through. I'll create dedicated issues for the most important/spammy ones because this patch did nicely surface those issues, so that in itself is really good.

timvandermeij commented 2 weeks ago

I have created the two issues above for the most important log spam problems here. I think once those are resolved we should be in a state where we can enable dumpio: true, which coming to think of it could also help with future patches because it'll surface new logs for PRs instead of them going by unnoticed as was the case until now.

Snuffleupagus commented 2 weeks ago

Can we perhaps revisit this one now, given the PRs that landed earlier today?

timvandermeij commented 2 weeks ago

Yes, overall the logs should be in a much better state now that the two mentioned issues have been fixed. @calixteman Could you rebase this patch and trigger a new test run to verify that the logs should be in a good enough state now that we can enable this?

calixteman commented 2 weeks ago

/botio test

moz-tools-bot commented 2 weeks ago

From: Bot.io (Windows)

Received

Command cmd_test from @calixteman received. Current queue size: 2

Live output at: http://54.193.163.58:8877/32bb151d23ef0ea/output.txt

moz-tools-bot commented 2 weeks ago

From: Bot.io (Linux m4)

Received

Command cmd_test from @calixteman received. Current queue size: 1

Live output at: http://54.241.84.105:8877/2495d314125b7a2/output.txt

moz-tools-bot commented 2 weeks ago

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/2495d314125b7a2/output.txt

Total script time: 29.39 mins

  different ref/snapshot: 19
  different first/second rendering: 1

Image differences available at: http://54.241.84.105:8877/2495d314125b7a2/reftest-analyzer.html#web=eq.log

moz-tools-bot commented 2 weeks ago

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/32bb151d23ef0ea/output.txt

Total script time: 42.76 mins

  different ref/snapshot: 6

Image differences available at: http://54.193.163.58:8877/32bb151d23ef0ea/reftest-analyzer.html#web=eq.log

timvandermeij commented 2 weeks ago

This has clearly proven its value, and the logs are much more under control now. Good idea to enable this! I'll create a meta-issue to clean up the logs further, but it's not blocking landing this anymore.