Closed LarsDenBakker closed 4 years ago
Aside from test coverage, we now have all the logging info we need. The focus has been on getting all this info, we definitely need to think about how we want to format it and which things we might not want to display.
This is the general format we have right now:
<test file a>:
<test name>: (failed on <browser name>)
at: <test file a>:<line>:<column>
debug in browser: <link to debug in browser>
error: <error msg>
<error rich diff or error stack>
<another test name>: (failed on <browser name>)
at: <test file a>:<line>:<column>
debug in browser: <link to debug in browser>
error: <error msg>
<error rich diff or error stack>
Browser logs:
<Log A>
<Log B>
<Browser name> logs:
<Log A>
Request 404s:
<path a>
<path b>
<Browser name> request 404s:
<path a>
<path b>
<test file b>:
Failed to load test file
<error stack>
<test file c>:
Unknown errors:
<browser name a>
<error stack>
<browser name b>
<error stack>
Test coverage: <total> %
Lines: xx.xx %, Statements: xx.xx %, Branches: xx.xx %, Functions: xx.xx %
Finished running tests!
Total: |██████████████████████████████| 34/34 test files | 212 passed, 7 failed
Chromium: |██████████████████████████████| 34/34 test files | 212 passed, 4 failed
Firefox: |██████████████████████████████| 34/34 test files | 210 passed, 6 failed
Webkit: |██████████████████████████████| 34/34 test files | 209 passed, 7 failed
Duration: 40s
Some general thoughts about the structure:
We show the filename multiple times, once as the "category" at the root and once per test with the column + line so that you can jump directly to a test. Along with potentially a link to debug in the browser it feels like quite a lot of clutter.
Should we just show one total progress bar, and no progress bar per browser? I do like being able to see which browser is being slower, so that I know the impact of running tests on multiple browser during development for example.
Jest shows a code frame with the error location, for examle the expect where it went wrong. Do we think this helps? It adds another 7 or so lines in the terminal for an error.
Should we have two separate error reports, a summary overview when there are a lot of errors and a detailed view when there are few errors or when you are focusing on a test?
How to display syntax errors? Right now we just let the logs from es-dev-server come in, but we need to place it somewhere probably.
imho I like it 👍
the debug in browser
we could display only if --debug
is supplied?
for the summary I would prefer a view like this
Total: |██████████████████████████████| 34/34 test files | 212 passed, 7 failed
--- Browsers? e.g. some separator, indentation seem strange ---
Chromium: |██████████████████████████████| 4 failed
Firefox: |██████████████████████████████| 6 failed
Webkit: |██████████████████████████████| 7 failed
for the browser I am only interested in how many failed... all the other details I have in Total
@daKmoR --debug
opens the browser already, this link would be for when you're just noticing a failed test and want to see what's going on. I think I'd use that a lot personally.
Test coverage was added now, I updated the format a few comments above. What I'm doing now is if your tests pass the coverage threshold, we just show the total coverage %. If you're failing one of the coverage markers, it displays the second line with the separate markers.
how will that work? 🤔
e.g. I do npm run test
and it finishes (e.g. server is no longer running) and then I click the link?
or will the link only be added while in watch mode?
Mhm yes it would be watch only
@thepassle and I improved the terminal logging a lot today. Let's go with this, and improve it over time. I'll close this issue, and we can create issues for separate topics.
One thing to improve still is https://github.com/LarsDenBakker/web-test-runner/issues/52
When we have all the information for the terminal available, we should do a complete review of the logging:
TODO: https://github.com/LarsDenBakker/web-test-runner/issues/37