search

tape-testing / tape

tap-producing test harness for node and browsers
MIT License
5.77k stars 307 forks source link

Only the first test file runs when using Karma & Webpack #525

Closed petermetz closed 4 years ago

petermetz commented 4 years ago

Branch where this can be observed/reproduced: https://github.com/suyukuoacn/cactus/tree/key_converter

Karma config: https://github.com/suyukuoacn/cactus/blob/key_converter/karma.conf.js

Reproduction steps:

  1. clone the branch linked
  2. cd to project root
  3. npm install
  4. npm run test:browser
  5. Observe that only the first test file's assertions get executed even though all the other files were loaded as well (which you can verify by setting the karma log level to debug)
petermetz commented 4 years ago

Additional finding: If I create a single file that imports all the other test cases and then configure karma to run just that one, then everything works as expected (all tests are executed). This leads me to believe that the issue lies with the separate webpack bundles not working together as they should. Not sure if this is a karma-webpack issue at this point or a karma-tap issue or something with tape itself.

ljharb commented 4 years ago

This does seem like a karma thing - certainly if you have multiple files, node 'test/**' won't work but tape 'test/**' will.

Can you confirm whether all the test files actually get included in the karma bundle?

petermetz commented 4 years ago

Can you confirm whether all the test files actually get included in the karma bundle?

@ljharb Yes, they are, when I attach a debugger I can see that in fact all the tests are running, but what happens is that because there is a separate bundle for each test file that was matched by the glob pattern (that's how karma-webpack does it apparently) the first bundle that gets loaded executes it's test cases (one test file) and then sends the notification to karma that the tests have completed, but it only mentions the test cases that were included in the first bundle (so I see 2 out of 2 succeeded instead of 44 out of 44 for example).

ljharb commented 4 years ago

ah, interesting. i think the issue is that either they all need to share the same tape instance, or they all need to be in the same bundle. Presumably karma-webpack can be configured to address that?

petermetz commented 4 years ago

ah, interesting. i think the issue is that either they all need to share the same tape instance, or they all need to be in the same bundle. Presumably karma-webpack can be configured to address that?

Looking into that as we speak, not yet sure if karma-webpack will allow me to specify my own entry points because their documentation says that the entry points are constructed directly from the karma config's own files array/globs/etc, but that's something to do with karma-webpack either way. In the meantime, I've been getting more and more confident that this isn't an issue with tape itself so thank you for the illuminating conversation it helped me think through the issue and with that said, I'll close this now. :-)