Open llkats opened 7 years ago
martinheidegger
commented
7 years ago I am not sure CI is possible. The issue that usually occurs is that on windows the tty doesn't work correctly. Testing it would mean to actually have a tty tool (that has different timings) and that is executed (per hand?) to make sure that the tty works.
ghost
commented
7 years ago Workshops can have other kinds of tests. This test ensures all the recommended solutions pass: https://github.com/workshopper/browserify-adventure/blob/master/test/solutions.js
llkats
commented
7 years ago I should try this again, since I misunderstood the initial case that @martinheidegger meant. Is there a way to test workshoppers against new and upcoming versions of node/npm? For example, TTY no longer working in Windows.
martinheidegger
commented
7 years ago @substack there is workshopper-adventure-test that should do that even further. But those are automated tests.
The real problem, that we can't tackle easily, are integration tests. Depending on Node-version there are various things that can fail. i.e.:
not-working, sometimes-working, `conditionally working ー this is by far the most annoying and difficult issue to deal with ー; also, the control characters could mingled resulting in wrong output)npm i sometimes on windows)So: In short: I think the tests that need interaction with other parties are of a nature to prevent bugs like above. I am not sure how many of those can be well automated considering that this my test matrix:
| Platform | Win 7 | Win 8 | Win 10 | Mac | Linux |
|---|---|---|---|---|---|
| Terminal | ☑️ | ☑️ | |||
| iTerm2 | ☑️ | ||||
| Hyperterm | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ |
| Command Prompt | ☑️ | ☑️ | ☑️ | ||
| Powershell | ☑️ | ☑️ | ☑️ | ||
| Cygwin | ☑️ | ☑️ | ☑️ | ||
| Github Shell | ☑️ | ☑️ | ☑️ | ☑️ | |
| Node Command Prompt | ☑️ | ☑️ | ☑️ | ||
| Cmder | ☑️ | ☑️ | ☑️ |
Let's discuss setting up a CI-type solution to test workshoppers comprehensively before release.jk sorta, please see this comment below.