plocket
commented
2 years ago A: ...it might make sense to make it a step, something like "Given all of the interviews listed on the server / listed on http:/... open". Not listing the server could always just fallback and use the server URL in the env vars, and it satisfies both of our concerns about not using env vars to control flow and not having to make a completely separate action with lots of duplicated work there.
B: ...we can prevent the Step from stopping like you did with accessibility and try to do a good job with the report...We'd need to abstract the whole going to the interview step into scope.js, but I've been thinking about doing that for a while anyway.
A: Exactly. I'm also still not sure how exactly people would want to run this "test the whole server at once" process. Testing installed packages is definitely useful if we want to auto-deploy-and-test packages from github, but that's just for one package at a time; there's not really a specific repo that the whole server test should be associated with IMO. So making it a step is useful, but maybe makes the scope a bit too big? Idk yet. +1 for a lighter steps.js and having more stuff in scope.js. Makes us a bit more independent of cucumber.
B...the action.yml needs to skip the setup step so that it doesn't try to pull the repo. And the takedown step... That can be in our action.yml or in the js itself
A: If we need to skip those steps, IMO that should be in the action.yml. Buried less deep in the code, so easier to find, and acts a bit closer to how yours actually would as a developer (do I need to install this to the playground? No, then I'll skip running npm run setup, etc.) That's what I mean by "I'm not sure how people will want to run the test the whole server at once thing", idk if they'll want it in a repo with any other code at all. Then it's just a matter of checking out the repo, and "is there anything to install? No? Then keep going!".
BryceStevenWilley
Not sure how possible this flow is, but we can give it a shot.
MVP: Us the server's list of interviews to open each interview to make sure each loads without problems. This should avoid installing anything on the server. I think we can use our same action script, just use a new input or environment variable in the dev's action script. Something like
JUST_TEST_INTERVIEW_LIST. It would set a flag that instead of thesetupscript, we'd run a different (new) script that would just test the interview list. That way, we can make this accessible to everyone and not make devs write a different kind of workflow file just for this.Enhancement: Allow devs to let each interview use its own test feature files to test the interviews. If there are none, just load the interview.