search

modernweb-dev / web

Guides, tools and libraries for modern web development.
https://modern-web.dev
MIT License
2.2k stars 278 forks source link

What to do about Saucelabs and browserstack tests? #2489

Open koddsson opened 11 months ago

koddsson commented 11 months ago

They seem to not be working or haven't worked for some time. They both fail on configuration as the accounts for both services aren't set as part of the CI run. Is there any interest in keeping these running or should we just remove them?

Example PR with failing tests: https://github.com/modernweb-dev/web/pull/2488. Example failed runs: https://github.com/modernweb-dev/web/actions/runs/6551403225/job/17792503350?pr=2488 https://github.com/modernweb-dev/web/actions/runs/6551403222/job/17792503355?pr=2488

A screenshot of a "checks" section of a pull request on github. It has five checks, two of which are browserstack and saucelabs and they are both failing.

cc-ing some people that might have oppinions; @LarsDenBakker @thepassle @Westbrook @bashmish

koddsson commented 10 months ago

I've come to the conclusion that we remove them and I'll do so in the near future. Ping me here if you've got other thoughts.

koddsson commented 10 months ago

I even wonder if we should deprecate the saucelabs and browserstack plugins. They can still exist as community plugins but not within the modern web repository since we don't seem to have the capacity to maintain them.

Alternatively if someone would update the secrets we could keep maintaining them. I just never use these plugins so I have a hard time figuring out how much they are used.

bashmish commented 10 months ago

I even wonder if we should deprecate the saucelabs and browserstack plugins. They can still exist as community plugins but not within the modern web repository since we don't seem to have the capacity to maintain them.

Alternatively if someone would update the secrets we could keep maintaining them. I just never use these plugins so I have a hard time figuring out how much they are used.

To me they were extremely useful plugins in the past. For library creators it's very useful, see Lit uses both https://github.com/lit/lit/blob/main/package.json#L267

If the fix is just about maintaining the access tokens for SauceLabs and BrowserStack, I can work on that. For the latter I have an open source account since a long time, for SauceLabs I think I can request one.