Is client side A/B testing always a bad idea in your experience?

In a word, yes.

In two words, yes yes.

From a performance point of view, it has so many drawbacks. In no particular and non-exhaustive order:

It typically blocks rendering. Most third parties are loaded in a synchronous manner so as to avoid the restyling and changing of content. This render- and parser-blocking behaviour contributes massively to delays.
Providers are almost always off-site. Most third-party providers exist on external origins, meaning trips to new domains: DNS, TCP, TLS overhead on the critical path ain’t the one.
It happens on every page load. Even though we might only take the download cost once, caching the file for subsequent use in-session, the runtime cost happens on every viewing of the tested page.
No user-benefitting reuse. Further to the above, because we’re not doing our A/B tests on the server, it’s not possible to build a variant once and serve it to many users. Every user has to build their own tests on their own device every time they visit a tested page.
They likely skip any governance process. While your engineers are subject to linting, code-reviews, tests, auditors, and more, your marketing team have free rein of the front-end. They can upload 6MB images, they can probably paste JavaScript into a WYSIWYG, you name it.

The benefits of client-side A/B tests really only suit the marketing team:

Test without a deploy. They don’t need developers, so they can experiment as much and as often as they like.
Available to everyone. All you need is longin details for your provider. You don’t need a coding background, write/push access to repos or CI, etc.

To mitigate the effects:

Exercise restraint. Don’t test huge variants. Subtle and small tests are not only more focused (and thus likely to provide more reliable insights), they’re also less expensive and less taxing to run.
Load asynchronously. If you are limiting yourself to just small, discrete variations, you may be able to get away with loading your A/B testing tool asynchronously.
Keep it clean. Regularly audit and review the tests that are live and remove any that are old or not active.
Preconnect off-site origins. If you do have trips to third party origins, use a preconnect HTTP header to warm up a connection.

csswizardry / ama