Allow to specify custom compilation options

[AndreyBelym]

Are you requesting a feature or reporting a bug?

Enhancement

What is the current behavior?

There is no API to tweak Babel compilation options, and .babelrc files are ignored.

What is the expected behavior?

Apply options from .babelrc files or/and provide an API to configure the options from JavaScript.

[Asher-]

Why does this seem to be standard practice on ES6 projects? ES6 code should never, ever, under any circumstances whatsoever, be tested under a transpilation environment. Only once ES6 code is working should any consideration of transpilation be considered at all. In no situation should the default cause transpilation.

If the transpiler works, so will the transpiled code (assuming it passes tests prior to transpilation). Testing under transpilation should be a special case for use exclusively prior to final deployment.

[Viperwow]

In my opinion, it would be great to support the custom compilation configurations. Because it opens much more options for developers in supporting the only features they need without limitations.

[marcinlesek]

+1, it would be really helpful to get this!

[anonmos]

Why does this seem to be standard practice on ES6 projects? ES6 code should never, ever, under any circumstances whatsoever, be tested under a transpilation environment. Only once ES6 code is working should any consideration of transpilation be considered at all. In no situation should the default cause transpilation.

If the transpiler works, so will the transpiled code (assuming it passes tests prior to transpilation). Testing under transpilation should be a special case for use exclusively prior to final deployment.

I'm assuming this has something to do with ES Modules still being experimental in NodeJS. Otherwise all of TestCafe's users would only be allowed to use the older, arguably less convenient CommonJS format for things like import/export.

That said, I really don't think this is a difficult change. I'm looking at interoperability with VueJS at the moment, and Babel seems to be constrained to these areas of the TestCafe code. If I end up going the route of making any modifications, I'll open a PR.

==================================================================== Follow up here: I was running into an issue where the TestCafe Babel process was trying to compile my Vue templates and failing. Debugging into the way that TestCafe handles Babel compilation of files with the .js extension, it is built to normally handle only the test files, for which it is properly configured internally. The idea is that your web server is serving your custom-Babel-pre-compiled-web code, and the test files are isolated and compiled by TestCafe itself. Any imports that mix the two will introduce failures and a need for custom Babel processing. The conclusions I've come to are these:

• Tests need to avoid importing anything from your web app code (in my case, an enum from a file with other imports leading to Vue templates that need custom Babel configs). The Babel core compiler that TestCafe uses will follow the import chain and eventually try to compile a file that it's not properly configured to handle.
• My use case doesn't justify a custom Babel config import API for TestCafe.

While I can see downsides to this approach (lack of ability to code share between the app and tests), it does keep the compilation process simple, and make the out-of-box install of TestCafe extremely straightforward (no configs!). I won't be opening a PR for this, now that I know how it all works. Perhaps making a note of this somewhere in a troubleshooting section would help alleviate this issue for other users?

[AlexKamaev]

Thank you for such a detailed description how TestCafe works with Babel internally. You are right, it's really better to isolate your test code from your app code. However, I think, it can be useful to have a capability to customize babel for other users, so we will keep this issue open. Besides, I think it's good idea to make a note in the docs, so I'll discuss it with the team.

[joshuwaaaaa]

Any updates on this feature? I noticed that %CPU usage spikes (100-250%) before every test which causes significant lag, and node --inspect profiling shows compileClientFunction at the top of the list:

[LavrovArtem]

@joshua-wan, Thank you for the provided information. I cannot give you any estimates on when the feature will be implemented.

[joshuwaaaaa]

@LavrovArtem thanks for the reply! Would it be possible to separate the build step from running the tests ? For example, after testcafe build bundles the test, then testcafe run can simply run the test without doing any compiling. Alternatively, maybe a webpack or babel plugin could be added to webpack/babel so that those tools bundle the test file in a way that TestCafe can consume directly without doing run-time compilation. Thoughts?

[joshuwaaaaa]

@LavrovArtem sorry for the double-posting , just want to share some additional findings

I noticed that this same this.fn is stringified over 2500 times during a test run

(...args) => {
/* eslint-disable no-undef */
const node = selector();
return customMethod.apply(customMethod, [node].concat(args));
/* eslint-enable no-undef */
}

for this line: https://github.com/DevExpress/testcafe/blob/master/src/client-functions/client-function-builder.js#L114

[joshuwaaaaa]

So I tried adding a cache for that to reduce the calls that are just stringify-ing the same code. Then, it didn't work because these are actually new instances of ClientFunctionBuilder altogether. I added a log to the constructor to see how many times it was getting instantiated. About 4000+ times 🤔 Is that normal?

[joshuwaaaaa]

As a follow-up, I noticed that the serialization step is a very expensive operation (below screenshot shows a lot of time spent in the replicator library) I am curious what the purpose of serialization is and if somehow an optimization can be done around it ? (I have not been able to find much information on StackOverflow, the DevExpress forum, or main docs)

[Dmitry-Ostashev]

@joshua-wan Could you please clarify if you are trying to import some third-party modules in ClientFunction? Also, I suggest you create a separate issue about serialization optimization in the https://github.com/inikulin/replicator repository.

As for your idea about separating the build step, we'll consider it during implementation of this feature.

[joshuwaaaaa]

@Dmitry-Ostashev thank you for the reply !

to answer your question, I did not bring in any new third-party libraries while experimenting. i just tried various simple ways to short-circuit the toString function by providing a pre-built string when possible. i later realized that was probably not the bottleneck (even though it does seem like the same thing goes through toString many times (like described above, it seemed like the same thing needed to be stringified thousands of times, but again, seems like not the bottleneck compared to the serialization/encoding step in replicator).

i can post an issue on that repository if you like. i am still curious about the design/architecture/intention behind the serialization in the first place (could not find many discussions about this anywhere online). something to do with why testcafe does not need to be based on selenium ? i am not implying this should be done but just as a question to indicate just how little i know: what happens if there is no serialization step at all ? what breaks ?

i suspect there is a limit to how much the serialization can be optimized, because there will be a cost no matter what in order to achieve the kind of robust (all edge cases handled, etc.) kind of serialization that replicator is designed for, so that is why i wish it could be skipped altogether.. but i am sure it has an important purpose

[AlexKamaev]

@joshua-wan Serialization is required because of our architecture. You can read about this in the following help topic How it works . In short, when you run tests you run them on the server side. Let me explain on a simple example. The following code is Node.js:

test('test', async t => {
    await t.click(Selector('h1').withText('Header'));
});

However, we need to find the h1 element with the Header text on the browser side. So, TestCafe serializes this selector, and runs it in the browser.

[joshuwaaaaa]

@AlexKamaev thank you for the explanation! some of the selectors we have are more complex than the h1 example, so it seems we may be bumping into an unhappy path for the serialization step because our selector is more complex. I am curious if you have any best practices in mind for these Selectors then . I may continue investigating our selectors to see what characteristics they have to make them lead to this degraded serialization performance but we are already using the testcafe-react-selectors and follow many of those patterns there

[miherlosev]

For the team: Use infrastructure implemented in this PR for option specifying.

[wentwrong]

To clarify the current state of implementation: in the v1.10.0 release, we introduced a new API to adjust compiler options. Currently, the implementation is limited to the TypeScript compiler, and compiler options are only applied to tests written in TypeScript. As for other compilers and JavaScript/CoffeScript test files, this functionality is not yet supported, but the new API I mentioned is designed to accept options for different compilers (like Babel) in future versions.

[miherlosev]

Hi folks,

We've implemented TypeScript Custom Compiler options. I will close this issue because it does not contain usage scenarios for custom Babel compilation options. If you have such a use case, please reopen this issue and describe it in detail.