Raises random End_of_file exceptions

[gaku-sei]

When using tailwind-ppx on circle ci we get random End_of_file exceptions:

Here is the whole command ran by bsc

/home/circleci/workspace/node_modules/bs-platform/linux/bsc.exe  -w -48 -warn-error +A+8+11+12+16+26+27+31+32+33+34+35+39+44+45+101+102+104+105 -color always -bs-jsx 3 -ppx '/home/circleci/workspace/node_modules/@dylanirlbeck/tailwind-ppx/tailwind-ppx -path ./assets/styles.css' -ppx '/home/circleci/workspace/node_modules/@baransu/graphql_ppx_re/ppx6 -lean-parse' -ppx /home/circleci/workspace/node_modules/decco/ppx -ppx /home/circleci/workspace/node_modules/bs-optic/lenses-ppx -bs-super-errors -open Relude_Globals -o src/components/pages/ContactUsComplete.reast -bs-syntax-only -bs-binary-ast /home/circleci/workspace/packages/app/src/components/pages/ContactUsComplete.re

I thought it could be related to https://github.com/BuckleScript/bucklescript/issues/4103, but there is only one run at a time 😕

It could be related to the issues with monorepos: https://github.com/BuckleScript/bucklescript/issues/4361, but we don't have any mutual dependencies, and everything works properly locally...

edit: The raised issues are pretty random, and we also get some Fatal error: exception: Failure("input_value: truncated object")

[dylanirlbeck]

Hmm this is weird, I haven't seen this before on my end. Can I ask what version of BuckleScript you're using?

[gaku-sei]

Sorry for the long hiatus, I'm still experiencing this issue (it's actually worse now the project is bigger....) CI fails ~80% of the time (when it works perfectly locally, and it still succeeds sometimes).

The issue seems to be related to Bucklescript, so I will open an issue on their repo, and link this issue 😕

As for the version used, we're using bs-platform 7.3.2, but we already faced this kind of issue with previous versions.

[gaku-sei]

After digging more into the issue on the Bucklescript repo, it looks like the issue could be related to the ppx (https://github.com/BuckleScript/bucklescript/issues/4469#issuecomment-648524896).

It could occur when the ppx is spawn multiple times simultaneously.

[dylanirlbeck]

@gaku-sei Gotcha, thanks for the heads up. I'm not sure where in the code we could be spawning the ppx multiple times simultaneously -- I'll need to ask in the Discord for some expert guidance on this. If you can, it'd also be worthwhile for you to go through the code to see if you know where the bug might be occurring (and potentially open a PR!)

[gaku-sei]

Thank you very much @dylanirlbeck 🙏

To be honest I'm completely clueless about what's going on here 😕 I tried several things, like caching lib and .graphql_ppx_cache but it still fails. It seems that when the compilation takes too much time, errors will occur (and since our project grows, errors heppen more often ^^), but I'm not even 100% certain...

If there is a way to enforce BuckleScript to use only one thread, and to spawn ppx calls sequentially, I will try it, so that we can at least be certain that's the issue 😕

[gaku-sei]

It seems not possible to force bucklescript to use one thread only, but i noticed something: after removing all the calls to the ppx [%tw ...] and even though the ppx is still called, ci always works 😕

[vdanchenkov]

Happens for us in random files but very rare, on average one build in 30 fails.

12:03:24.698  
/vercel/4cdbb568/node_modules/bs-platform/linux/bsc.exe   -color always -bs-jsx 3 -ppx /vercel/4cdbb568/node_modules/@dylanirlbeck/tailwind-ppx/tailwind-ppx -ppx /vercel/4cdbb568/node_modules/re-formality/ppx -ppx /vercel/4cdbb568/node_modules/decco/ppx -bs-super-errors -o components/Menu.reast -bs-syntax-only -bs-binary-ast /vercel/4cdbb568/components/Menu.re
12:03:24.698  
Fatal error: exception End_of_file
12:03:24.698  
  We've found a bug for you!
12:03:24.698  
  /vercel/4cdbb568/components/Menu.re
12:03:24.698  

12:03:24.699  
  Error while running external preprocessor
12:03:24.699  
Command line: /vercel/4cdbb568/node_modules/@dylanirlbeck/tailwind-ppx/tailwind-ppx '/tmp/camlppx6d1f17' '/tmp/camlppx2b6434'

As you see there are several ppx in use, but I'm posting it here simply because I found this issue in global search.

[dylanirlbeck]

Thanks everyone for commenting -- I'll try and look into this more when I find some time, but I agree that this issue could be very frustrating.

[tsnobip]

I don't know if it's related, but after cleaning up, my first build systematically fails with:

 tailwind.css file not found in project hierarchy. You may need to manually set the path to the file with the -path argument.

then it builds index.css and the following incremental builds work correctly. I also use a very simple monorepo setup with yarn, might be related.

[dylanirlbeck]

@tsnobip Do you mind filing a separate issue (with your directory structure and other important info.) for bookkeeping purposes? My impression is that this is not related, but I very well might be wrong.

[tsnobip]

Sure, no problem :)

[vdanchenkov]

@gaku-sei I suggest you to check on latest version (0.7.9). We are seeing no build errors so far after the update.

[gaku-sei]

Thank you @vdanchenkov 😄

Actually, I had to disable the ppx some times ago since it failed all the time and became impossible to keep 😢 But I really would like to bring it back actually, and we have a script ready for when the bug will be fixed.

I tried it again with version 0.7.9 but it still fails randomly with the same error. Just so you know we have more than 800 ppx calls in our app, and the more you call the ppx the more you face this error 😕

Still unsure why though, is it an issue with the file system on Circle CI? Or with the way the ppx open/read/close files?

[vdanchenkov]

@dylanirlbeck Key for bug reproduction is .tailwind_ppx_cache. If file is present I see no errors. If I delete it before the build, I'm having local error just like on CI.

Knowing that I made reproduction repo https://github.com/vdanchenkov/tailwind-ppx-test

Added CircleCI there, but I'm not sure if build are visible by public.

Build on that repo fails more often than succeeds on my laptop.

[dylanirlbeck]

@vdanchenkov Thanks a lot for that tip. I'll be looking into this issue (and others) as soon as I get some time freed up.

[gaku-sei]

Not 100% sure about that :/

When starting using the ppx we had ~100 ppx calls and started to notice the error reported above. So the first thing we tried was to cache the .tailwind_ppx_cache file in CircleCi which did solve the issue.... For a while. When reaching > 500 ppx calls (~1000 now) we face the issue again 😞 (Could be related to a stale .tailwind_ppx_cache file though, i will try again next week 👍)

[gaku-sei]

I got a really interesting reply from @bobzhang https://github.com/rescript-lang/rescript-compiler/issues/4469#issuecomment-671728853

[dylanirlbeck]

Can you all try installing v0.8.0 and see if this fixes things? If not, I'll re-open this issue.

[gaku-sei]

Thank you @tatchi and @dylanirlbeck !

Just so you know, the bsb -make-world -- -j 4 "hack" actually helped, so I could restore all the tailwind ppx calls, and the ci passes 90% of the time!

I'm eager to test the v0.8.0 without the above hack, I will let you know as soon as possible 👍

[gaku-sei]

Humm... Not sure exactly why, but it seems 0.8.0 is only available on https://www.npmjs.com/package/tailwind-ppx, not https://www.npmjs.com/package/@dylanirlbeck/tailwind-ppx. Also, the purge css part is absent from the release in https://www.npmjs.com/package/tailwind-ppx. Is it normal?

Otherwise, it seems to work perfectly now, thank you again 🎊

[dylanirlbeck]

Just to confirm, does it work perfectly with v0.8.0 or the bsb -make-world hack? I'm not sure why the package got published under a different name, that's definitely an error.

[gaku-sei]

My bad, it works with the new version, no need for the hack anymore.

(It passed the Circle ci build twice in a row, but I will try again tomorrow just in case 😕 ).

Edit: I ran the ci build several times, and it seems the last release works like a charm, no need for the "hack" anymore!

[dylanirlbeck]

@gaku-sei Alright so you should be able to try v0.8.0 for the NPM package @dylanirlbeck/tailwind-ppx. Let me know if there are any issues!

[gaku-sei]

Thanks @dylanirlbeck Unfortunately it seems the version 0.8.0 comes without the js part (for purge css), is it by design (to split the ppx and the js part?)

This is what gets installed:

When it used to be (notice the js folder):

And it raises this error now:

[dylanirlbeck]

Hmm yeah that's weird, I'll look into it and open a separate issue. For now I'd just manually copy the file into your project if needed. Sorry for all the hassle on this.

[tsnobip]

I wonder if there's not an issue with postinstall, I think tailwind-ppx doesn't get copy from bin to the root folder when I tried v0.0.8

[dylanirlbeck]

Yeah we recently overhauled the CI process, including the postinstall script -- for now I'd just use an earlier version and I'll look into this asap.

[dylanirlbeck]

This should be fixed in v0.8.1! Please file a separate issue if you still encounter problems. Thanks everyone 😄

[gaku-sei]

Works like a charm! Thank you @dylanirlbeck 😄