search

pushkin-consortium / pushkin-cli

A CLI designed to facilitate the setup process of a Pushkin stack
MIT License
0 stars 4 forks source link

Running two instances of `yarn` causing issues? #108

Open jodeleeuw opened 1 year ago

jodeleeuw commented 1 year ago

On a clean install of pushkin, attempting to run pushkin install site I get this kind of error:

image

I think this is fixable by just running the command again, but I suspect the root cause of this is that pushkin-cli is trying to run multiple parallel instances of yarn when it builds the api and front-end as async events.

https://github.com/pushkin-consortium/pushkin-cli/blob/d0c3750724fb6c9153c1a3b06a22f14e5319e3e2/src/commands/sites/index.js#L159-L160

Before I make a PR on this, I just wanted to see if I'm missing anything. Clearly there was some deliberate choice to make all of the fs operations async in this part of the code, so I'm wondering if there's a reason to not await the completion of each of those before moving on to the next.

jkhartshorne commented 1 year ago

@jessestorbeck this is definitely related to https://github.com/pushkin-consortium/pushkin-cli/issues/110, it's maybe a different problem.

So the --mutex flag is supposed to prevent race conditions in yarn. But I used it on yarn install but not yarn build. Also, it looks like that in at least some cases the -mutex flag must come immediately after yarn. Currently testing.

What I can't tell in @jodeleeuw is whether the problem is that --mutex wasn't working or that it was. The fact that the error says it's waiting for another instance to finish suggests to me that perhaps once instance was waiting for the freeze on yarn to be released, but it never was.

See also https://github.com/pushkin-consortium/pushkin-exptemplates-basic/pull/35

jkhartshorne commented 1 year ago

@jodeleeuw -- addressing another thing you raised above, the CLI definitely needs to be more consistent in terms of what is done synchronously and what is asynchronous. I've tried to use async wherever possible in order to speed things up. But there are definitely things that could be async and aren't. And there may be things that shouldn't be async. Usually I use async fs when we won't read the updated file any time soon.

One complication is that Node isn't very good about knowing when terminal processes have finished. I remember I spent a lot of time on this two years ago and hacked things to the point where they mostly don't crash as a result, but no doubt it's less than ideal. And since it's been a couple years since I did that work, it's quite possible there are options now that weren't available then.

We do accept pull requests :)

jkhartshorne commented 1 year ago

@jessestorbeck @ayang21 have either of you noticed whether this issue has been run into lately?

ayang21 commented 12 months ago

Pushkin install site has been working just fine for me. No errors have appeared.

jessestorbeck commented 12 months ago

@jkhartshorne -- I missed this earlier, but I also haven't seen this problem in a while.

ellissc commented 11 months ago
pushkin_error pushkin_error2

@jkhartshorne I've been running into a couple errors lately, though I'm not sure if they're due to Docker or Yarn. Do you think they may be related?

Pushkin version 3.4.0 Recommended site and experiment versions when installing. Docker version 4.24.0

AryaanMishra commented 11 months ago

@jkhartshorne we installed the basic version of the site template and experiment template and we have been running into issue on docker. We tried prep twice, using stop, kill and armageddon and also docker system prune, but it doesn't seem to help.

Pushkin version 3.4.0 Docker version 4.24.0 Screenshot 2023-10-10 at 2 42 02 PM

ellissc commented 11 months ago

docker system prune worked (even though I had been running stop kill armageddon), but the localhost then gives a network error, similar to the stochastic issue (https://github.com/pushkin-consortium/pushkin/issues/225#issuecomment-1651817821).

network error

Update: this network error is stochastic as well. The network error went away after running stop kill armageddon and prep start --nocache a few times. Leaving the error message in case other people run into the same issue.

jessestorbeck commented 11 months ago

@ellissc @AryaanMishra -- Can we do a bit of housekeeping here first? This issue is about an error during pushkin install site. @ellissc, I can't tell if your original screenshot is from prep or start, but I can see that @AryaanMishra's is from prep. If these are errors you haven't seen before, they should probably start as discussions. We can decide from there -- after some attempt to diagnose the problem -- whether to elevate it to an issue. The last general thing is that v4.24.0 is your version of Docker Desktop. The more helpful version number is for Docker Engine. You can get this version number with the command docker --version.

@ellissc, it looks like the top error for you is this ENOSPC thing. I can't say I really know what's causing it, but taking it at face value, it kind of makes sense that clearing out Docker would fix the problem of not having enough space.

@AryaanMishra, looking back here, I have actually seen this issue with the --load flag before, and upgrading Docker Engine (and restarting my machine) apparently fixed it. Can you find out first what version of Docker Engine you're using? I'm on v24.0.5. If you need to upgrade, it may be easiest to just re-install Docker Desktop and that should update Docker Engine too.

ellissc commented 11 months ago

@jessestorbeck @AryaanMishra Thank you! Yeah, it looks like Docker Engine was causing the issue. For some reason, our lab computer was running Docker Engine v20, even though we had the newest version of Docker Desktop. I was able to get Docker Engine v24.0.6, and that fixed the issue. Thank you for your help!