search

racket / pkg-build

Other
5 stars 9 forks source link

pkg-build server's memory limit #15

Closed mflatt closed 11 months ago

mflatt commented 1 year ago

The rhombus-prototype build failed a few times by being killed. I think it’s running up against a 1GB memory limit.

Of course, 1GB is a lot… Part of the issue is that pkg-build uses -j 2, so a package can get past 1GB by having two pieces that are each in the neighborhood of 500MB. That amount of memory use for one document is not unprecedented, as illustrated by peaks in the blue and green regions of https://build-plot-cs.racket-lang.org/. That is, the main-distribution build stays under 1GB only with -j 1.

I will continue efforts to make rhombus-prototype build in less memory, but even if I succeed, the issue is likely to show up again with some package. Would it be possible and make sense to increase the memory limit, either to 1.5GB ot 2GB? Or could pkg-build provide a way for a package to request -j 1?

samth commented 1 year ago

I think we should just increase the memory limit, as it has been a problem for other packages as well.

jryans commented 1 year ago

It looks like the pkg-builder machine only has 2 GB of physical memory... I worry that performance may fall off a cliff (as virtual memory swapping increases) if we raised the builder's limit all the way to 2 GB. We could try 1.5 GB for now.

Perhaps it may also be good to move to different machine with slightly more memory, like say 4 GB? The current machine appears to be a t2.small EC2 instance. Perhaps we could move to the next level up (t2.medium)?

jryans commented 1 year ago

Not sure if on-demand pricing is being used or some other payment plan, but at least for on-demand, it looks like, the newer generations are cheaper than the older ones, so I suppose we'd really want t3a.medium for an x86 instance. If we can handle an Arm instance, then t4g.medium is even cheaper.

samth commented 11 months ago

I just tried increasing to 1.5GB. Let's see if that addresses the problem for the moment; otherwise I'll do something with jobs.

samth commented 11 months ago

All of the problematic packages that I know of (shplait, rhombus-prototype, graphite) are now building successfully. If this becomes a problem again we can revisit.