JuliaLang / julia

The Julia Programming Language
https://julialang.org/
MIT License
45.87k stars 5.49k forks source link

mmap failure with address space quotas #10390

Closed JonathanAnderson closed 8 years ago

JonathanAnderson commented 9 years ago

I'm having a problem where when I build from 3c7136e when I run julia, I get the error could not allocate pools If I run as a different user, Julia runs successfully.

I think there might be something specific to my user on this box, but I am happy to help identify what is happening here.

I think this is related to https://github.com/JuliaLang/julia/pull/8699

also, from the julia-users group: https://groups.google.com/forum/#!topic/julia-users/FSIC1E6aaXk

floswald commented 8 years ago

i was using binaries. building is a nightmare on that system as well, I run into diskspace quota exceeded on the login node all the time, and i can't get the build to work on a compute node either. If you can trigger an unofficial 0.4.5 build that would save my week. thanks.

tkelman commented 8 years ago

might take a while to build, but check back at https://build.julialang.org/builders/package_tarball64/builds/435 and when it's done it should be available at https://julianightlies.s3.amazonaws.com/bin/linux/x64/0.4/julia-0.4.6-c7cd8171df-linux64.tar.gz (assuming you want 64 bit linux, and dropping by a factor of 8 will get you below your ulimit)

floswald commented 8 years ago

Awesome! Thanks.

On Wednesday, 8 June 2016, Tony Kelman notifications@github.com wrote:

might take a while to build, but check back at https://build.julialang.org/builders/package_tarball64/builds/435 and when it's done it should be available at julianightlies.s3.amazonaws.com/bin/linux/x64/0.4/julia-0.4.6-c7cd8171df-linux64.tar.gz (assuming you want 64 bit linux)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JuliaLang/julia/issues/10390#issuecomment-224552328, or mute the thread https://github.com/notifications/unsubscribe/AA-WdofbXDWPFFjmNbso-fAX6Dq4Dk_Bks5qJpt5gaJpZM4Doy6D .

floswald commented 8 years ago

@tkelman thanks so much works out of the box like a charm! so much for "broken software". outstanding support as usual. 👍 👍 👍

mauro3 commented 8 years ago

In case someone else stumbles over this: I was under the impression that this issue was resolve but it still surfaced for me with 0.5-rc4 binaries and source build for rc4, see #18477.

The error now looks a bit different for me with either just hanging at the tests when running make testall or when doing addprocs with a suitably high number I get Master process (id 1) could not connect within 60.0 seconds. The fix is as before but now in file src/gc-pages.c.

StefanKarpinski commented 8 years ago

Reopened to be fixed in 0.5.x.

yuyichao commented 8 years ago

As mentioned in the related issue, this is really https://github.com/JuliaLang/julia/issues/17987. It does not fail because we are asking for a huge fixed size anymore and the remaining is better handled by allowing users with special memory constraint to specify that directly.

floswald commented 8 years ago

Sorry to bother with this but I am still looking for a solution to this problem. I am working on a cluster where I have to request the max amount of virtual and physical memory that I will be using, and I have to request very large amounts in order for my job to run at all. This puts me on a significantly longer queue, because I basically need an entire compute node all for myself. julia v0.5-rc3.

my job has the following memory requirements when run on a single compute node on that same cluster.

Any advice for how to deal with this greatly appreciated.