search

ddavisqa / google-refine

Automatically exported from code.google.com/p/google-refine
0 stars 0 forks source link

Java HotSpot(TM) 64-Bit Server VM warning: Failed to reserve shared memory (errno = 28) #333

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
Hi,

While trying to load and work with "big" csv files (2Go) I have the java 
exception : 
Java HotSpot(TM) 64-Bit Server VM warning: Failed to reserve shared memory 
(errno = 28) 
... or a classic java heap (server's choice ...).

Here is my configuration : 
Google refine version : Version 2.0 [r1836]
Os version : Ubuntu Lucid
Jvm version : Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03, mixed mode)
Hardware : Intel, 64 bits, 16 go ram, dedicated machine.

Ini configuration and other java opts I tried to play with : 
REFINES_MEMORY=12000M
JAVA_OPTIONS="-XX:+UseParallelGC -XX:+UseLargePages -XX:-UseGCOverheadLimit"
In sysctl : 
kernel.shmmax = 16106127360
vm.nr_hugepages = 3072
Browser : ie8

I have a comfortable machine to run Google Refine, can you please give me some 
advice on how to tune the jvm and give Refine the ability to load these big 
files. I'm afraid I'm currently underskilled for this kind of tuning despite a 
lot of trial and errors.

Many thanks

VinZ

Original issue reported on code.google.com by vincent....@gmail.com on 14 Feb 2011 at 5:57

GoogleCodeExporter commented 8 years ago 
Here is my output (short) when starting Refine.

Starting Google Refine at 'http://0.0.0.0:3333/'

Java HotSpot(TM) 64-Bit Server VM warning: Failed to reserve shared memory 
(errno = 28).
18:07:18.993 [            refine_server] Starting Server bound to 
'0.0.0.0:3333' (0ms)
18:07:18.994 [            refine_server] Max memory size: 15000M (1ms)
18:07:19.005 [            refine_server] Initializing context: '/' from 
'/mnt/grefine/google-refine-2.0/webapp' (11ms)

Original comment by vincent....@gmail.com on 14 Feb 2011 at 6:09

GoogleCodeExporter commented 8 years ago 
The issue database is for bug reports and enhancement requests.  Please use the 
 mailing list for questions about tuning advice.  (Although, honestly, you may 
get quicker/better advice in a Java forum with expertise in running JVMs with 
large heap sizes)

Your two postings talk about two different heap sizes 15g and 12g.  Which is 
it?  15g definitely seems too big for a 16g machine.

Original comment by tfmorris on 14 Feb 2011 at 6:26

GoogleCodeExporter commented 8 years ago 
ok, thanks for your suggestion.
Ill ask in a Java forum, ur right.

Original comment by vincent....@gmail.com on 14 Feb 2011 at 9:40