ioos / erddap-gold-standard

Contains the 'gold standard' ERDDAP configuration, with datasets compliant with IOOS Metadata Profile 1.2
https://standards.sensors.ioos.us/erddap/index.html
8 stars 16 forks source link

ERDDAP Memory Question #37

Open Dylan-Pugh opened 1 year ago

Dylan-Pugh commented 1 year ago

Hi everyone - quick question about ERDDAP memory allocation. The default values for the ERDDAP_MIN_MEMORY & ERDDAP_MAX_MEMORY environment variables are currently different, but the ERDDAP documentation states that they should be the same:

The -Xmx and -Xms memory settings are important because ERDDAP works better with more memory. Always set -Xms to the same value as -Xmx.

From docker-compose.yaml:

   environment:
      ERDDAP_MIN_MEMORY: 1G
      ERDDAP_MAX_MEMORY: 2G

My understanding is that (if set) these env variables are mapped to -Xmx and -Xms in setenv.sh:

NORMAL="-server -Xms${ERDDAP_MIN_MEMORY:-${MEMORY}} -Xmx${ERDDAP_MAX_MEMORY:-${MEMORY}}"

I also noticed this line from the documentation:

With 32 bit Java, even with abundant physical memory, Tomcat and Java won't run if you try to set -Xmx much above 1500M (1200M on some computers).

The current version of setenv.sh isn't passing the -d64 flag to enable 64 bit Java, so it seems like 2G (and the fallback value of 4G) would be too high for the max memory.

Are these potential problems? Or is this a case of the ERDDAP documentation being out of date?

Thanks!

kwilcox commented 1 year ago

The -Xmx and -Xms memory settings are important because ERDDAP works better with more memory. Always set -Xms to the same value as -Xmx.

This is a good general suggestion without getting into the whys and why nots and diving deep in java/tomcat tuning. Overall as long as -Xmx is high enough then ERDDAP/Java will eat that memory up when it needs to and overflow when it runs out. You should tune to your own experiences and when in doubt just set -Xms to whatever you are setting -Xmx to.

The current version of setenv.sh isn't passing the -d64 flag to enable 64 bit Java, so it seems like 2G (and the fallback value of 4G) would be too high for the max memory.

Recent JVMs detects this so it isn't required to explicitly pass in, although it can't hurt!