search

Alluxio / alluxio

Alluxio, data orchestration for analytics and machine learning in the cloud
https://www.alluxio.io
Apache License 2.0
6.84k stars 2.94k forks source link

getConf ignores ALLUXIO_*_JAVA_OPTS #13070

Open jiacheliu3 opened 3 years ago

jiacheliu3 commented 3 years ago

Alluxio Version: 2.5

Describe the bug bin/alluxio getConf without --master will load whatever is defined in alluxio-site.properties and ALLUXIO_JAVA_OPTS.

In K8s environment, we typically pass properties to pods with configmap, which is using environment variables like ALLUXIO_WORKER_JAVA_OPTS. In other words, in K8s deployment, we do NOT use alluxio-site.properties file to pass properties (the file has nothing but default values) and define java opts like ALLUXIO_JAVA_OPTS and ALLUXIO_WORKER_JAVA_OPTS to pass java properties to the relevant process.

In this scenario, bin/alluxio getConf will not be able to show all the properties passed to a specific process, like master or worker. The getConf command is a client process, so it will only show alluxio-site.properties + ALLUXIO_JAVA_OPTS + ALLUXIO_USER_JAVA_OPTS + (something else passed directly to the command like -D).

This makes it hard to see what are the properties applied to a certain process.

Expected behavior getConf should be able to show properties applied to a certain process.

Urgency Describe the impact and urgency of the bug.

Additional context Add any other context about the problem here.

Edit: I originally though ALLUXIO_JAVA_OPTS is not loaded. But in fact it is. This should be a feature instead of a bug.

jiacheliu3 commented 3 years ago

One easy workaround is actually:

  1. Find out the process with jps
  2. jinfo <pid> to see all the java properties for this process
apc999 commented 3 years ago

@jiacheliu3 are you asking for a switch on getConf to turn on/off loading ALLUXIO_JAVA_OPTS ?

jiacheliu3 commented 3 years ago

@jiacheliu3 are you asking for a switch on getConf to turn on/off loading ALLUXIO_JAVA_OPTS ?

No I mean getConf will show you alluxio-properties.site + ALLUXIO_JAVA_OPTS + ALLUXIO_USER_JAVA_OPTS, but when you are running this command the intention may be to check the properties loaded into the worker/job worker. We currently don't have a good way (jinfo is, maybe) to see that.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in two weeks if no further activity occurs. Thank you for your contributions.