[DEV] Allow developers to have a local/modify opensearch.yml or ENV

[dblock]

Is your feature request related to a problem? Please describe.

I was trying to figure out how to expose 0.0.0.0 from my EC2 test instance on which I do development. What worked was to run ./gradlew -Dtests.opensearch.http.host=0.0.0.0 run.

Describe the solution you'd like

I'd like to be able to do any (combination) of these:

a) modify distribution/src/config/opensearch.yml or another yml and have it get picked up by ./gradlew run b) use an equivalent of dotenv to permanently override settings, e.g. OPENSEARCH_HTTP_HOST=0.0.0.0 ./gradlew run which would allow one to create .env and run dotenv ./gradlew run c) some other set it and forget it mechanism

Describe alternatives you've considered

Run ./gradlew assemble to build the ZIP for the distribution. Extract the zip and edit config/opensearch.yml. But this is more time consuming.

Additional context

[kohinoor98]

Hey @sejli

I would like to give this issue a shot, please could you assign this to me?

[kohinoor98]

Hello @dblock,

I'm seeking further context to effectively address the issue at hand.

File Reference: I'm currently working within the file:

buildSrc/src/main/java/org/opensearch/gradle/testclusters/RunTask.java

Within this file, I encountered the map additionalSettings which appears to capture all properties prefixed with "tests.opensearch.*" from System.getProperties(). Could you clarify where System.getProperties() is referencing?

YML Configuration:

Do I need to ensure compatibility for the key-value pair: http.host: 0.0.0.0 within the YML? If I create something like

ymlSettings.forEach(node::setting); 

within the RunTask.java file, will it automatically bind the aforementioned key-value pair?

Clarification: For point (b) above, is the expected command to be:

OPENSEARCH_HTTP_HOST=0.0.0.0 ./gradlew run

Additional context on this would be greatly appreciated.

Thank you for your guidance.

Warm regards, Kohinoor Chatterjee

[dblock]

Could you clarify where System.getProperties() is referencing? will it automatically bind the aforementioned key-value pair?

I don't know the answer to these two. Try it out!

For point (b) above, is the expected command to be ...

Yes, that's the same as mentioned in b), or am I not seeing a difference?

Stepping back, the goal is that I can override and save settings somewhere and get them picked up by ./gradlew run, other than having to specify -Dtests...., which is difficult (I have to copy-paste very long command lines into text files and save those for next time I need them).

[kohinoor98]

Thanks for the response @dblock

My approach to implement the above requirement:

1. Ingest flags from .env file which would contain flags like:

   OPENSEARCH_HTTP_HOST=0.0.0.0
   SOME_OTHER_FLAG=VALUE

2. Create a way to pass the flags to gradle

I would appreciate your insights on this approach. Additionally, if you have any alternative solutions or recommendations, I would be keen to consider them.

Thank you for your guidance.

[dblock]

For 1 if you supported getting those from the environment, then you could use an existing tool like dotenv which is the standard in many languages. So no need to explicitly read a file.

For 2 I don't have any better ideas than (1)!