More intelligent scanning for REPLACE_ENV*

[ChipWolf]

Describe the problem

Currently, the find command for environment variable substitution is very resource intensive as it's checking all files in the volume for each variable to be replaced. This could be improved by first finding any files with the prefix, then using that list as locations to run the replacements.

Container definition

  containers:
    - name: minecraft-forge-charmeleon-prod
      image: 'itzg/minecraft-server:2021.16.0-java8-openj9'
      ports:
        - name: minecraft
          containerPort: 25565
          protocol: TCP
        - name: rcon
          containerPort: 25575
          protocol: TCP
        - name: prometheus
          containerPort: 9225
          protocol: TCP
        - name: voice
          containerPort: 24454
          protocol: UDP
      env:
        - name: EULA
          value: 'TRUE'
        - name: TYPE
          value: FORGE
        - name: FORGEVERSION
          value: 14.23.5.2854
        - name: VERSION
          value: 1.12.2
        - name: DIFFICULTY
          value: '0'
        - name: WHITELIST
        - name: OPS
        - name: ICON
        - name: MAX_PLAYERS
          value: '120'
        - name: MAX_WORLD_SIZE
          value: '2.9999984e+07'
        - name: ALLOW_NETHER
          value: 'true'
        - name: ANNOUNCE_PLAYER_ACHIEVEMENTS
          value: 'true'
        - name: ENABLE_COMMAND_BLOCK
          value: 'true'
        - name: FORCE_GAMEMODE
          value: 'false'
        - name: FORCE_REDOWNLOAD
          value: 'TRUE'
        - name: GENERATE_STRUCTURES
          value: 'true'
        - name: HARDCORE
          value: 'false'
        - name: MAX_BUILD_HEIGHT
          value: '256'
        - name: MAX_TICK_TIME
          value: '-1'
        - name: SPAWN_ANIMALS
          value: 'true'
        - name: SPAWN_MONSTERS
          value: 'true'
        - name: SPAWN_NPCS
          value: 'true'
        - name: VIEW_DISTANCE
          value: '4'
        - name: SEED
        - name: MODE
          value: '0'
        - name: MOTD
          value: forge-charmeleon
        - name: PVP
          value: 'false'
        - name: LEVEL_TYPE
          value: DEFAULT
        - name: GENERATOR_SETTINGS
        - name: LEVEL
          value: world
        - name: ONLINE_MODE
          value: 'false'
        - name: MEMORY
          value: 14G
        - name: JVM_OPTS
          value: >-
            -Xmns3G -Xmnx7G -Xgcpolicy:balanced -Xdisableexplicitgc
            -Dfml.readTimeout=300
        - name: JVM_XX_OPTS
        - name: ENABLE_RCON
          value: 'true'
        - name: RCON_PASSWORD
          valueFrom:
            secretKeyRef:
              name: minecraft-forge-charmeleon-prod
              key: rcon-password
        - name: ALLOW_FLIGHT
          value: 'true'
        - name: CFG_ECONOMY_DB
          value: economylite
        - name: CFG_GTS_DB
          value: globaltrade
        - name: CFG_MASK_APIKEY
          value: # REDACTED
        - name: CFG_NUVOTE_TOKEN
          value: # REDACTED
        - name: CFG_PIXELMONOPTIMIZER_PASSWORD
          value: # REDACTED
        - name: CFG_TEBEX_KEY
          value: # REDACTED
        - name: CFG_VC_HOST
          value: # REDACTED
        - name: GENERIC_PACK
          value: # REDACTED
        - name: NETWORK_COMPRESSION_THRESHOLD
          value: '-1'
        - name: OVERRIDE_SERVER_PROPERTIES
          value: 'TRUE'
        - name: REPLACE_ENV_VARIABLES
          value: 'TRUE'
        - name: REPLACE_ENV_VARIABLES_EXCLUDE_PATHS
          value: /data/pixelmon /data/config/aquadaycare/decorations
        - name: SPAWN_PROTECTION
          value: '0'

Container logs

Notice the gap between the timestamps:

2021-11-15T20:27:03Z [init] Replacing env variables in configs that match the prefix CFG_ ...
2021-11-15T20:27:03Z [init] Replacing CFG_ECONOMY_DB with economylite ...
2021-11-15T20:33:01Z [init] Replacing CFG_GTS_DB with globaltrade ...

[itzg]

Please upgrade to https://github.com/itzg/docker-minecraft-server/releases/tag/2021.21.0 since I completely overhauled that whole process.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Please add a comment describing the reason to keep this issue open.

[ChipWolf]

@itzg still an issue, especially with folders containing thousands of files A suitable fix would be to whitelist files to replace env variables in rather than the current blacklist, we know which files we're interested in replacements occurring, not necessarily needing to check all files arbitrarily. Thanks

[itzg]

@ChipWolf did you upgrade the image? The whole thing is now implemented in Java and selectively processes only the appropriate files.

Please write up a new enhancement request for file whitelisting.