Why do variables.env and configuration.py us different DNS names for the same services?

[freemansoft]

What are the DNS names for the redis and milvus server? They are referred to as localhost in some places and as redis and mivlus in others. They are pingable from traefic as redis and milvus

Redis

1. README.md redis://localhost:6379/0
2. variables.env redis://redis:6379/0
3. chain_server/configuration.py redis://localhost:6379/0
4. container name redis

Milvus

1. README.md http://localhost:19530
2. variables.env http://milvus:19530
3. chain_server/configuration.py http://localhost:19530
4. container name milvus

[brainwavecoder9]

The host names are redis and milvus but it can be more complex because the containerized aspect introduces additional scopes. The short version is that it depends on where you are referencing them from. Generally you can think of this as 3 scopes:

1. The host where you are running AI Workbench. In many cases you won't have access to this, or at least, it's less relevant because your AI Workbench environment will be entirely inside of it's own container.
2. The AI Workbench container. This is what you see being built, and where everything happens when you run your environment. In most cases, this is is the only reference you will even be aware of.
3. Other containers / networks / bindings / etc. In some cases, such as this NIM Anywhere project, an approach is used where multiple containers are at play. This is atypical and less common, but is possible to encounter. There are a number of ways this can be accomplished so specifics will depend on each unique deployment, but in this case redis/milivus are each their own containers (and you are able to access them from within the AI Workbench container). Note that per documentation the current approach used for NIM Anywhere is expected to change. I'm not tapped into any details, but I would assume they will introduce some type of orchestration approach which will allow your AI Workbench environment to spin up multiple containers instead of just the one we have today.

So... given all of that, the reference to a given hostname or "localhost" could actually mean: The actual host, The AI Workbench Container, one of the other containers, and a localhost port could actually be bound between the host-container or another container. The short version is that there are a lot of ways this can be accomplished, so each setup could have it's own unique approach.

Since there's not a single common way of doing this, and because there are so many potential configurations including exposing and mapping a port outside of a container (either on the host, a container, a container network, or other) you need to figure out where you are and look at what you are referencing to understand what you are actually looking at.

If you want to go down the rabbit hole, you can find more info in the Docker Networking docs, but that doesn't even get into more abstract configurations like Docker-in-Docker or Docker-out-of-Docker (which is what NIM Anywhere uses); these and other approaches add additional potential layers of complexity.

[freemansoft]

That is an awesome explanation. It still confuses me about the configuration in this project

How are the different files intended to be consumed with their different values? It isn't obvious what processes use which file under which conditions.

• variables.env
  • Config for milvus defaults to milvus:. Doesn't seem to be referenced.
  • Config for redis defaults to redis:. Doesn't seem to be referenced.
• frontend/configuration.py,
  • Not part of this
• code/config.yaml
  • Config for llm-0: defaults to container name
  • Config for embedding defaults to the container name
  • Config for reranking defaults to the container name
  • It uses the container name for locally running containers and not localhost
• chain_server/config.yaml
  • Config for milvus defaults to localhost
  • Config for redis defaults to localhost
  • Config for llm defaults to NVIDIA endpoints or the container name if local.
  • Config for re-ranker defaults to NVIDIA endpoints or the container name if local.
  • Cnofig for embedding defaults to NVIDIA endpoints or the container name if local.

Healthchecks point at localhost. This is because they test from inside the target container.

The NIM-ANywhere project is a multi-container project. The AI Workbench container sees the Milvus and Redis containers as their container names because they are part of the same docker instance. There are 6 services in this project that all run in their own containers or are deployed remotely. Why are they not in variables.env?

• milvus. Runs locally. Avaialble in-cluster milvus.
  • 👍 Connect from project-nim-anywhere to milvus:6379
  • 👎 Cannot connect from project-nim-anywhere to localhost:6389
• redis. Runs locally. Available in-cluster under redis.
  • 👍 Connect from project-nim-anywhere to redis:19530
  • 👎 Connect from project-nim-anywhere to localhost:19530

[brainwavecoder9]

Reading a little between the lines I think you're ultimately wanting to know "Why not just add the hostnames in one place like variables.env?"

The files offer a lot of flexibility, but do not seem to be used in this setup. What's in place is misleading because inside any container, localhost refers only to services within that container (not other containers) so I don't think localhost is actually used anywhere. I think that localhost did at one time make sense for a typical chain-server deployed on a traditional server, but any references to localhost are likely just unused remnants of default configuration for a non-containerized deployment.

As I mention in my comment below, there is an option to use an environment variable as a global override, which they do for redis & milvus in variables.env - and it seems like that they're likely planning for this as a pattern to manage more complex systems.

The team probably didn't feel compelled to bother cleaning up the older bits that referenced localhost. As for why any self respecting team would choose to move forward in such a state of disarray, I can't help but think of a phrase that an old developer friend used to say, and that I fully agree with: "Premature optimization is the root of all evil." It might be nice for consistency, but isn't necessary for functionality (and whether or not it even should be cleaned up could lead to a spirited discussion, as we now know that any references are unused).

tl;dr - Don't look a gift horse in the mouth. References to localhost aren't used. All containers are networked together so the container names are available as hostnames and relevant references are set in variables.env.

[brainwavecoder9]

PS - this may be all you ever needed 😎

Looking back at your original comment... the example config file in the README , and configuration.py, both reference localhost The available override (_APP_FIELD__SUBFIELD) is defined in variables.env as redis (and similar for milvus).

There is a section in the README about Config from env vars: "Values specified as environment variables will take precedence over all values from files."