Open freemansoft opened 2 months ago
The host names are redis
and milvus
but it can be more complex because the containerized aspect introduces additional scopes. The short version is that it depends on where you are referencing them from. Generally you can think of this as 3 scopes:
So... given all of that, the reference to a given hostname or "localhost" could actually mean: The actual host, The AI Workbench Container, one of the other containers, and a localhost port could actually be bound between the host-container or another container. The short version is that there are a lot of ways this can be accomplished, so each setup could have it's own unique approach.
Since there's not a single common way of doing this, and because there are so many potential configurations including exposing and mapping a port outside of a container (either on the host, a container, a container network, or other) you need to figure out where you are and look at what you are referencing to understand what you are actually looking at.
If you want to go down the rabbit hole, you can find more info in the Docker Networking docs, but that doesn't even get into more abstract configurations like Docker-in-Docker or Docker-out-of-Docker (which is what NIM Anywhere uses); these and other approaches add additional potential layers of complexity.
That is an awesome explanation. It still confuses me about the configuration in this project
How are the different files intended to be consumed with their different values? It isn't obvious what processes use which file under which conditions.
variables.env
milvus
defaults to milvus:
. Doesn't seem to be referenced.redis
defaults to redis:
. Doesn't seem to be referenced.frontend/configuration.py
,
code/config.yaml
llm-0:
defaults to container nameembedding
defaults to the container namereranking
defaults to the container name localhost
chain_server/config.yaml
milvus
defaults to localhost
redis
defaults to localhost
llm
defaults to NVIDIA endpoints or the container name if local.re-ranker
defaults to NVIDIA endpoints or the container name if local.embedding
defaults to NVIDIA endpoints or the container name if local.Healthchecks point at localhost
. This is because they test from inside the target container.
The NIM-ANywhere project is a multi-container project. The AI Workbench container sees the Milvus and Redis containers as their container names because they are part of the same docker instance. There are 6 services in this project that all run in their own containers or are deployed remotely. Why are they not in variables.env?
milvus
.
project-nim-anywhere
to milvus:6379
project-nim-anywhere
to localhost:6389
redis
.
project-nim-anywhere
to redis:19530
project-nim-anywhere
to localhost:19530
Reading a little between the lines I think you're ultimately wanting to know "Why not just add the hostnames in one place like variables.env?"
The files offer a lot of flexibility, but do not seem to be used in this setup. What's in place is misleading because inside any container, localhost
refers only to services within that container (not other containers) so I don't think localhost is actually used anywhere. I think that localhost
did at one time make sense for a typical chain-server deployed on a traditional server, but any references to localhost are likely just unused remnants of default configuration for a non-containerized deployment.
As I mention in my comment below, there is an option to use an environment variable as a global override, which they do for redis & milvus in variables.env
- and it seems like that they're likely planning for this as a pattern to manage more complex systems.
The team probably didn't feel compelled to bother cleaning up the older bits that referenced localhost. As for why any self respecting team would choose to move forward in such a state of disarray, I can't help but think of a phrase that an old developer friend used to say, and that I fully agree with: "Premature optimization is the root of all evil." It might be nice for consistency, but isn't necessary for functionality (and whether or not it even should be cleaned up could lead to a spirited discussion, as we now know that any references are unused).
tl;dr - Don't look a gift horse in the mouth. References to localhost aren't used. All containers are networked together so the container names are available as hostnames and relevant references are set in variables.env
.
PS - this may be all you ever needed 😎
Looking back at your original comment... the example config file in the README , and configuration.py, both reference localhost
The available override (_APP_FIELD__SUBFIELD) is defined in variables.env as redis
(and similar for milvus
).
There is a section in the README about Config from env vars: "Values specified as environment variables will take precedence over all values from files."
What are the DNS names for the redis and milvus server? They are referred to as
localhost
in some places and asredis
andmivlus
in others. They are pingable fromtraefic
asredis
andmilvus
Redis
Milvus