dagster-daemon cannot access grpc in multi-container deployment

[brandybobandyistaken]

Dagster version

dagster, version 1.5.5

What's the issue?

I've followed the multi-container deployment found here:

• https://docs.dagster.io/deployment/guides/docker
• https://github.com/dagster-io/dagster/tree/1.5.5/examples/deploy_docker

And after docker compose up, the issue I've run into coming from the dagster-daemon container is:

• "Error loading repository location user_code:dagster._core.errors.DagsterUserCodeUnreachableError: Could not reach user code server. gRPC Error code: UNAVAILABLE"

Per the deployment guide, the same image/container and thus dagster.yaml/workspace.yaml are used to run the webserver and the daemon, aside from entrypoint.

QUESTIONS:

• Does the daemon actually need access to the user_code container via gRPC? If so, what for?
• I'm unfamilair with gRPC, is it expected both the webserver container and the daemon container should be able to load from the single gRPC server running on the user_code container? The webserver container loads from it fine, but the daemon cannot.

What did you expect to happen?

No issue loading user code from user_code container into dagster-daemon container via single gRPC server run form user_code container. Webserver has no problem loading it, but daemon does.

How to reproduce?

Follow the deployment from dagster provided repo:

• https://github.com/dagster-io/dagster/tree/1.5.5/examples/deploy_docker

Deployment type

Docker Compose

Deployment details

The only difference from dagster example deployment is customizing the user_code container to contain actual user code.

Additional information

Thank you so much for your time and help in advance!

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

[benoitsdm]

Same issue here, any news or workaround ?