olljanat
commented
5 years ago @stevenmiller first of I can see that this is your first issue on GitHub so let me say welcome :)
Then some side comments (which you don't need to comment but maybe it is good idea to check them out):
- Some best practices with Windows containers: https://github.com/MicrosoftDocs/Virtualization-Documentation-Private/issues/1531
- You can set default logging driver to all Docker hosts so you don't need specify it separately for all containers.
- Environment variable ASPNETCORE_ENVIRONMENT gives me hint that your applications are ASP.NET Core based so unless your application really needs something Windows specific features I highly recommend to use Linux version of these images. It sounds bit weird idea but based on my two years experience with Linux and Windows containers I still highly recommend it.
- Kestrel web server which is used by .NET Core have been supported to publish directly to internet starting from version 2.1 but I would still recommend to consider configuration where you run example NGINX as reverse proxy front of it and make it running on Linux nodes even if your applications are running on Windows nodes.
- Consider to use Portainer instead of Swarmpit. It example have support for native Windows containers which is missing from Swarmpit and especially their agent deployment is very interesting as you can manage Linux and Windows containers from one UI and even connect multiple Swarms to it if needed.
- If you need Windows containers then consider to get Microsoft Premier Support contract (if you don't have it yet).
- Microsoft recommends to use host mode in production
- You can find me with same nick from Docker and Portainer Slacks if you want hear some real world examples how this stuff works on production.
Then what comes to this issue, IMO your stack file looks to be just like documentation says that how it should work but in real life it looks that Windows implementation of overlay network / components used behind it (example: hcsshim) have some undocumented weaknesses.
stevenmiller
docker-robott
Howdy!
I have a two node hybrid swarm, with one Ubuntu Linux 18.04 node acting as manager and one Windows 2019 node participating as a worker. We have a sample environment we are testing which are entirely Windows containers. We are able to reach the published port for these services from the Windows node (ex. 10.20.1.121:8888) but we are unable to from the Linux node (ex. 10.20.1.122:8888). Our understanding of the ingress mesh networking is that this should work fairly seamlessly as it will route traffic within the swarm to the correct node.
This issue is also occurring with the swarmpit UI stack we have deployed to the swarm. The UI (10.20.1.122:888) is not able to be reached from the Windows node IP (10.20.1.121:888).
Expected behavior
Services running on one node should be accessible from published port on another node
Actual behavior
Published Windows container services are not accessible from linux node and vice versa
Information
Published services and ports:
Both the environment we have pulled from a private repo and the swarmpit stack exhibit this behavior. I am unable to reach the swarmpit UI from port 888 on the Windows node IP.
Steps to reproduce the behavior
Here is the stack with our private repo removed:
teststack.zip
Let me know if I can provide any other info.