Visual Studio 2022 17.3.5: Docker-Compose needs rebuild to debug new changes

aschwab commented 2 years ago

Whenever I change something in my ASP.NET Microservice, I need to rebuild the Project or everything for the changes to be available for debugging on the container, otherwise It will try to debug outdated source which ends up with failing my breakpoints.

This problem occurs since updating to the latest version of Visual Studio / Docker. I'm using VS 2022 17.3.5 with Docker 4.12.0 and Container Tools 1.17.0.

It seems like like the problem is, that my docker-compose.dcproj (and launchSettings.json) does not rebuild my local projects, this could also be because my changes are in a referenced project.

docker-compose.dcproj

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="15.0" Sdk="Microsoft.Docker.Sdk">
  <PropertyGroup Label="Globals">
    <ProjectVersion>2.1</ProjectVersion>
    <DockerTargetOS>Linux</DockerTargetOS>
    <ProjectGuid>3665ba0b-49be-4e79-9c48-a1657281978a</ProjectGuid>
    <DockerLaunchAction>None</DockerLaunchAction>
    <DockerServiceUrl>{Scheme}://localhost:{ServicePort}/{Scheme}://{ServiceHost}:{ServicePort}</DockerServiceUrl>
    <DockerServiceName>configurationservice.api</DockerServiceName>
  </PropertyGroup>
  <ItemGroup>
    <None Include=".dockerignore" />
    <None Include="docker-compose.override.yml">
      <DependentUpon>docker-compose.yml</DependentUpon>
    </None>
    <None Include="docker-compose.yml" />
  </ItemGroup>
</Project>

launchSettings.json

{
  "profiles": {
    "Docker Compose": {
      "commandName": "DockerCompose",
      "commandVersion": "1.0",
      "serviceActions": {
        "configurationservice.api": "StartDebugging",
        "configurationservice.api-dapr": "StartWithoutDebugging",
        "identityservice.api": "StartDebugging",
        "identityservice.api-dapr": "StartWithoutDebugging",
        "legacyproxyservice.api": "StartDebugging",
        "legacyproxyservice.api-dapr": "StartWithoutDebugging",
        "yarp": "StartDebugging",
        "yarp-dapr": "StartWithoutDebugging",
        "redis": "StartWithoutDebugging",
        "zipkin": "StartWithoutDebugging"
      }
    }
  }
}

I expect my visual studio projects (named like the docker-compose containers) to be rebuilt whenever they're out of date. When I rebuild all manually, breakpoints and debugging work like expected. I guess it actually rebuild the whole solution when building manually causing this to work.

danegsta commented 2 years ago

Do you use depends_on declarations for services in your docker-compose.yml or docker-compose.override.yml? We've identified a bug related to how our extension parses the depends_on config that could prevent us from properly registering service projects as build time dependencies.

aschwab commented 2 years ago

Hi @danegsta , thanks for your reply, yes I do use dependencies, here are my docker-compose Files:

version: '3.4'

services:
  yarp:
    image: ${DOCKER_REGISTRY-}yarp
    build:
      context: .
      dockerfile: Yarp/Yarp/Dockerfile
    depends_on:
    - identityservice.api
    - configurationservice.api
    - legacyproxyservice.api
  yarp-dapr:
    image: "daprio/daprd:1.4.2"
    depends_on:
    - yarp
    network_mode: "service:yarp"

  configurationservice.api:
    image: ${DOCKER_REGISTRY-}configurationserviceapi
    build:
      context: .
      dockerfile: ConfigurationService/ConfigurationService.API/Dockerfile
  configurationservice.api-dapr:
    image: "daprio/daprd:1.4.2"
    depends_on:
    - configurationservice.api
    - redis
    network_mode: "service:configurationservice.api"

  identityservice.api:
    image: ${DOCKER_REGISTRY-}identityserviceapi
    build:
      context: .
      dockerfile: IdentityService/IdentityService.API/Dockerfile
    depends_on:
    - configurationservice.api
  identityservice.api-dapr:
    image: "daprio/daprd:1.4.2"
    depends_on:
    - identityservice.api
    - redis
    network_mode: "service:identityservice.api"

  legacyproxyservice.api:
    image: ${DOCKER_REGISTRY-}legacyproxyserviceapi
    build:
      context: .
      dockerfile: LegacyProxyService/LegacyProxyService.API/Dockerfile
    depends_on:
    - configurationservice.api
  legacyproxyservice.api-dapr:
    image: "daprio/daprd:1.4.2"
    depends_on:
    - legacyproxyservice.api
    network_mode: "service:legacyproxyservice.api"

  redis:
    image: "redis:6.2.6-alpine"

  zipkin:
    image: openzipkin/zipkin-slim

version: '3.4'

services:

  yarp:
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      - ASPNETCORE_URLS=https://+:443;http://+:80
      - DAPR_HTTP_PORT=3500
    ports:
      - "7000:80"
      - "7001:443"
      - "50001:50001"
    volumes:
      - ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
      - ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro
    dns:
      - 8.8.8.8
      - 127.0.0.11

  yarp-dapr:
    command: ["./daprd",
     "-app-id", "apigateway",
     "-app-port", "80",
     "-dapr-http-port", "3500",
     "-components-path", "/dapr/components",
     "-config", "/dapr/dapr-config.yaml",
     "-log-level", "info"]
    volumes:
        - "./dapr-components/docker:/dapr"

  identityservice.api:
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      - ASPNETCORE_URLS=https://+:443;http://+:80
    ports:
      - "7100:80"
      - "7101:443"
      - "50002:50001"
    volumes:
      - ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
      - ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro

  identityservice.api-dapr:
    command: ["./daprd",
     "-app-id", "identityservice",
     "-app-port", "80",
     "-dapr-http-port", "3500",
     "-components-path", "/dapr/components",
     "-config", "/dapr/dapr-config.yaml",
     "-log-level", "info"]
    volumes:
        - "./dapr-components/docker:/dapr"

  configurationservice.api:
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      - ASPNETCORE_URLS=https://+:443;http://+:80
    ports:
      - "7200:80"
      - "7201:443"
      - "50004:50001"
    volumes:
      - ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
      - ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro

  configurationservice.api-dapr:
    command: ["./daprd",
     "-app-id", "configurationservice",
     "-app-port", "80",
     "-dapr-http-port", "3500",
     "-components-path", "/dapr/components",
     "-config", "/dapr/dapr-config.yaml",
     "-log-level", "info"]
    volumes:
        - "./dapr-components/docker:/dapr"

  legacyproxyservice.api:
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      - ASPNETCORE_URLS=https://+:443;http://+:80
    ports:
      - "7300:80"
      - "7301:443"
      - "50005:50001"
    volumes:
      - ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
      - ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro

  legacyproxyservice.api-dapr:
    command: ["./daprd",
     "-app-id", "legacyproxyservice",
     "-app-port", "80",
     "-dapr-http-port", "3500",
     "-components-path", "/dapr/components",
     "-config", "/dapr/dapr-config.yaml",
     "-log-level", "debug"]
    volumes:
        - "./dapr-components/docker:/dapr"

  redis:
    ports:
      - "6379:6379"

  zipkin:
    ports:
      - "9411:9411"

Please don't be irritated by some paths / file / project-names, I removed the customer- / product name.

aschwab commented 2 years ago

I removed all dependencies from the containers, did a clean on the solution and the docker-compose project, docker purge and built the whole solution.

It did not fix the problem and whats interesting is, that I get an error saying that the main dll's for the projects could not be copied which actually means that the projects from the docker-compose Project haven't been built at all when having docker-compose set as startup project.

The changes I make in my project are not directly in the web-project itself but in a referenced library, maybe that's why it's not tracked?!

Also my Containers window throws an exception (An error occured while attempting to refresh the containers.) but shows the Containers as expected:

danegsta commented 2 years ago

The issue with depends_on is a bug that is preventing the extension from parsing the short form the the configuration; we've created a fix that will ship with VS 17.4, but you can workaround the issue in the meantime by switching from the short version of the syntax to the long version (see: https://docs.docker.com/compose/compose-file/#depends_on). You would also have to remove the version: '3.4' declaration from the top of your compose yaml files as the long form is only supported in the current compose spec (which no longer uses a version declaration, but is backwards compatible with the earlier 3.x and 2.x specifications).

For example,

depends_on:
- identityservice.api
- redis

would become

depends_on:
  identityservice.api:
    condition: service_started
  redis:
    condition: service_started

Making that change for every usage of depends_on should allow the extension to properly track and report the service projects as build dependencies.

For the COM error, I'd usually recommend trying to repair your VS install as a first step to see if that fixes the problem; sometimes various component caches get out of sync and a repair can ensure VS is loading all required assemblies at the correct times.

aschwab commented 1 year ago

I've tried to apply your workaround by replacing all of the depends_on statements with the long form, but the problem persists. I still need to manually rebuild to debug / run the newer version of the source after I applied my changes.

Is there any other known issue according to project / container mapping?

Could my project structure somehow affect this?

/
  docker-compose.yaml
  docker-compose.override.yaml
    /Yarp/Yarp/Dockerfile
    /Yarp/Yarp/Yarp.csproj
    /IdentityService/IdentityService.API/Dockerfile
    /IdentityService/IdentityService.API/IdentityService.API.csproj
    /ConfigurationService/ConfigurationService.API/Dockerfile
    /ConfigurationService/ConfigurationService.API/ConfigurationService.API.csproj
    /LegacyProxyService/LegacyProxyService.API/Dockerfile
    /LegacyProxyService/LegacyProxyService.API/LegacyProxyService.API.csproj

Also my VS-Solution is structured like this:

/
  docker-compose (project)
  /V2
    /Yarp/Yarp (project)
    /IdentityService/IdentityService.API (project)
    /IdentityService/IdentityService.Domain (project) -> changes are made here which don't affect the build
    /ConfigurationService/ConfigurationService.API (project)
    /LegacyProxyService/LegacyProxyService.API (project)

aschwab commented 1 year ago

@danegsta I also tried this with VS 17.3.0 (Fixed Installer), but the problem also persists. Are there any news on this? If this helps I can offer to debug this together, my Discord-Handle would be Alsc#4353.

danegsta commented 1 year ago

@aschwab would you be able to share your updated docker-compose.yaml and docker-compose.override.yaml files? I can run them through our parser to see if anything else there is causing issues. Which service projects have a ProjectReference to IdentityService.Domain? If you update the service projects, but not not the IdentityService.Domain project, do the changes get picked up during a normal build/F5?

aschwab commented 1 year ago

@danegsta here are my updated Docker-Compose files:

services:
  yarp:
    image: ${DOCKER_REGISTRY-}yarp
    build:
      context: .
      dockerfile: Yarp/Yarp/Dockerfile
    depends_on:
      identityservice.api:
        condition: service_started
      configurationservice.api:
        condition: service_started
      legacyproxyservice.api:
        condition: service_started
  yarp-dapr:
    image: "daprio/daprd:1.4.2"
    depends_on:
      yarp:
        condition: service_started
    network_mode: "service:yarp"

  configurationservice.api:
    image: ${DOCKER_REGISTRY-}configurationserviceapi
    build:
      context: .
      dockerfile: ConfigurationService/ConfigurationService.API/Dockerfile
  configurationservice.api-dapr:
    image: "daprio/daprd:1.4.2"
    depends_on:
      configurationservice.api:
        condition: service_started
      redis:
        condition: service_started
    network_mode: "service:configurationservice.api"

  identityservice.api:
    image: ${DOCKER_REGISTRY-}identityserviceapi
    build:
      context: .
      dockerfile: IdentityService/IdentityService.API/Dockerfile
    depends_on:
      configurationservice.api:
        condition: service_started
  identityservice.api-dapr:
    image: "daprio/daprd:1.4.2"
    depends_on:
      identityservice.api:
        condition: service_started
      redis:
        condition: service_started
    network_mode: "service:identityservice.api"

  legacyproxyservice.api:
    image: ${DOCKER_REGISTRY-}legacyproxyserviceapi
    build:
      context: .
      dockerfile: LegacyProxyService/LegacyProxyService.API/Dockerfile
    depends_on:
      configurationservice.api:
        condition: service_started
  legacyproxyservice.api-dapr:
    image: "daprio/daprd:1.4.2"
    depends_on:
      legacyproxyservice.api:
        condition: service_started
    network_mode: "service:legacyproxyservice.api"

  redis:
    image: "redis:6.2.6-alpine"

  zipkin:
    image: openzipkin/zipkin-slim

services:

  yarp:
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      - ASPNETCORE_URLS=https://+:443;http://+:80
      - DAPR_HTTP_PORT=3500
    ports:
      - "7000:80"
      - "7001:443"
      - "50001:50001"
    volumes:
      - ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
      - ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro
    dns:
      - 8.8.8.8
      - 127.0.0.11

  yarp-dapr:
    command: ["./daprd",
     "-app-id", "apigateway",
     "-app-port", "80",
     "-dapr-http-port", "3500",
     "-components-path", "/dapr/components",
     "-config", "/dapr/dapr-config.yaml",
     "-log-level", "info"]
    volumes:
        - "./dapr-components/docker:/dapr"

  identityservice.api:
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      - ASPNETCORE_URLS=https://+:443;http://+:80
    ports:
      - "7100:80"
      - "7101:443"
      - "50002:50001"
    volumes:
      - ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
      - ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro

  identityservice.api-dapr:
    command: ["./daprd",
     "-app-id", "identityservice",
     "-app-port", "80",
     "-dapr-http-port", "3500",
     "-components-path", "/dapr/components",
     "-config", "/dapr/dapr-config.yaml",
     "-log-level", "info"]
    volumes:
        - "./dapr-components/docker:/dapr"

  configurationservice.api:
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      - ASPNETCORE_URLS=https://+:443;http://+:80
    ports:
      - "7200:80"
      - "7201:443"
      - "50004:50001"
    volumes:
      - ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
      - ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro

  configurationservice.api-dapr:
    command: ["./daprd",
     "-app-id", "configurationservice",
     "-app-port", "80",
     "-dapr-http-port", "3500",
     "-components-path", "/dapr/components",
     "-config", "/dapr/dapr-config.yaml",
     "-log-level", "info"]
    volumes:
        - "./dapr-components/docker:/dapr"

  legacyproxyservice.api:
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      - ASPNETCORE_URLS=https://+:443;http://+:80
    ports:
      - "7300:80"
      - "7301:443"
      - "50005:50001"
    volumes:
      - ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
      - ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro

  legacyproxyservice.api-dapr:
    command: ["./daprd",
     "-app-id", "legacyproxyservice",
     "-app-port", "80",
     "-dapr-http-port", "3500",
     "-components-path", "/dapr/components",
     "-config", "/dapr/dapr-config.yaml",
     "-log-level", "debug"]
    volumes:
        - "./dapr-components/docker:/dapr"

  redis:
    ports:
      - "6379:6379"

  zipkin:
    ports:
      - "9411:9411"

The depends_on attribute doesn't change a thing, not even removing it.

The only Service-Project referencing IdentityService.Domain is IdentityService.API which is also the Name of the Container (Docker-Compose). It does not help changing anything in the IdentityService.API project, it also does not build when changing something in the "main"-project.

danegsta commented 1 year ago

@aschwab 17.4 Preview 3 which contains our fix for the depends_on parsing issue has been released; can you do a quick test with the latest preview build to see if that helps with the issue. In the meantime, I'll continue to do some additional testing on my end to see if there's something else that's been missed.

aschwab commented 1 year ago

@danegsta I've downloaded the newest VS 17.4 Preview 3 and it still does not work

Steps to reproduce:

Start VS
Set docker-compose as Startup Project
Hit F5 and run into breakpoint
Change something in the projects built with docker-compose
Hit F5 again
See that the breakpoint is not bound / hit because the source code does not match the debugged file

I can't say if this problem is a container tools problem or a visual studio problem. However as I said im happy to show you my solution in Teams / Discord.. whatever.

danegsta commented 1 year ago

@aschwab if you want to shoot me an email at my username at microsoft.com, I can see about scheduling a Teams meeting with you to try and debug the problem.

itsoli91 commented 1 year ago

Hi Same problem here.

danegsta commented 1 year ago

@itsoli91 are you using depends_on in any of your docker-compose yaml?

itsoli91 commented 1 year ago

Yes, I use in docker-compose.override.yml

danegsta commented 1 year ago

Have you had a chance to try with VS 2022 17.4 Preview? https://visualstudio.microsoft.com/vs/preview/ We shipped a fix in the preview for an issue with depends_on parsing that would prevent the compose project from properly declaring its dependent projects for build.

itsoli91 commented 1 year ago

Not yet, but tomorrow I will try and let you know if it fixes the issue

danegsta commented 1 year ago

Sounds good; hopefully that gets you unblocked, but I'll keep an eye out for an update if that doesn't work for you.

itsoli91 commented 1 year ago

Hey, I updated to 17.4 and still the problem was there. But somehow I could fix this problem. Right click on solution => select Project build order In my case docker-compose was on top of the list, it means first docker-compose was built then other projects. Then I selected the dependencies tab and made docker-compose dependent on all other projects. Now docker-compose is at the bottom of the list and everything works fine. But I'm not sure it would be the actual way to fix this problem.

danegsta commented 1 year ago

@itsoli91 that should be a decent workaround for the issue as you're effectively manually setting what our dependency checks should automatically be declaring. I believe there's likely an issue related to parsing some non-standard project type within the VS solution that's causing our dependency check to fail and report no dependencies. I haven't managed to find a project type yet that triggers a failure, but I'm continuing to investigate.

darthmolen commented 1 year ago

We have the same issue with our current solution, which heavily relies on dapr. I don't know if this is happenstance or not. I will test with preview for 17.4 as well.

danegsta commented 1 year ago

We've identified what should be the root cause for this issue and have a fix in place that will first release in the upcoming 17.5 Preview 2. I'll update this issue once that release is available.

esfernandez commented 1 year ago

I have the version 17.6, and still the problem was there. I test to change compilation order but not work for me.

danegsta commented 1 year ago

@esfernandez when you right click on your docker-compose project and view the Build Dependencies > Project Dependencies dialog, does it let you remove the dependencies on service projects or does it give you a warning that a dependency was added by the Project System?

netcompany-runeviumsondergaard commented 3 months ago

Still seems to be a problem for me. I am using the long form of depends_on.

microsoft / DockerTools

Visual Studio 2022 17.3.5: Docker-Compose needs rebuild to debug new changes #358