[image]: cadence-dev is not building (convert to Docker Compose)

[jefflill]

The cadence-dev container image isn't building. I did remove tini from the base Ubuntu layer but this shouldn't impact anything (the image doesn't use this).

I'm going to disable the image build for now and perhaps we should just convert to Docker Compose like we do for Temporal.

NOTE: Note that unit testing will still work because a working image is still over at DockerHub and that's what CadenceFixture is still configured to load.

ERROR LOG (for completeness): There appears to be a problem with checking out the tagged commit.

===========================================================                                                                                                                                                                                                                     
* CADENCE-DEV:v0.5.9                                                                                                                                                                                                                                                            
* GO_VERSION:1.12.6                                                                                                                                                                                                                                                             
* CADENCE_VERSION:v0.5.9                                                                                                                                                                                                                                                        
* CADENCE_UI_VERSION:3.3.1                                                                                                                                                                                                                                                      
===========================================================                                                                                                                                                                                                                     
[+] Building 39.7s (19/28)                                                                                                                                                                                                                                                      
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                       0.0s 
 => => transferring dockerfile: 32B                                                                                                                                                                                                                                        0.0s 
 => [internal] load .dockerignore                                                                                                                                                                                                                                          0.0s 
 => => transferring context: 2B                                                                                                                                                                                                                                            0.0s 
 => [internal] load metadata for docker.io/library/openjdk:8-jdk-stretch                                                                                                                                                                                                   1.8s 
 => [internal] load metadata for docker.io/library/golang:1.12.6                                                                                                                                                                                                           1.9s 
 => [auth] library/openjdk:pull token for registry-1.docker.io                                                                                                                                                                                                             0.0s 
 => [auth] library/golang:pull token for registry-1.docker.io                                                                                                                                                                                                              0.0s 
 => [stage-1  1/19] FROM docker.io/library/openjdk:8-jdk-stretch@sha256:d8d62fe0d8e9f3e6e62921c7d738c9a962efd6887b2b282dc2f852d7f1ee4512                                                                                                                                   0.0s 
 => CACHED [builder 1/2] FROM docker.io/library/golang:1.12.6@sha256:35200a727dc44175d9221a6ece398eed7e4b8e17cb7f0d72b20bf2d5cf9dc00d                                                                                                                                      0.0s 
 => [internal] load build context                                                                                                                                                                                                                                          0.0s 
 => => transferring context: 174B                                                                                                                                                                                                                                          0.0s 
 => ERROR [builder 2/2] RUN go get -u github.com/golang/dep/cmd/dep     && go get -u golang.org/x/lint/golint     && git clone https://github.com/uber/cadence.git /go/src/github.com/uber/cadence     && cd /go/src/github.com/uber/cadence     && git checkout tags/v0  37.8s 
 => CACHED [stage-1  2/19] RUN apt-get update && apt-get install -y --no-install-recommends         gettext-base         gnupg         wget         tcpdump         netcat         python-pip         curl         python-setuptools         build-essential     && rm -r  0.0s 
 => CACHED [stage-1  3/19] RUN apt-get update     && curl -sL https://deb.nodesource.com/setup_10.x | bash -     && apt-get install -y nodejs                                                                                                                              0.0s 
 => CACHED [stage-1  4/19] RUN apt-get update     && echo "deb https://downloads.apache.org/cassandra/debian 311x main" | tee -a /etc/apt/sources.list.d/cassandra.sources.list     && curl https://downloads.apache.org/cassandra/KEYS | apt-key add -     && apt-get up  0.0s 
 => CACHED [stage-1  5/19] RUN mkdir /cadence     && mv /usr/sbin/cassandra /cadence     && rm -f /etc/cassandra/cassandra.yaml                                                                                                                                            0.0s 
 => CACHED [stage-1  6/19] RUN git clone https://github.com/uber/cadence-web.git /usr/cadence-web     && cd /usr/cadence-web     && git checkout tags/3.3.1                                                                                                                0.0s 
 => CACHED [stage-1  7/19] COPY ./entrypoint.sh /cadence/entrypoint.sh                                                                                                                                                                                                     0.0s 
 => CACHED [stage-1  8/19] COPY ./config_template.yaml /cadence/config/docker_template_cassandra.yaml                                                                                                                                                                      0.0s 
 => CACHED [stage-1  9/19] COPY ./cassandra.yaml /etc/cassandra/cassandra_template.yaml                                                                                                                                                                                    0.0s 
 => CACHED [stage-1 10/19] COPY ./setup.sh /cadence/setup.sh                                                                                                                                                                                                               0.0s 
------                                                                                                                                                                                                                                                                          
 > [builder 2/2] RUN go get -u github.com/golang/dep/cmd/dep     && go get -u golang.org/x/lint/golint     && git clone https://github.com/uber/cadence.git /go/src/github.com/uber/cadence     && cd /go/src/github.com/uber/cadence     && git checkout tags/v0.5.9 && make bi
ns:                                                                                                                                                                                                                                                                             
#19 17.13 Cloning into '/go/src/github.com/uber/cadence'...                                                                                                                                                                                                                     
#19 36.62 Note: checking out 'tags/v0.5.9'.                                                                                                                                                                                                                                     
#19 36.62                                                                                                                                                                                                                                                                       
#19 36.62 You are in 'detached HEAD' state. You can look around, make experimental                                                                                                                                                                                              
#19 36.62 changes and commit them, and you can discard any commits you make in this                                                                                                                                                                                             
#19 36.62 state without impacting any branches by performing another checkout.                                                                                                                                                                                                  
#19 36.62                                                                                                                                                                                                                                                                       
#19 36.62 If you want to create a new branch to retain commits you create, you may                                                                                                                                                                                              
#19 36.62 do so (now or later) by using -b with the checkout command again. Example:                                                                                                                                                                                            
#19 36.62                                                                                                                                                                                                                                                                       
#19 36.62   git checkout -b <new-branch-name>                                                                                                                                                                                                                                   
#19 36.62                                                                                                                                                                                                                                                                       
#19 36.62 HEAD is now at 0d2e1c1b... Release 0.5.9 docker                                                                                                                                                                                                                       
#19 36.65 go get './vendor/go.uber.org/thriftrw'                                                                                                                                                                                                                                
#19 37.57 # cd /go/src/github.com/uber/cadence; git pull --ff-only                                                                                                                                                                                                              
#19 37.57 You are not currently on a branch.                                                                                                                                                                                                                                    
#19 37.57 Please specify which branch you want to merge with.                                                                                                                                                                                                                   
#19 37.57 See git-pull(1) for details.                                                                                                                                                                                                                                          
#19 37.57                                                                                                                                                                                                                                                                       
#19 37.57     git pull <remote> <branch>                                                                                                                                                                                                                                        
#19 37.57                                                                                                                                                                                                                                                                       
#19 37.57 package github.com/uber/cadence/vendor/go.uber.org/thriftrw: exit status 1                                                                                                                                                                                            
#19 37.57 Makefile:89: recipe for target 'yarpc-install' failed                                                                                                                                                                                                                 
#19 37.57 make: *** [yarpc-install] Error 1                                                                                                                                                                                                                                     
------                                                                                                                                                                                                                                                                          
executor failed running [/bin/sh -c go get -u github.com/golang/dep/cmd/dep     && go get -u golang.org/x/lint/golint     && git clone https://github.com/uber/cadence.git /go/src/github.com/uber/cadence     && cd /go/src/github.com/uber/cadence     && git checkout tags/$V
ERSION && make bins]: exit code: 2                                                                                                                                                                                                                                              
Exec: *** FAILED:  docker build -t "${registry}:$tag" --build-arg "VERSION=$version" --build-arg "GO_VERSION=$goVersion" --build-arg "UI_VERSION=$uiVersion" .                                                                                                                  
At C:\src\neonKUBE\Images\includes.ps1:52 char:9                                                                                                                                                                                                                                
+         throw "Exec: $ErrorMessage"                                                                                                                                                                                                                                           
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                                                                                                                                                                                           
    + CategoryInfo          : OperationStopped: (Exec: *** FAILE...=$uiVersion" . :String) [], RuntimeException                                                                                                                                                                 
    + FullyQualifiedErrorId : Exec: *** FAILED:  docker build -t "${registry}:$tag" --build-arg "VERSION=$version" --build-arg "GO_VERSION=$goVersion" --build-arg "UI_VERSION=$uiVersion" .                                                                                    

[johncburns1]

I think the best thing to do here might just be convert to docker-compose like suggested. Less custom stuff to maintain especially since we are not going to be touching this thing anymore really.

[johncburns1]

I'm pretty deep in the temporal message alignment ATM but once I'll take some time to do that conversion.

[jefflill]

Sure. This isn't really a big priority since the old cadence-dev image is still over at DockerHub so unit testing will still work.

[jefflill]

It turns out that the old cadence-dev image isn't actually working either and this is a problem for Loopie. I'm going to assign this to myself and update both the current branch as well as the 1.x branch Loopie is still on.

[johncburns1]

I can take a look at this tonight if you want. Sorry I did not prioritize this because did not know Loopie was experiencing these issues.

[jefflill]

I've updated CadenceFixture in the master and release-1.5.1 branches to use docker-compose rather than the fancy cadence-dev container image Jack built. The advantage here is that we're using the stock Cadence images rather than needing to build Cadence ourselves so we shouldn't see this kind of problem again. This is what we're doing for Temporal as well.

NOTE: This means that the fixture will use the latest version of Cadence server rather than this being pinned to whatever version we built for our old container image. Cadence is supposed to be backwards compatible with earlier versions so this shouldn't really matter.

Here's the new Cadence DockerCompose file:

version: '3'
services:
  cassandra:
    image: cassandra:3.11
    ports:
      - 9042:9042
    environment:
      - HEAP_NEWSIZE=1M
      - MAX_HEAP_SIZE=1024M
    deploy:
      resources:
        limits:
          memory: 1536M
  statsd:
    image: graphiteapp/graphite-statsd
    ports:
      - 8080:80
      - 2003:2003
      - 8125:8125
      - 8126:8126
  cadence:
    image: ubercadence/server:master-auto-setup
    ports:
     - 7933:7933
     - 7934:7934
     - 7935:7935
     - 7939:7939
    environment:
      - CASSANDRA_SEEDS=cassandra
      - STATSD_ENDPOINT=statsd:8125
      - DYNAMIC_CONFIG_FILE_PATH=config/dynamicconfig/development.yaml
    depends_on:
      - cassandra
      - statsd
  cadence-web:
    image: ubercadence/web:latest
    environment:
      - CADENCE_TCHANNEL_PEERS=cadence:7933
    ports:
      - 8088:8088
    depends_on:
      - cadence

NOTE: This also adds graphite-statsd to the mix.

@johncburns1: All of the Cadence unit tests on master are passing except for Test_EndToEnd.BaseDomain. I'm not sure we care about this, but if its easy...

As an aside, the Temporal test fixture compose file is currently failing for me:

System.Exception : Cannot launch application [temporal-dev] - [exitcode=1]: Creating network "temporal-dev_default" with the default driver
    Found orphan containers (temporal-admin-tools, temporal-postgresql) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
    Pulling temporal (temporalio/auto-setup:1.1.0)...
    Pulling temporal-web (temporalio/web:1.1.0)...
    Creating temporal-dev_cassandra_1 ... 
    
    Creating temporal-dev_cassandra_1 ... done
    Recreating temporal               ... 
    
    Recreating temporal               ... done
    Recreating temporal-web           ... 
    Host is already in use by another container
    
    Recreating temporal-web           ... error
    
    ERROR: for temporal-web  Cannot start service temporal-web: driver failed programming external connectivity on endpoint temporal-dev_temporal-web_1 (d145e4c4cc82954c1e881c94d1cde534c4d847a70490bf2904bbb8d7037cb791): Bind for 0.0.0.0:8088 failed: port is already allocated
    ERROR: for temporal-web  Cannot start service temporal-web: driver failed programming external connectivity on endpoint temporal-dev_temporal-web_1 (d145e4c4cc82954c1e881c94d1cde534c4d847a70490bf2904bbb8d7037cb791): Bind for 0.0.0.0:8088 failed: port is already allocated
    Encountered errors while bringing up the project.

[jefflill]

I'm in the middle of back-porting this change to the Loopie 1.51 release branch. This is getting painful. We're going to need to have Loopie upgrade soon.

[jefflill]

Fixed for neonLIBRARY v1.5.1 and v2.11.0