CFPM_INSTALL doesn't appear to be doing anything.

[bennadel]

I just went to start my local Docker setup, which I haven't done in a little while, and things aren't working. I was using:

FROM ortussolutions/commandbox:adobe2021-3.4.4

... and this was giving me an issue saying something like the "packages are not being installed because I am on Updater 2 and they require Updater 5". So then, to try and fix that, I figured that I should pull the latest CommandBox image and adjust my ENV values. So now, I have this in my Dockerfile:

FROM ortussolutions/commandbox:adobe2021

... and I've updated my docker-compose.yml to have:

  cfml:
    build: "./build/cfml"
    ports:
      - "8080:8080"
    volumes:
      - "../bennadel.com:/app"
      - "./build/cfml/.cfconfig.json:/app/wwwroot/.cfconfig.json"
    environment:
      BOX_SERVER_APP_CFENGINE: "adobe@2021.0.05+330109"
      BOX_SERVER_PROFILE: "development"
      CFPM_INSTALL: "mail,image"
      APP_DIR: "/app/wwwroot"

Notice the CFPM_INSTALL: "mail,image" line. However, when I go to docker-compose up my server, I see this in the logs:

image package will not be deployed as it is not installed.
mail package will not be deployed as it is not installed.

I'm not sure what I'm doing wrong. I'm not very knowledgeable about containers, so I assume I am just missing something silly. But, I cannot for the life of me figure out what it is.

[bennadel]

I should also mention that when I build it, I use the --no-cache flag:

docker-compose build --no-cache cfml

[bennadel]

Also, I can log into the CFIDE, and have confirmed that it's the correct ACF version and JDK. So, I'm definitely running the correct image.

[bdw429s]

@bennadel This issue has been discussed on Stackoverflow, the Adobe forums, and CFML Slack several times in the past few days. Basically, Adobe's package manager sucks and you are not using the very latest Adobe CF core version. I explained the issue here

https://tracker.adobe.com/#/view/CF-4215444

Now, that said, you seem to be saying you've switched to use adobe@2021.0.05+330109 but I'm not clear if that setting is actually in use ATM. Keep in mind you can't just switch the engine version on an existing docker container and if you inspect the console output when it starts you'll see a note explaining why. You need to remove the image and start a fresh one in order to get Adobe update 5 actually installed, at which point cfpm will work again.

[bennadel]

@bdw429s Hmm, I'm not seeing any issues in the logging. All I see is this that might indicate version:

cfml_1   |    | Server start command: 
cfml_1   |    |     /opt/java/openjdk/bin/java 
cfml_1   |    |     -Djdk.io.File.enableADS=true 
cfml_1   |    |     -jar /usr/local/lib/CommandBox/lib/runwar-4.7.14.jar 
cfml_1   |    |     --background=false 
cfml_1   |    |     --host 0.0.0.0 
cfml_1   |    |     --stop-port 41979 
cfml_1   |    |     --processname wwwroot [adobe 2021.0.05+330109] 

But, reading your comments in the Adobe Tracker , this is unfortunate - if only there were decades worth of prior art for installing packages :troll:. Ok, I'll keep playing around with it.

[bdw429s]

@bennadel The relevant portion of the console logs would be above what you included. Can you copy the FULL console logs into a GIST (look out for any sensitive info). Actually come to think of it- depending on how your images are setup, the logs in question may be part of the image build phase. I'm talking about the "interactive job" ASCII table output that shows the steps of starting the server including the CF Engine being downloaded and installed, etc.

Like this

 √ | Starting Server
   | √ | Setting Server Profile to [development]
   | √ | Installing package [java:openjdk11:lockVersion]
   | √ | Loading CFConfig into server
   | √ | Loading FusionReactor

except it's expanded with all the log lines inside of it.

You may also be pointing out that a version does display in your console logs, but basically you can't trust that. There is sometimes a difference between the version that was requested and the actual version you get if your server has already started a previous version and "locked" that into the server home. It's all explained in the logs I'm talking about.

[bennadel]

Here's my docker-compose up output for the CFML container. I don't see any warning about the version:

https://gist.github.com/bennadel/1e1bb3e5238c473177324db56653b00f

[bdw429s]

So, the portion of the logs I was looking for just shows this

WAR/zip archive already installed.

which means these logs don't represent the first time the server was started. This is a subsequent start. I can't see your actual configs, docker compose, or env vars, so I can't really speak to what may be wrong or not. Everyone else who reported this issue last week resolved it right away by simply updating to the latest version of Adobe CF. I'm not really clear if you've done that or how your docker files are set up so it's hard to help at this point without more information.

[bennadel]

@bdw429s Hmm, no worries. All I know is that I used the --no-cache flag when building the container. But, to be honest, all of the layer-caching really confuses me in Docker. For the time-being, I was able to get it working by switching over to the native ACF Docker image; though, Charlie Arehart had to help me there with replicating the APP_DIR functionality that CommandBox provides:

https://github.com/carehart/awesome-cf-compose/issues/2

This isn't a knock against CommandBox -- I was trying to unblock my local development.

Re: I can't see your actual configs, docker compose, or env vars, so I can't really speak to what may be wrong or not, I have the basics of it at the top of this thread. Literally the only thing that I left out was the COPY of the MySQL JAR file. My Dockerfile literally has two lines in it (FROM and COPY). The rest is controlled by the docker-compose which I have at the top.

Docker is still mostly a mystery to me :D

[bdw429s]

@bennadel Sorry to hear you've completely jumped ship on the image over something so simple. I did in fact miss you had posted the exact image tag above and that actually reveals the problem

FROM ortussolutions/commandbox:adobe2021

You've locked your docker images into a specific tag from 9 months ago which is hard-coded to Adobe CF 2021 update 2! See the tag details here:

https://hub.docker.com/layers/ortussolutions/commandbox/adobe2021-3.4.4/images/sha256-429c026029da87829dcdef3c187b6f96a5b36a2884cbd59e310bfb143032ee79?context=explore

So, your

BOX_SERVER_APP_CFENGINE: "adobe@2021.0.05+330109"

env var is basically ignored here! I always recommend people stop using the pre-warmed images because of this very reason. If you want to use a custom ACF version that you specify in an env var that means you just want to use the vanilla image!

FROM ortussolutions/commandbox

That said, I'm still a little confused why the output I was expecting didn't appear in your console but it's hard to say. Normally in this scenario, there's a message that says something along the lines of "You requested version xyz but your server has a hard-coded server home with version abc already installed."

So my original reply to you here still stands as true. One of the following needs to happen

• Update your server to actually be starting update 5 (which means stop using that prewarmed image from 9 months ago!)
• or change your cfpm install list to include compatible versions of each extension that will install into update 2

... and then the CommandBox images will work just fine.

In the future, if you're having enough issues you're looking to spend the time to completely switch image, please hop into CFML Slack where you can get help in real time. I hate to think how much time you spent switching images and when all that was needed was to remove the :adobe2021 image tag. Also, apologies for not noticing that sooner.

[bennadel]

over something so simple.

... simple for you 😛 I had misunderstood what the tags were. I though the :adobe2021 meant that the base image came with ACF 2021, but that the updaters would still be installed based on the app-engine ENV.

What's strange though, is that I could swear that when I was logging into the CFIDE, it was showing me the correct version of ACF + update level as well. Let me revert back and see I maybe I just was confused by the version (and missed that updater 2 was still installed).

[bdw429s]

simple for you

Well yes, which was why I was trying to help you. 😃 I said from the beginning you were just starting an older version of Adobe despite what your env vars said and I was just trying to get the right logs to be able to pinpoint it for you. That's why it was a little disappointing for me to see you spend all that time giving up before we even got a few message in here in nailing down the source.

I had misunderstood what the tags were

Don't feel to bad about that-- you're not the first! Basically, once you lock in a tag with a pre-warmed image, you get that image regardless (CommandBox won't change CF engines when a server is already stared AND there is a hard-coded server home dir, which the docker images use)

I rarely suggest anyone use the tags for any reason unless they just want a super quick and dirty server that doesn't need to download the image. Once you start building anything of substance where you are creating a custom image or using env vars to customize stuff, I always recommend the vanilla image which installs everything fresh when it builds based on your settings.

And on a related note, here's that warning message I was expecting to see in your logs.

https://github.com/Ortus-Solutions/commandbox/blob/development/src/cfml/system/services/ServerEngineService.cfc#L257-L259

[bennadel]

So, I went back this morning and tried to get my CommandBox configuration in place, just to make sure I wasn't going crazy. Here's my Dockerfile:

FROM ortussolutions/commandbox:adobe2021

RUN box config set server.singleServerMode=false

ADD ./lib/mysql-connector-java-8.0.22.jar /usr/local/lib/serverHome/WEB-INF/cfusion/lib/
RUN rm /usr/local/lib/serverHome/WEB-INF/cfusion/lib/mysql-connector-java-8.0.25.jar

And, here's my docker-compose.yml:

  cfml:
    build: "./build/cfml"
    ports:
      - "8080:8080"
    volumes:
      - "../bennadel.com:/app"
      - "./build/cfml/.cfconfig.json:/app/wwwroot/.cfconfig.json"
    environment:
      BOX_SERVER_APP_CFENGINE: "adobe@2021.0.05+330109"
      BOX_SERVER_PROFILE: "development"
      CFPM_INSTALL: "mail,image"
      APP_DIR: "/app/wwwroot"
    depends_on:
      mysql:
        condition: service_healthy
    healthcheck:
      test: "echo hello"

I then run:

docker-compose build --no-cache cfml
docker-compose up

Now, when this comes online, it said mail, and image was not deployed because it was not installed. However, when I go into the CFIDE once the container starts-up, I do, at least, see that the correct version of ACF is running:

So, it seems like the version is correct. And maybe that's just because that's what is baked into the :adobe2021 image at this point. Which leads me to believe that it's not the BOX_SERVER_APP_CFENGINE that is being ignored - that it's the CFPM_INSTALL that is being ignored. Does this also get ignore for the pre-warmed images?

[bennadel]

Ok, I tried to go back and use the tag-less image, and it's still a no-go. Here's my current Dockerfile:

FROM ortussolutions/commandbox
RUN box config set server.singleServerMode=false

And my docker-compose:

  cfml:
    build: "./build/cfml"
    ports:
      - "8080:8080"
    volumes:
      - "../bennadel.com:/app"
      - "./build/cfml/.cfconfig.json:/app/wwwroot/.cfconfig.json"
    environment:
      BOX_SERVER_APP_CFENGINE: "adobe@2021.0.05+330109"
      BOX_SERVER_PROFILE: "development"
      CFPM_INSTALL: "mail,image"
      APP_DIR: "/app/wwwroot"
    depends_on:
      mysql:
        condition: service_healthy
    healthcheck:
      test: "echo hello"

When I run:

docker-compose down
docker-compose build --no-cache cfml

Here's the output I get:

docker-compose build --no-cache cfml
Building cfml
[+] Building 16.3s (6/6) FINISHED                                                                                                                                           
 => [internal] load build definition from Dockerfile                                                                                                                   0.0s
 => => transferring dockerfile: 1.22kB                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/ortussolutions/commandbox:latest                                                                                            0.9s
 => CACHED [1/2] FROM docker.io/ortussolutions/commandbox@sha256:db99c4359d975573cdb68fe0816f86687b4ae1f83ef889a4316c2a26e7f21a31                                      0.0s
 => [2/2] RUN box config set server.singleServerMode=false                                                                                                            14.6s
 => exporting to image                                                                                                                                                 0.7s
 => => exporting layers                                                                                                                                                0.6s
 => => writing image sha256:38a0097831ccd30d94cf4532002283fb4eba083e37060a98ef7a33be5c87fe40                                                                           0.0s
 => => naming to docker.io/library/bennadelcom-docker_cfml                                                                                                             0.0s

Then, when I run:

docker-compose up

... here's the output (in a gist): https://gist.github.com/bennadel/6fb2ea839bef1c172ba1b61aa2d36a5a

In that, I can see, Eureka, 'adobe@2021.0.05+330109' has been installed!; but, then I also see:

image package will not be deployed as it is not installed.
mail package will not be deployed as it is not installed.

So it seems that even when I use the FROM ortussolutions/commandbox image, no CFPM_INSTALL ENV is working.

[jclausen]

@bennadel I'm looking in to this and don't think it's you or your dockerfile. It looks like one of the environment variables that script expects is empty at runtime. I will push up a patch for this later today.

If you are building an image using a Dockerfile on an already seeded image, I would perform the installation of the modules in the Dockerfile.

If you go from the base image, that would look like this:

FROM ortussolutions/commandbox

# Always install the latest version
ENV BOX_SERVER_CFENGINE=adobe@2021

# Warm up the server to seed the engine
RUN $BUILD_DIR/util/warmup-server.sh

# Install our necessary modules
RUN box cfpm install mail,image

[bennadel]

@jclausen Interesting. I'll try that. I'm still a bit fuzzy on what goes in the Dockerfile vs. the docker-compose. The picture is slowly coming together.

[jclausen]

@bennadel For advanced apps where additional dependencies are needed ( e.g. wkhtmltopdf or additional CommandBox modules), I generally use the Dockerfile to build and specify that in my Compose file.

This also makes it re-usable for building production images.

# syntax = edrevo/dockerfile-plus
INCLUDE+ build/docker/app/Dockerfile

COPY ./ ${APP_DIR}/

The INCLUDE syntax won't work with a vanilla docker build command, so you need to specify the buildkit for the above to work

DOCKER_BUILDKIT=1; docker build -f build/env/production/Dockerfile .

I have resolved this issue and will be publishing a release later this afternoon. For now, ortussolutions/commandbox:adobe2021-snapshot should resolve the issues with the CFPM_INSTALL environment variable support.

Once I publish later today versions of the image >= 3.6.1 will have this resolved.

[bdw429s]

Thanks for jumping in Jon and looking at the cfpm issue.

@bennadel So it looks like there ended up being two issues.

• The base image tag being pinned to 2021.0.2 which you fixed by updating/removing the tag
• A bug in the actual docker image that was ignoring your CFPM_INSTALL so it wasn't running, which Jon is fixing ASAP

In the mean time, this command

RUN box cfpm install mail,image

does the equivalent of the CFPM_INSTALL env var. The only real down side is it does require you to have a DockerFile.

If you really don't want to create a docker file, you can also put the cfpm install mail,image in your server.json's scripts section.

{
    "scripts":{
        "onServerInstall":"cfpm install mail,image"
    }
}

Of course, none of these workarounds will be needed as soon as Jon gets the CFPM_INSTALL env var working again. Just wanted to give you some options.

[bennadel]

I just wanted to circle back and say that this is working now. I had briefly switched over to the ACF containers. But, yesterday I switched over to my new MacBook M1 and it doesn't look like Adobe has ARM builds. So, back to the CommandBox images and everything is working perfectly:

FROM ortussolutions/commandbox
ADD ./lib/mysql-connector-java-8.0.22.jar /usr/local/lib/serverHome/WEB-INF/cfusion/lib/

  cfml:
    build: "./build/cfml"
    ports:
      - "8080:8080"
    volumes:
      - "../bennadel.com:/app"
      - "./build/cfml/.cfconfig.json:/app/wwwroot/.cfconfig.json"
    environment:
      BOX_SERVER_APP_CFENGINE: "adobe@2021.0.05+330109"
      BOX_SERVER_PROFILE: "development"
      CFPM_INSTALL: "mail,image,debugger"
      APP_DIR: "/app/wwwroot"
    depends_on:
      mysql:
        condition: service_healthy
    healthcheck:
      test: "echo hello"

And I can confirm that the modules are all installed properly. Awesome stuff fellas!

[bdw429s]

Thanks for the update @bennadel and I'm glad it's all working for you now.