ng build needs to be optimized for docker

[tbnovaes]

I use angular cli version 9, and I'm trying to run my app on docker, however, I don't want to install dev dependencies when doing a production build.

Unfortunately, angular-cli does not allows me to build my project using a globally installed cli, so every time I'm generating my docker image, I have to wait 5 min for installing @angular/cli and another 5 min to install the production packages.

On top of that angular takes around other 8 min to build my project, because it gets stuck on 92% for quite some time, and another 5 min to copy files over to my nginx public directory and spin de server. So basically we are talking about 23 min.

Notes: all docker cache gets invalidated with any changes on package.json, so considering that we are always adding/updating/removing a dependency, or bumping the version, we can't rely on cached layers.

[danieldiazastudillo]

Is this in your local machine or Docker Hub? (or some CD)

[tbnovaes]

Both when running docker locally and on the gitlab CD (shared runners).

My local docker is setup with 4 core, 16gb ram and 4gb swap.

[flash-me]

5 min for installing @angular/cli and another 5 min to install the production packages.

This is a lot.

1. This sounds like you are installing the cli globally on the container. If so, why though? You don't have to install the cli globally.
2. You might consider trying npm ci. But this requires a package-lock.json to be available.
3. npm can perform faster with setting some options, especially with progress=false, and since you are in a container, also consider setting audit=false
4. Did you monitored the RAM usage? nodejs limits at ~2GB by default. When reached, every further step require GC, which slows down massively. You can pass a flag --max_old_space_size to nodejs to increase memory limit Since v8 it is also possible to set NODE_OPTIONS via environment variables which comes pretty handy.
5. Even though docker container are not persisted, we worked this around with mounting a volume and set the npm cache to this mount. Works also with multiple container as long as they do not write new packages to the cache (which is prevented by using npm ci, though)

cheers flash :zap:

[tbnovaes]

No, I'm not installing it globally.

This is how my Dockerfile looks like:

### STAGE 1: Build ###
FROM node:12.3.1-alpine as builder

WORKDIR /usr/src/app

COPY . .

RUN npm i @angular/cli --no-progress --loglevel=error
RUN npm i --only=production --no-progress --loglevel=error

RUN npm run build:app

### STAGE 2: Setup ###
FROM nginx:alpine

COPY nginx/default.conf /etc/nginx/conf.d/
RUN rm -rf /usr/share/nginx/html/*
COPY --from=builder /usr/src/app/dist /usr/share/nginx/html
CMD ["nginx", "-g", "daemon off;"]
EXPOSE 4200

And this is my build:app script on packages.json:

build:app: node --max_old_space_size=16384 ./node_modules/@angular/cli/bin/ng build --progress=false --prod

[flash-me]

@tbnovaes seems to be fine. But still...
at 92% terser fires, this takes some time in general

• 5min each time for npm install
• 5min for deployment to nginx

Are you ensured that the bottleneck is the container itself and not the network? Some sort of aggressive firewall or overchallenged company gateway? Without knowing how much data is actually being transfered, but are we talking about 10mb? 100mb? But even 100mb should not take 5min.

cheers flash :zap:

[tbnovaes]

@flash-me final bundle has 12.8MB, I'm now using the npm ci as you recommended, and now it is taking 5 min to install prod + dev, so it already better, but still taking about 15 min.

[flash-me]

I'm now using the npm ci as you recommended, and now it is taking 5 min to install prod + dev, so it already better, but still taking about 15 min.

Fine. Did you also adjust your dockerfile? Its faster to npm ci first and then do npm i @angular/cli, since npm ci removes existing node_modules dir.

final bundle has 12.8MB

Unless you are not uploading hundreds of small files I'd say you may have a look at your network, firewall, repository manager (like artifactory or nexus). +
Give a shot for the mounted volume as npm cache for your containers.

Cheers flash :zap:

[Zwartpet]

@tbnovaes could be https://github.com/angular/angular-cli/pull/17028

[flash-me]

@tbnovaes could be #17028

don't think so, since @tbnovaes wrote

Both when running docker locally and on the gitlab CD (shared runners). My local docker is setup with 4 core, 16gb ram and 4gb swap.

I'm still thinking it's more network or firewall related. Besides the 5min for npm install, it also takes 5min for deployment of 13mb

and another 5 min to copy files over to my nginx public directory and spin de server

But I might be wrong. I'm curious how this will end

cheers flash :zap:

[tbnovaes]

@Zwartpet not sure... I set all the computer resources for my docker, and it still runs slow. @flash-me so far I'm stuck in the same 15 min run... would you be able to provide me a guide for the mounted volume as npm cache for your containers.

[flash-me]

I think you don't even need a mount for your use cases. More info here

cheers flash :zap:

[tbnovaes]

@flash-me I was doing the steps on the link before, but since I always bump the package.json version, package.json was always invalidating the cache.

[flash-me]

Then you may use a mount for your container

• create the container with a mount docker run....-v 'LOCAL_PATH:CONTAINER_PATH'
• within your DOCKERFILE set your npm cache path with npm config set cache CONTAINER_PATH
• ???
• profit

cheers flash :zap:

[nicolae536]

@flash-me I'm having the same issue docker caches node_modules but could you provide a way to cache this part of the ng build ? Seems a bit redundant to redo this on every docker build.

Compiling @angular/core : es2015 as esm2015
Compiling @angular/common : es2015 as esm2015
Compiling @angular/cdk/platform : es2015 as esm2015
Compiling @angular/cdk/bidi : es2015 as esm2015
Compiling @angular/cdk/keycodes : es2015 as esm2015
Compiling @angular/platform-browser : es2015 as esm2015
Compiling @angular/animations : es2015 as esm2015
Compiling @angular/cdk/observers : es2015 as esm2015
Compiling @angular/animations/browser : es2015 as esm2015
Compiling @angular/cdk/a11y : es2015 as esm2015
Compiling @angular/platform-browser/animations : es2015 as esm2015
Compiling @angular/forms : es2015 as esm2015
Compiling @angular/material/core : es2015 as esm2015
Compiling @angular/cdk/collections : es2015 as esm2015
Compiling @angular/cdk/scrolling : es2015 as esm2015
Compiling @angular/cdk/portal : es2015 as esm2015
Compiling @angular/cdk/overlay : es2015 as esm2015
Compiling @angular/material/form-field : es2015 as esm2015
Compiling @angular/common/http : es2015 as esm2015
Compiling @angular/material/button : es2015 as esm2015
Compiling @angular/cdk/layout : es2015 as esm2015
Compiling @ngrx/store : es2015 as esm2015
Compiling @angular/cdk/text-field : es2015 as esm2015
Compiling @angular/material/icon : es2015 as esm2015
Compiling @angular/material/select : es2015 as esm2015
Compiling @angular/material/input : es2015 as esm2015
Compiling @angular/material/tooltip : es2015 as esm2015
Compiling @angular/core/testing : es2015 as esm2015
Compiling @angular/material/dialog : es2015 as esm2015
Compiling @angular/cdk/accordion : es2015 as esm2015
Compiling @angular/material/divider : es2015 as esm2015
Compiling @angular/cdk/stepper : es2015 as esm2015
Compiling @angular/cdk/table : es2015 as esm2015
Compiling @angular/material/paginator : es2015 as esm2015
Compiling @angular/material/sort : es2015 as esm2015
Compiling @angular/cdk/tree : es2015 as esm2015
Compiling @angular/platform-browser-dynamic : es2015 as esm2015
Compiling @angular/platform-browser/testing : es2015 as esm2015
Compiling @angular/compiler/testing : es2015 as esm2015
Compiling @angular/common/testing : es2015 as esm2015
Compiling @angular/router : es2015 as esm2015
Compiling @ngrx/effects : es2015 as esm2015
Compiling @ngx-translate/core : es2015 as esm2015
Compiling @angular/material/checkbox : es2015 as esm2015
Compiling @angular/material/progress-spinner : es2015 as esm2015
Compiling @agm/core : es2015 as esm2015
Compiling @angular/animations/browser/testing : es2015 as esm2015
Compiling @angular/cdk/clipboard : es2015 as esm2015
Compiling @angular/cdk/drag-drop : es2015 as esm2015
Compiling @angular/common/http/testing : es2015 as esm2015
Compiling @angular/material/autocomplete : es2015 as esm2015
Compiling @angular/material/badge : es2015 as esm2015
Compiling @angular/material/bottom-sheet : es2015 as esm2015
Compiling @angular/material/button-toggle : es2015 as esm2015
Compiling @angular/material/card : es2015 as esm2015
Compiling @angular/material/chips : es2015 as esm2015
Compiling @angular/material/datepicker : es2015 as esm2015
Compiling @angular/material/expansion : es2015 as esm2015
Compiling @angular/material/grid-list : es2015 as esm2015
Compiling @angular/material/icon/testing : es2015 as esm2015
Compiling @angular/material/list : es2015 as esm2015
Compiling @angular/material/menu : es2015 as esm2015
Compiling @angular/material/progress-bar : es2015 as esm2015
Compiling @angular/material/radio : es2015 as esm2015
Compiling @angular/material/sidenav : es2015 as esm2015
Compiling @angular/material/slide-toggle : es2015 as esm2015
Compiling @angular/material/slider : es2015 as esm2015
Compiling @angular/material/snack-bar : es2015 as esm2015
Compiling @angular/material/stepper : es2015 as esm2015
Compiling @angular/material/table : es2015 as esm2015
Compiling @angular/material/tabs : es2015 as esm2015
Compiling @angular/material/toolbar : es2015 as esm2015
Compiling @angular/material/tree : es2015 as esm2015
Compiling @angular/platform-browser-dynamic/testing : es2015 as esm2015
Compiling @angular/router/testing : es2015 as esm2015
Compiling @ngrx/effects/testing : es2015 as esm2015
Compiling @ngrx/entity : es2015 as esm2015
Compiling @ngrx/store/testing : es2015 as esm2015
Compiling @ngrx/store-devtools : es2015 as esm2015
Compiling @ngx-translate/http-loader : es2015 as esm2015
Compiling ngx-file-drop : es2015 as esm2015
Compiling ngx-mat-select-search : es2015 as esm2015

[Zwartpet]

Why are you building the application inside a docker container? It can be solved if the project/node_modules directory is on a persistent volume. You could also cache the node_modules directory and re-add it to the docker container on the next run

[flash-me]

Why are you building the application inside a docker container? It can be solved if the project/node_modules directory is on a persistent volume. You could also cache the node_modules directory and re-add it to the docker container on the next run

@Zwartpet thanks 😅. Depending on the setup, this can be problematic. At least we had issues if the node_modules were persisted along multiple builds, such as leftovers, wrong versions etc.

But it could fit into your use cases. Worth a try @tbnovaes @nicolae536

cheers flash :zap:

[Zwartpet]

I'm using an md5 generated hash of the packages.json and packages-lock.json files to cache the node_modules. That way it should never have wrong versions.

[nicolae536]

@Zwartpet @flash-me We use a docker container in our ci :)) and the container contains both frontend and backend so the solution for caching also the ng build for external packages I should try to make project/node_modules a persitent volume I'm getting this right ?

[Zwartpet]

Thats one way yes, which ci tool do you use? Most ci tools have the caching strategy i mentioned before.

[SmallhillCZ]

Hello, we are building our apps using DockerHub and I agree with @nicolae536 that the ESM compilation should be made possible to run as a separate step. This way DockerHub would cache it using the standard Docker caching mechanism.

To illustrate with Dockerfile:

FROM node:lts as build

# From this point it is run when package.json or package-lock.json change
COPY package.json package-lock.json ./
RUN npm ci

# HERE I NEED TO RUN ESM COMPILATION

# From this point it is run when application source changes
COPY . .
RUN ng build --prod

FROM nginx:alpine
WORKDIR /usr/share/nginx/html
COPY --from=build /home/node/app/dist ./

[nicolae536]

@Zwartpet we use an agent from azure but on some of the project we use bitbucket pipelines

[flash-me]

@SmallhillCZ You can manually trigger the ESM Compilation this way:

./node_modules/.bin/ngcc --properties es2015

cheers flash :zap:

[nicolae536]

@flash-me Thanks I added a step in docker with this and works like a charm with default docker caching mechanism

[flash-me]

Can you provide before and after times? I'm kinda curious

cheers flash :zap:

[nicolae536]

@flash-me Full docker build frontend + backend without the inprovement 6:05 minutes. Full docker build frontend + backend with the inprovement 2:30 minutes.

[mhamri]

for me ./node_modules/.bin/ngcc --properties es2015 didn't work but this worked fine for me RUN npx ngcc --properties es2015 browser module main --create-ivy-entry-points I took this ngcc command from npm post-install. it had a --first-level-only but with try and error, found out that the argument isn't helping and still on using npm build it does the compilation

[alan-agius4]

Hello, we reviewed this issue and determined that it doesn't fall into the bug report or feature request category. This issue tracker is not suitable for support requests, please repost your issue on StackOverflow using tag angular-cli.

If you are wondering why we don't resolve support issues via the issue tracker, please check out this explanation.

[mhamri]

@alan-agius4 it's a bug, whatever command angular is running on the first run, isn't what it's running on the npm install, that's why it's failing on multi-stage build. it's a bug that needs to be resolved from the angular side and to be precise it needs to run the same command on npm install and npm build so the ngcc results being cached. you can see my solution to know why it's a bug.

unfortunately, you failed to understand the problem and closed the issue wrongly. please reopen it.

[SchnWalter]

@mhamri, you need to brush up on your docker skill. First of all, you should cache your node_modules outside of the docker container. Also, don't run your npm commands or scripts as the root user, the npm postinstall is not executed if you run as root. thus you get no Ivy caches

https://stackoverflow.com/questions/47748075/npm-postinstall-not-running-in-docker

[mhamri]

@SchnWalter I'm confident about my skills, thanks for the suggestion. but I believe you should brush up your googling skills, that result is for 2017 and it's not the case anymore since npm 6.

looking at your answer you are not familiar with the latest best practices for nodejs apps docker builds. nowadays node_module is being cached in the docker.

all the npm installs are run as node user. so, I don't think that is the case.

FROM node:lts-alpine as nodeinstall 
ARG srcPath
ARG containerPath
ARG npmCache
ARG homePath
ENV TERM xterm-256color

RUN apk update && apk add --no-cache tini
WORKDIR ${containerPath}
COPY ${srcPath}/package*.json ./
RUN mkdir -p node_modules && chown -R node:node ${containerPath}
USER node
RUN npm set unsafe-perm true
RUN npm config set cache ${npmCache}
RUN /sbin/tini -g -- npm install
RUN /sbin/tini -g -- npx ngcc --properties es2015 browser module main --create-ivy-entry-points
COPY --chown=node:node ${srcPath} .

as you can see, even with setting the unsafe-prem still angular is unable to build correctly and need an extra ngcc. we have many other docker including vue and bare nodejs apps that doesn't have these problems.

as I mentioned I did my due diligence before coming posting in here. hope you can triage and find the bug at your side too before closing it for sake of closing it.

[angular-automatic-lock-bot[bot]]

This issue has been automatically locked due to inactivity. Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}