deno server memory is larger than nodejs

[jiawei397]

My team has been using Deno for more than a year. But I found that the memory consumption in docker container is relatively large, which is more than twice that of nodejs with the same function.

This is frustrating, but strangely, if I use Deno inside the program Deno.memoryUsage() is even better than nodejs' process which used process.memoryUsage(). The value is smaller, but it is another case in docker stats.

Here is the test sample of docker I used, I use the origin nodejs http and origin Deno http test, and also test koa and oak.

The console logs:

deno_1   | heapTotal 3.53MB，heapUsed 3.07MB，rss 3.70MB，external:0.00MB
oakjs_1  | heapTotal 5.42MB，heapUsed 3.95MB，rss 4.74MB，external:0.00MB
oak_1    | heapTotal 6.78MB，heapUsed 5.23MB，rss 6.27MB，external:0.00MB
koa_1    | heapTotal 6.93MB，heapUsed 5.50MB，rss 24.12MB，external:0.78MB
node_1   | heapTotal 4.92MB，heapUsed 4.22MB，rss 18.33MB，external:0.52MB

But docker stats：

        CONTAINER           CPU %               MEM USAGE / LIMIT       MEM %               NET I/O             BLOCK I/O           PIDS
deno    c55528ee5171        0.00%               9.355 MiB / 3.858 GiB   0.24%               2.27 kB / 656 B     0 B / 0 B           5
oakjs   a1074c23261b        0.00%               18.3 MiB / 3.858 GiB    0.46%               1.88 kB / 656 B     0 B / 0 B           5
oak     efafc26ffd90        0.00%               24.75 MiB / 3.858 GiB   0.63%               1.88 kB / 656 B     0 B / 0 B           5
koa     78b030ee5801        0.00%               9.227 MiB / 3.858 GiB   0.23%               1.97 kB / 656 B     0 B / 0 B           7
node    ae1d64819c6f        0.00%               6.715 MiB / 3.858 GiB   0.17%               3.1 kB / 656 B      0 B / 0 B           7

The oak bundled js seems to be better than oak which started on typescript.

The following are my two actual Deno projects, which is also the reason why I did this test. The console logs:

project1: heapTotal 50.32MB，heapUsed 47.62MB，rss 49.13MB，external:22.68MB
project2: heapTotal 9.14MB， heapUsed 8.04MB， rss 8.63MB， external:1.16MB

docker stats:

          CONTAINER           CPU %               MEM USAGE / LIMIT       MEM %               NET I/O             BLOCK I/O           PIDS
project1  43144c7cdcb4        0.03%               272.2 MiB / 3.858 GiB   6.89%               51.1 kB / 21.9 kB   0 B / 0 B           5
project2  64700e6c8c7b        0.03%               97.79 MiB / 3.858 GiB   2.48%               516 B / 516 B       0 B / 0 B           6

The above tests were conducted in the docker environment of Linux, nodejs image is node:16-alpine and deno image is denoland/deno:alpine-1.23.2：

Server Version: 1.13.1
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 3
CPUs: 4

It's very comfortable to develop with Deno. I personally like it better than nodejs, but in the actual operation process, the memory occupation is embarrassing. I hope to solve this problem. I believe that the bottom layer is the deno of Rust, In this regard, it should be the advantage, not the opposite.

[btwiuse]

I have similar experience with deno. The memory consumption is usually twice as large as that of node when running the same application.

[kitsonk]

One of the big differences of Deno versus Node.js is that when using TypeScript and type checking, the TypeScript type checker (tsc) is loaded into memory. We don't unload this once we are bootstrapped IIRC. It is something we should look into.

Have you tried comparing a workload generated by deno compile against Node.js? The TypeScript type checker is never loaded in those cases.

[jiawei397]

One of the big differences of Deno versus Node.js is that when using TypeScript and type checking, the TypeScript type checker (tsc) is loaded into memory. We don't unload this once we are bootstrapped IIRC. It is something we should look into.

Have you tried comparing a workload generated by deno compile against Node.js? The TypeScript type checker is never loaded in those cases.

Thank you for your reply.

I just tested it with deno compile, and only made the following modifications. It is indeed half of the memory. It dropped from 21.11M to 12.78M.

I changed the dockerfile of oak like this:

FROM denoland/deno:alpine-1.23.2

EXPOSE 8000

WORKDIR /app

# Prefer not to run as root.
RUN chown -R deno /app
RUN chmod 755 /app

ADD . .

# CMD deno run --allow-net mod.ts
RUN deno compile --allow-net -o oak mod.ts \
&& chmod +x ./oak

CMD ./oak

However, this leads to additional problems.

First, using the above method to build will cause the docker image volume to increase. Of course, I can try docker's as builder, but I don't have a suitable Linux machine to test for the time being.

Another problem is that I found before that I use decorators in my project, but it does not use its information when using deno compile to build. Here is one of my test cases:

import { Reflect } from "https://deno.land/x/deno_reflect@v0.2.1/mod.ts";

function Prop(target: any, propertyKey: string) {
  const type = Reflect.getMetadata("design:type", target, propertyKey);
  console.log("type", type); // type [Function: String]
}
class Person {
  @Prop
  name!: string;
}

Normally, it should print [function: string], when my tsconfig.json has the following configuration enabled:

{
  "compilerOptions": {
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
  }
}

But when I use deno compile --config tsconfig.json aa.ts and when executing the generated file, it print type undefined.

Personally, I hope to make less changes when using docker, such as adding a configuration to ignore the TS check at runtime. Of course, don't forget to read tsconfig.json first so that my decorator can still take effect.

[cjihrig]

Of course, I can try docker's as builder, but I don't have a suitable Linux machine to test for the time being.

Do you mean a multi-stage Docker build? If so, you should be able to do that without a Linux machine.

[jiawei397]

Of course, I can try docker's as builder, but I don't have a suitable Linux machine to test for the time being.

Do you mean a multi-stage Docker build? If so, you should be able to do that without a Linux machine.

Yes, my Linux server can't upgrade docker temporarily, so I can't use this function.

I tried with MBP. It may be the optimization relationship of M1 pro, and the statistical memory occupation is more exaggerated. This makes me more confused.

CONTAINER ID   NAME                             CPU %     MEM USAGE / LIMIT     MEM %     NET I/O       BLOCK I/O   PIDS
1f4c93753884   deno_node_docker-oak_compile-1   0.04%     118MiB / 7.667GiB     1.50%     1.02kB / 0B   0B / 0B     7
f7fbd7ce25e1   deno_node_docker-node-1          0.00%     7.562MiB / 7.667GiB   0.10%     1.09kB / 0B   0B / 0B     7
17e462a92932   deno_node_docker-oakjs-1         0.02%     125.5MiB / 7.667GiB   1.60%     1.02kB / 0B   0B / 0B     7
86ae2b7448d9   deno_node_docker-koa-1           0.00%     10.43MiB / 7.667GiB   0.13%     1.02kB / 0B   0B / 0B     7
5f27d980543c   deno_node_docker-deno-1          0.02%     112MiB / 7.667GiB     1.43%     1.02kB / 0B   0B / 0B     7
fdcfc9d30df5   deno_node_docker-oak-1           0.01%     143.9MiB / 7.667GiB   1.83%     1.02kB / 0B   0B / 0B     7

Here I write the dockerfile file in this way:

FROM denoland/deno:alpine-1.23.2 as builder

WORKDIR /app

COPY . .

RUN deno compile --allow-net -o oak mod.ts \
    && chmod +x oak

FROM denoland/deno:alpine-1.23.2

WORKDIR /app

COPY --from=builder  /app/oak /app

CMD ./oak

If the from in line 10 is modified to alpine:latest, an error is reported when running:

qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory

I may need to find a minimum image that contains this file.

[arnaudbzn]

@jiawei397 This multi-stage build Dockerfile is working (deployment tested with flyio). You have to use the same alpine variant (frolvlad/alpine-glibc:alpine-3.17 ) as deno base image. The image size is 148 MB. The memory usage displayed in flyio is 58 MB.

ARG DENO_VERSION=1.36.3
FROM denoland/deno:alpine-${DENO_VERSION} AS builder

WORKDIR /app

# Cache the dependencies as a layer
COPY deno.json .

# These steps will be re-run upon each file change in your working directory:
ADD . .

# Compile the main app
RUN deno compile --allow-net --allow-read --allow-env main.ts -o main 

# Use a Docker multi-stage build to create a lean production image.
# Use the same alpine image as deno one because of glibc requirement.
FROM frolvlad/alpine-glibc:alpine-3.17 as runner

WORKDIR /app

# Prefer not to run as root.
RUN addgroup --gid 1000 deno \
    && adduser --uid 1000 --disabled-password deno --ingroup deno \
    && chown deno:deno /app/
USER deno

COPY --from=builder /app/main .

# The port that your application listens to.
EXPOSE 8000

CMD ["./main"]

[jiawei397]

Good job! I've written a simple Oak example, and it works perfectly. It's fantastic However, my project uses a lot of decorators, and it doesn't run after compilation. The bugs in Deno's bundle and compile have not been fixed, which is quite unfortunate.

[arnaudbzn]

That's a pity about the issue with the decorators. Hopefully, it will be resolved in the next Deno release 🤞.

[jiawei397]

Update by 2023-09-16, Deno 1.36.2.

This time, I added the Docker comipile builder.

deno_node_docker-deno-1                 | heapTotal 8.52MB，heapUsed 6.64MB，rss 30.17MB，external:0.07MB
deno_node_docker-oak_compile_builder-1  | heapTotal 9.96MB，heapUsed 7.85MB，rss 32.70MB，external:0.07MB
deno_node_docker-koa-1                  | heapTotal 7.51MB，heapUsed 5.90MB，rss 24.87MB，external:0.81MB
deno_node_docker-node-1                 | heapTotal 5.00MB，heapUsed 4.51MB，rss 18.88MB，external:0.55MB
deno_node_docker-oak-1                  | heapTotal 9.96MB，heapUsed 7.84MB，rss 47.28MB，external:0.07MB
deno_node_docker-oakjs-1                | heapTotal 9.41MB，heapUsed 7.94MB，rss 36.78MB，external:0.07MB

Below is docker stats:

CONTAINER ID   NAME                                     CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O         PIDS
0c68c6a5bed1   deno_node_docker-deno-1                  0.00%     11.09MiB / 15.67GiB   0.07%     7.79kB / 2kB      0B / 305kB        9
6d9fa0470f41   deno_node_docker-oak-1                   0.00%     25.21MiB / 15.67GiB   0.16%     835kB / 31.8kB    0B / 13.5MB       9
3de09fa86a55   deno_node_docker-oak_compile_builder-1   0.00%     13.97MiB / 15.67GiB   0.09%     126B / 0B         0B / 0B           8
016485ff328c   deno_node_docker-koa-1                   0.00%     9.414MiB / 15.67GiB   0.06%     126B / 0B         0B / 0B           7
29a4d092116d   deno_node_docker-node-1                  0.00%     7.18MiB / 15.67GiB    0.04%     126B / 0B         0B / 0B           7
e321aeecc851   deno_node_docker-oakjs-1                 0.00%     16.09MiB / 15.67GiB   0.10%     7.79kB / 1.93kB   0B / 594kB        9

It seems that the effect after compiling is better than the JS version, and even better than the TS version.