bloomberg / blazingmq

A modern high-performance open source message queuing system
https://bloomberg.github.io/blazingmq/
Apache License 2.0
2.54k stars 132 forks source link

Screen frozen when running docker compose #79

Closed jcalahor closed 1 year ago

jcalahor commented 1 year ago

Is there an existing issue for this?

Current Behavior

docker compose is freezing see below when running. the machine i'm using is a ec2 machine: t2.micro

ubuntu@ip-172-31-85-234:~/development/blazingmq$ docker compose -f docker/single-node/do                                                                                                                      cker-compose.yaml up --build -d
[+] Running 1/1
 ✘ bmqtool Error                                                                   0.1s
[+] Building 92.6s (10/20)
 => [bmqbrkr internal] load build definition from Dockerfile                                                                                                                                             0.0s
 => => transferring dockerfile: 2.13kB                                                                                                                                                                   0.0s
 => [bmqbrkr internal] load .dockerignore                                                                                                                                                                0.0s
 => => transferring context: 2B                                                                                                                                                                          0.0s
 => [bmqbrkr internal] load metadata for docker.io/library/ubuntu:22.04                                                                                                                                  0.2s
 => CACHED [bmqbrkr builder  1/12] FROM docker.io/library/ubuntu:22.04@sha256:0bced47fffa3361afa981854fcabcd4577cd43cebbb808cea2b1f33a3dd7f508                                                           0.0s
 => [bmqbrkr internal] load build context                                                                                                                                                                0.2s
 => => transferring context: 74.84kB                                                                                                                                                                     0.2s
 => CACHED [bmqbrkr builder  2/12] RUN apt update &&     apt -y install ca-certificates                                                                                                                  0.0s
 => CACHED [bmqbrkr builder  3/12] RUN apt install -y --no-install-recommends     build-essential     gdb     curl     python3.10     ninja-build     pkg-config     bison     libfl-dev     libbenchma  0.0s
 => CACHED [bmqbrkr builder  4/12] RUN mkdir -p /deps_build/CMake     && cd /deps_build/CMake     && curl -s -L -O https://github.com/Kitware/CMake/releases/download/v3.27.1/cmake-3.27.1-linux-x86_64  0.0s
 => CACHED [bmqbrkr builder  5/12] WORKDIR /workspace                                                                                                                                                    0.0s
 => CACHED [bmqbrkr builder  6/12] COPY docker/build_deps.sh bin/build_deps.sh                                                                                                                           0.0s
 => [bmqbrkr builder  7/12] RUN bin/build_deps.sh                                                                                                                                                       92.1s
 => => # [487/1488] Generating /workspace/srcs/bde/groups/bsl/bslma/bslma_sharedptrinplacerep_cpp03.h
 => => # [488/1488] Generating /workspace/srcs/bde/groups/bsl/bslma/bslma_stdallocator_cpp03.h
 => => # [489/1488] Generating /workspace/srcs/bde/groups/bsl/bslma/bslma_allocatortraits_cpp03.h
 => => # [490/1488] Generating /workspace/srcs/bde/groups/bsl/bslma/bslma_managedptr_cpp03.h
 => => # [491/1488] Generating /workspace/srcs/bde/groups/bsl/bslma/bslma_constructionutil_cpp03.h
 => => # [492/1488] Building CXX object groups/bsl/CMakeFiles/bslh-iface.dir/bslh/bslh_fibonaccibadhashwrapper.cpp.o

Expected Behavior

container should be able to be created

Steps To Reproduce

1) create a ec2: t2.micro machine 2) ssh from a client like mobaexterm or visual studio code 3) clone the code 4) perform docker compose -f docker/single-node/docker-compose.yaml up --build -d

BlazingMQ Version

1.0

Anything else?

No response

678098 commented 1 year ago

Hi @jcalahor , thank you for the feedback.

It looks like a common memory related problem with ec2 t2.micro: https://stackoverflow.com/questions/63171780/building-python-wheel-via-docker-compose-on-aws-ec2-gets-stuck https://www.reddit.com/r/aws/comments/6gvyoe/docker_container_seems_to_crash_my_ec2_instance/

Could you try to rebuild the image with less parallel jobs? I hope it might help to occupy less RAM during compilation. You need to clone the repo and then change some nums with any text editor:

https://github.com/bloomberg/blazingmq/blob/69f0f8f5b188ca1eb07b9d262093f949729db538/docker/build_deps.sh#L32

https://github.com/bloomberg/blazingmq/blob/69f0f8f5b188ca1eb07b9d262093f949729db538/docker/build_deps.sh#L41

https://github.com/bloomberg/blazingmq/blob/69f0f8f5b188ca1eb07b9d262093f949729db538/docker/Dockerfile#L51

Here you can see 8 - it's a number of parallel jobs, replacing 8->1 in these 3 places might help.

Could you also check what htop shows during compilation? Interesting to see if it's a memory related problem indeed.

jcalahor commented 1 year ago

image

hi i tried reducing to 3 and same error, img as reference. i'm going to try locally ty

jcalahor commented 1 year ago

decided to try locally on my machine, (i don't want to try non free ec2 options), and so far it appears to be working, this issue can be closed thx for follow up