search

milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
29.78k stars 2.86k forks source link

[Bug]: Unable to create milvus image from docker dev container #33523

Closed roy-akash closed 2 months ago

roy-akash commented 4 months ago

Is there an existing issue for this?

Environment

- Milvus version: master commit - a26d6cd
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Using the documentation mentioned in this link we are trying to build milvus for amd machines.

We are using macbook for our developing features in milvus. So far we are able to build the milvus codebase with our changes and test these changes in local stand alone as well. Now we wish to create a image for amd machine type and test it on AWS eks kubernetes.

After running the make milvus command on docker container we are facing errors : ConanException: Error 2 while executing cmake --build "/home/milvus/.conan/data/aws-sdk-cpp/1.9.234/_/_/build/3c0c7713bd4c3ddc274da74498cc1d0516a06525/build/Release" '--' '-j4'

Expected Behavior

Milvus make should be successful in the official dev container image

Steps To Reproduce

1. Create dev container in docker
2. run `make milvus`

Milvus Log

`[ 71%] Building CXX object aws-cpp-sdk-sqs/CMakeFiles/aws-cpp-sdk-sqs.dir/ub_SQS.cpp.o c++: fatal error: Killed signal terminated program cc1plus compilation terminated. make[3]: [aws-cpp-sdk-config/CMakeFiles/aws-cpp-sdk-config.dir/build.make:77: aws-cpp-sdk-config/CMakeFiles/aws-cpp-sdk-config.dir/ubCONFIG.cpp.o] Error 1 make[3]: Leaving directory '/home/milvus/.conan/data/aws-sdk-cpp/1.9.234//_/build/3c0c7713bd4c3ddc274da74498cc1d0516a06525/build/Release' make[2]: [CMakeFiles/Makefile2:395: aws-cpp-sdk-config/CMakeFiles/aws-cpp-sdk-config.dir/all] Error 2 make[2]: *** Waiting for unfinished jobs.... [ 72%] Linking CXX static library libaws-cpp-sdk-sqs.a make[3]: Leaving directory '/home/milvus/.conan/data/aws-sdk-cpp/1.9.234///build/3c0c7713bd4c3ddc274da74498cc1d0516a06525/build/Release' [ 72%] Built target aws-cpp-sdk-sqs [ 73%] Linking CXX static library libaws-cpp-sdk-monitoring.a make[3]: Leaving directory '/home/milvus/.conan/data/aws-sdk-cpp/1.9.234///build/3c0c7713bd4c3ddc274da74498cc1d0516a06525/build/Release' [ 73%] Built target aws-cpp-sdk-monitoring

[ 74%] Linking CXX static library libaws-cpp-sdk-iam.a make[3]: Leaving directory '/home/milvus/.conan/data/aws-sdk-cpp/1.9.234///build/3c0c7713bd4c3ddc274da74498cc1d0516a06525/build/Release' [ 74%] Built target aws-cpp-sdk-iam make[2]: Leaving directory '/home/milvus/.conan/data/aws-sdk-cpp/1.9.234///build/3c0c7713bd4c3ddc274da74498cc1d0516a06525/build/Release' make[1]: [Makefile:136: all] Error 2 make[1]: Leaving directory '/home/milvus/.conan/data/aws-sdk-cpp/1.9.234///build/3c0c7713bd4c3ddc274da74498cc1d0516a06525/build/Release' aws-sdk-cpp/1.9.234: aws-sdk-cpp/1.9.234: ERROR: Package '3c0c7713bd4c3ddc274da74498cc1d0516a06525' build failed aws-sdk-cpp/1.9.234: WARN: Build folder /home/milvus/.conan/data/aws-sdk-cpp/1.9.234///build/3c0c7713bd4c3ddc274da74498cc1d0516a06525/build/Release ERROR: aws-sdk-cpp/1.9.234: Error in build() method, line 441 cmake.build() ConanException: Error 2 while executing cmake --build "/home/milvus/.conan/data/aws-sdk-cpp/1.9.234///build/3c0c7713bd4c3ddc274da74498cc1d0516a06525/build/Release" '--' '-j4' conan install failed make: [Makefile:218: build-3rdparty] Error 1`

Anything else?

We were intially facing issues like "too many files open : bad file descriptor" this we were able to resolve using gRPC FUSE instead of virtioFS in docker application in macbook

roy-akash commented 4 months ago

another thing to notice is, if i manually run the cmake command using --j 1 then it succeeds

cmake --build "/home/milvus/.conan/data/aws-sdk-cpp/1.9.234/_/_/build/3c0c7713bd4c3ddc274da74498cc1d0516a06525/build/Release" '--' '-j1'

roy-akash commented 4 months ago

how do i set this variable for this specific package ?

SimFG commented 4 months ago

@roy-akash After you compile manually, are you still unable to execute make milvus? Before executing make, did you execute the ./scripts/install_deps.sh script?

roy-akash commented 4 months ago

nope make milvus still fails as it tries to build this package again with -j 4 no i did not try ./scripts/install_deps.sh yet, let me try this .

roy-akash commented 4 months ago

this is the output for install_deps.sh `root@42c1314c7314:/go/src/github.com/milvus-io/milvus# ./scripts/install_deps.sh ./scripts/install_deps.sh: line 22: sudo: command not found ./scripts/install_deps.sh: line 27: sudo: command not found cmake version is 3.27 cargo exists info: syncing channel updates for '1.73-x86_64-unknown-linux-gnu'

1.73-x86_64-unknown-linux-gnu unchanged - rustc 1.73.0 (cc66ad468 2023-10-03)

info: checking for self-update info: using existing install for '1.73-x86_64-unknown-linux-gnu' info: default toolchain set to '1.73-x86_64-unknown-linux-gnu'

1.73-x86_64-unknown-linux-gnu unchanged - rustc 1.73.0 (cc66ad468 2023-10-03)`

alexanderguzhva commented 4 months ago

this means that the install_deps.sh does not install sudo from repository. It needs to be added there.

roy-akash commented 4 months ago

is sudo mandatory for this to work ?

roy-akash commented 4 months ago

btw same error after running this script [100%] Linking CXX shared library ../../gnu_9.4_cxx11_64_release/libtbb.so make[4]: *** write jobserver: Bad file descriptor. Stop. make[4]: *** Waiting for unfinished jobs.... make[4]: *** write jobserver: Bad file descriptor. Stop. lto-wrapper: fatal error: make returned 2 exit status compilation terminated. /usr/bin/ld: error: lto-wrapper failed collect2: error: ld returned 1 exit status make[3]: *** [src/tbb/CMakeFiles/tbb.dir/build.make:530: gnu_9.4_cxx11_64_release/libtbb.so.12.9] Error 1 make[3]: Leaving directory '/root/.conan/data/onetbb/2021.9.0/_/_/build/2d40c058a87ec4fe9054f14169fd22dc815d0b62/build/Release' make[2]: *** [CMakeFiles/Makefile2:134: src/tbb/CMakeFiles/tbb.dir/all] Error 2 make[2]: Leaving directory '/root/.conan/data/onetbb/2021.9.0/_/_/build/2d40c058a87ec4fe9054f14169fd22dc815d0b62/build/Release' make[1]: *** [Makefile:156: all] Error 2 make[1]: Leaving directory '/root/.conan/data/onetbb/2021.9.0/_/_/build/2d40c058a87ec4fe9054f14169fd22dc815d0b62/build/Release' onetbb/2021.9.0: onetbb/2021.9.0: ERROR: Package '2d40c058a87ec4fe9054f14169fd22dc815d0b62' build failed onetbb/2021.9.0: WARN: Build folder /root/.conan/data/onetbb/2021.9.0/_/_/build/2d40c058a87ec4fe9054f14169fd22dc815d0b62/build/Release ERROR: onetbb/2021.9.0: Error in build() method, line 143 cmake.build() ConanException: Error 2 while executing cmake --build "/root/.conan/data/onetbb/2021.9.0/_/_/build/2d40c058a87ec4fe9054f14169fd22dc815d0b62/build/Release" '--' '-j4' conan install failed make: *** [Makefile:218: build-3rdparty] Error 1

i ran make milvus with and and without sudo and got same error

SimFG commented 4 months ago

Looking at the current error, it seems to be caused by the c++ environment. In addition, the current error is actually different. Now it is a failure to compile onetbb, and it was aws-sdk-cpp before. I'm not sure if it will work. You can try to delete the /root/.conan/data/onetbb directory and make again. This may be caused by the wrong package being downloaded before.

yanliang567 commented 4 months ago

/assign @roy-akash please try as comments above

/unassign

roy-akash commented 4 months ago

I have tried the required suggestion the error is same. Is it possible for you guys to give it a try at your end ?

If someone in your team has a macbook please try the steps related to dev container here : https://github.com/milvus-io/milvus/blob/master/build/README.md#dev-containers

yellow-shine commented 4 months ago

@roy-akash I noticed you mentioned wanting to create an image for an AMD machine type. Could you please confirm if you are referring to the AMD64 architecture?

I followed the documentation available at Milvus Build Guide and successfully built Milvus on Ubuntu 20.04 with the AMD64 architecture without any exceptions.

roy-akash commented 4 months ago

yes i want to build the image for AMD machine type.

Are you using macbook ? whats your docker version ?

yellow-shine commented 4 months ago

yes i want to build the image for AMD machine type.

Are you using macbook ? whats your docker version ?

@roy-akash , Yes, I am being using a MacBook to compile from scratch (without any Conan package cache), and it will take about an hour to complete. So far, everything has been working well. I will inform you if any exceptions occur. My MacBook is running MacOS 12 (M1).

By the way, one of the Milvus CI environments is also based on MacOS 12 (AMD64 architecture), ensuring that code compilation works smoothly on MacOS.

yellow-shine commented 4 months ago

@roy-akash I finished compiling milvus from scratch. everything looks good to me. As I mentioned early, my MacBook is running MacOS 12 (M1).

yellow-shine commented 4 months ago

@roy-akash

I revisited your message again. The log you shared indicates that the compilation process was interrupted due to a "Killed signal terminated program cc1plus" error. This typically occurs when the system runs out of memory during the compilation process.

Here are some steps to potentially resolve this issue:

  1. Increase Available Memory: Ensure that your system has enough free memory available. You might need to close other applications or processes that are consuming significant amounts of memory.
  2. Adjust Compiler Flags: If increasing memory isn't feasible, consider adjusting the compiler flags to reduce memory usage. This make sense why it succeeds when you manually run the cmake command using --j 1.

Please let me know if these steps help or if you need further assistance.

roy-akash commented 4 months ago

@samhuang-z are you using docker environment in macbook to build the amd64 image or are you building milvus on macos directly ?

tha latter works for us currently but we intend to build a amd64 image from macos using dockerized build environment.

roy-akash commented 4 months ago

I reran the build in docker container on my mac, this time i had increased docker resources and container resources both. the container is running with 6 cores and 16 GB memory

I cleaned the conan cache using below command : conan remove "*" -s -b -f

following is the error i got [ 13%] Building CXX object thirdparty/knowhere/knowhere-build/CMakeFiles/diskann.dir/thirdparty/DiskANN/src/logger.cpp.o [ 13%] Building CXX object thirdparty/knowhere/knowhere-build/CMakeFiles/diskann.dir/thirdparty/DiskANN/src/utils.cpp.o make[3]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' [ 23%] Built target milvus-storage [ 24%] Linking CXX static library ../../../lib/libdiskann.a make[3]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' [ 24%] Built target milvus_proto make[3]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' [ 24%] Built target diskann make[2]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' make[1]: *** [Makefile:146: all] Error 2 make[1]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' make: *** [Makefile:234: build-cpp] Error 1

My OS version is 14.4.1 and machine is M1 Max

yellow-shine commented 4 months ago

@samhuang-z are you using docker environment in macbook to build the amd64 image or are you building milvus on macos directly ?

tha latter works for us currently but we intend to build a amd64 image from macos using dockerized build environment.

@roy-akash

  1. I am using a Docker environment. Since my MacBook runs on an M1 chip, the target architecture for building Milvus by default is ARM64.
  2. Given that compiling Milvus is both CPU- and memory-intensive, I allocate as many compute resources as possible. In my case, I provide 10 CPU cores and 32GB of memory.
yellow-shine commented 4 months ago

[ 13%] Building CXX object thirdparty/knowhere/knowhere-build/CMakeFiles/diskann.dir/thirdparty/DiskANN/src/logger.cpp.o [ 13%] Building CXX object thirdparty/knowhere/knowhere-build/CMakeFiles/diskann.dir/thirdparty/DiskANN/src/utils.cpp.o make[3]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' [ 23%] Built target milvus-storage [ 24%] Linking CXX static library ../../../lib/libdiskann.a make[3]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' [ 24%] Built target milvus_proto make[3]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' [ 24%] Built target diskann make[2]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' make[1]: [Makefile:146: all] Error 2 make[1]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' make: [Makefile:234: build-cpp] Error 1

  1. Yeah, 6 cores and 16GB of memory should be sufficient, although it is a little bit on the lower end for such intensive tasks. It’s good to hear that the error you mentioned previously is now resolved.

  2. Unfortunately, the message you provided doesn't give specific details about why the compilation is failing.

To help identify the exact cause of the CPP error, please follow these steps:

roy-akash commented 4 months ago

following is a bit more on the stack trace

make[4]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build/3rdparty_download/jemalloc-subbuild/src/jemalloc_ep' [ 62%] Performing install step for 'jemalloc_ep' make[4]: Entering directory '/go/src/github.com/milvus-io/milvus/cmake_build/3rdparty_download/jemalloc-subbuild/src/jemalloc_ep' /usr/bin/install -c -m 755 bin/jemalloc-config /go/src/github.com/milvus-io/milvus/internal/core/output/bin /usr/bin/install -c -m 644 include/jemalloc/jemalloc.h /go/src/github.com/milvus-io/milvus/internal/core/output/include/jemalloc Missing xsltproc. doc/jemalloc.html not (re)built. make[4]: *** write jobserver: Bad file descriptor. Stop. make[4]: *** Waiting for unfinished jobs.... Missing xsltproc. doc/jemalloc.3 not (re)built. make[4]: *** write jobserver: Bad file descriptor. Stop. make[3]: *** [thirdparty/jemalloc/CMakeFiles/jemalloc_ep.dir/build.make:106: 3rdparty_download/jemalloc-subbuild/src/jemalloc_ep-stamp/jemalloc_ep-install] Error 2 make[3]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' make[2]: *** [CMakeFiles/Makefile2:1886: thirdparty/jemalloc/CMakeFiles/jemalloc_ep.dir/all] Error 2 make[2]: *** Waiting for unfinished jobs.... /usr/bin/install -c -m 644 jemalloc.pc /go/src/github.com/milvus-io/milvus/internal/core/output/lib/pkgconfig /usr/bin/install -c -m 755 lib/libjemalloc.a /go/src/github.com/milvus-io/milvus/internal/core/output/lib /usr/bin/install -c -m 755 bin/jemalloc.sh /go/src/github.com/milvus-io/milvus/internal/core/output/bin /usr/bin/install -c -m 755 bin/jeprof /go/src/github.com/milvus-io/milvus/internal/core/output/bin /usr/bin/install -c -m 755 lib/libjemalloc_pic.a /go/src/github.com/milvus-io/milvus/internal/core/output/lib warning: unused import:futures::executor::block_on` --> src/index_writer.rs:1:5 1 use futures::executor::block_on; ^^^^^^^^^^^^^^^^^^^^^^^^^^^

= note: #[warn(unused_imports)] on by default

warning: unused import: IndexWriter --> src/index_writer.rs:4:38 | 4 | use tantivy::{doc, tokenizer, Index, IndexWriter, SingleSegmentIndexWriter}; | ^^^^^^^^^^^

warning: unused import: crate::index_writer --> src/index_writer.rs:7:5 | 7 | use crate::index_writer; | ^^^^^^^^^^^^^^^^^^^

warning: variable does not need to be mutable --> src/index_writer.rs:101:19 101 pub fn finish(mut self) { ----^^^^
help: remove this mut 
= note: `#[warn(unused_mut)]` on by default
warning: variants Keyword, I64, F64, and Bool are never constructed --> src/data_type.rs:4:5 2 pub enum TantivyDataType { --------------- variants in this enum 3 // Text, 4 Keyword, ^^^^^^^ 5 // U64, 6 I64, ^^^ 7 F64, ^^^ 8 Bool, ^^^^

= note: #[warn(dead_code)] on by default

warning: tantivy-binding (lib) generated 5 warnings (run cargo fix --lib -p tantivy-binding to apply 4 suggestions) Finished release [optimized] target(s) in 6m 31s make[3]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' [ 62%] Built target tantivy_binding_target make[3]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' [ 62%] Built target knowhere make[2]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' make[1]: [Makefile:146: all] Error 2 make[1]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' make: [Makefile:234: build-cpp] Error 1 root@2b89ede6cb8e:/go/src/github.com/milvus-io/milvus# root@2b89ede6cb8e:/go/src/github.com/milvus-io/milvus#`

roy-akash commented 4 months ago

I have increased resources for the container to 10 cores and 32 gigs memory.

following is the order of commands i am running :

  1. apt-get update
  2. apt-get install sudo
  3. ./scripts/install_deps.sh
  4. make clean
  5. make milvus

let me see how it works, also i did not do conan cache remove this time. is that mandatory ?

It would be great if you could share your order of commands in docker env.

roy-akash commented 4 months ago

i got the following error

/usr/bin/ar:u' modifier ignored since D' is the default (seeU') /usr/bin/ar: u' modifier ignored sinceD' is the default (see U') make[4]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build/3rdparty_download/jemalloc-subbuild/src/jemalloc_ep' [ 62%] Performing install step for 'jemalloc_ep' make[4]: Entering directory '/go/src/github.com/milvus-io/milvus/cmake_build/3rdparty_download/jemalloc-subbuild/src/jemalloc_ep' Missing xsltproc. doc/jemalloc.3 not (re)built. make[4]: *** write jobserver: Bad file descriptor. Stop. make[4]: *** Waiting for unfinished jobs.... /usr/bin/install -c -m 755 bin/jemalloc-config /go/src/github.com/milvus-io/milvus/internal/core/output/bin /usr/bin/install -c -m 644 include/jemalloc/jemalloc.h /go/src/github.com/milvus-io/milvus/internal/core/output/include/jemalloc Missing xsltproc. doc/jemalloc.html not (re)built. make[4]: *** write jobserver: Bad file descriptor. Stop. make[3]: *** [thirdparty/jemalloc/CMakeFiles/jemalloc_ep.dir/build.make:106: 3rdparty_download/jemalloc-subbuild/src/jemalloc_ep-stamp/jemalloc_ep-install] Error 2 make[3]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' make[2]: *** [CMakeFiles/Makefile2:1886: thirdparty/jemalloc/CMakeFiles/jemalloc_ep.dir/all] Error 2 make[2]: *** Waiting for unfinished jobs.... /usr/bin/install -c -m 755 lib/libjemalloc.a /go/src/github.com/milvus-io/milvus/internal/core/output/lib /usr/bin/install -c -m 755 bin/jemalloc.sh /go/src/github.com/milvus-io/milvus/internal/core/output/bin /usr/bin/install -c -m 755 bin/jeprof /go/src/github.com/milvus-io/milvus/internal/core/output/bin /usr/bin/install -c -m 755 lib/libjemalloc_pic.a /go/src/github.com/milvus-io/milvus/internal/core/output/lib make[3]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' [ 62%] Built target knowhere warning: unused import:futures::executor::block_on` --> src/index_writer.rs:1:5 1 use futures::executor::block_on; ^^^^^^^^^^^^^^^^^^^^^^^^^^^

= note: #[warn(unused_imports)] on by default

warning: unused import: IndexWriter --> src/index_writer.rs:4:38 | 4 | use tantivy::{doc, tokenizer, Index, IndexWriter, SingleSegmentIndexWriter}; | ^^^^^^^^^^^

warning: unused import: crate::index_writer --> src/index_writer.rs:7:5 | 7 | use crate::index_writer; | ^^^^^^^^^^^^^^^^^^^

warning: variable does not need to be mutable --> src/index_writer.rs:101:19 101 pub fn finish(mut self) { ----^^^^
help: remove this mut 
= note: `#[warn(unused_mut)]` on by default
warning: variants Keyword, I64, F64, and Bool are never constructed --> src/data_type.rs:4:5 2 pub enum TantivyDataType { --------------- variants in this enum 3 // Text, 4 Keyword, ^^^^^^^ 5 // U64, 6 I64, ^^^ 7 F64, ^^^ 8 Bool, ^^^^

= note: #[warn(dead_code)] on by default

warning: tantivy-binding (lib) generated 5 warnings (run cargo fix --lib -p tantivy-binding to apply 4 suggestions) Finished release [optimized] target(s) in 5m 58s make[3]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' [ 62%] Built target tantivy_binding_target make[2]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' make[1]: [Makefile:146: all] Error 2 make[1]: Leaving directory '/go/src/github.com/milvus-io/milvus/cmake_build' make: [Makefile:234: build-cpp] Error 1`

yellow-shine commented 4 months ago

@roy-akash The stack trace you provided is still not detailed enough to diagnose the issue fully. However, I suspect that your Conan cache might be causing the problem. Please try the following steps:

If the issue persists, please provide full compiling logs, and we will continue troubleshooting from there.

roy-akash commented 4 months ago

after i delete the .docker directory i get the following error.

root@c48b8c4c6576:/go/src/github.com/milvus-io/milvus# make milvus Download milvus-proto repo ... Build 3rdparty ... WARN: Cannot find /go/src/github.com/milvus-io/milvus/internal/core/output/lib/libjemalloc.so /go/src/github.com/milvus-io/milvus/cmake_build /go/src/github.com/milvus-io/milvus ERROR: Can't write version file in '/home/milvus/.conan/version.txt': [Errno 2] No such file or directory: '/home/milvus/.conan/version.txt' ERROR: Can't write version file in '/home/milvus/.conan/version.txt': [Errno 2] No such file or directory: '/home/milvus/.conan/version.txt' Running on ubuntu20.04 ERROR: Can't write version file in '/home/milvus/.conan/version.txt': [Errno 2] No such file or directory: '/home/milvus/.conan/version.txt' conan install failed make: *** [Makefile:218: build-3rdparty] Error 1

roy-akash commented 4 months ago

if i try to create the directory i see the following statements

root@c48b8c4c6576:/go/src/github.com/milvus-io/milvus# mkdir -p /home/milvus/.conan/ mkdir: cannot create directory '/home/milvus/.conan/': File exists root@c48b8c4c6576:/go/src/github.com/milvus-io/milvus# cd /home/milvus/.conan/ cd: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory

btw the .docker container had following stuff root@c48b8c4c6576:/go/src/github.com/milvus-io/milvus/.docker# ls amd64-ubuntu20.04-ccache amd64-ubuntu20.04-conan amd64-ubuntu20.04-go-mod amd64-ubuntu20.04-vscode-extensions

yellow-shine commented 4 months ago

if i try to create the directory i see the following statements

root@c48b8c4c6576:/go/src/github.com/milvus-io/milvus# mkdir -p /home/milvus/.conan/ mkdir: cannot create directory '/home/milvus/.conan/': File exists root@c48b8c4c6576:/go/src/github.com/milvus-io/milvus# cd /home/milvus/.conan/ cd: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory

btw the .docker container had following stuff root@c48b8c4c6576:/go/src/github.com/milvus-io/milvus/.docker# ls amd64-ubuntu20.04-ccache amd64-ubuntu20.04-conan amd64-ubuntu20.04-go-mod amd64-ubuntu20.04-vscode-extensions

Well~

  1. first stop container via ./scripts/devcontainer.sh down
  2. then start container ./scripts/devcontainer.sh up
  3. come to the container, docker exec -ti milvus-builder-1 bash
  4. build milvus with make milvus
roy-akash commented 4 months ago

hey i gave it a try as you have instructed , following is the entire stack trace

milvus-build-1.txt

roy-akash commented 4 months ago

@samhuang-z would it possible to connect on a live debugging session for this ? i can share the corrective steps in this thread later.

Presburger commented 4 months ago

@roy-akash hi, try 4C32G container for the compile task.

roy-akash commented 4 months ago

I am using 10 cores and 32 GB memory for this. Do u want me to reduce the resources for the container to 4 cores and 32 GB ?

Presburger commented 4 months ago

I am using 10 cores and 32 GB memory for this. Do u want me to reduce the resources for the container to 4 cores and 32 GB ?

Yes

roy-akash commented 4 months ago

decreasing the resources did not work, what worked for me is :

  1. turn off this option in docker settings -> Use Rosetta for x86/amd64 emulation on Apple Silicon
  2. use gRPC FUSE for file sharing implementation in docker settings
roy-akash commented 4 months ago

@Presburger @samhuang-z finally now that milvus binary is ready, can you tell me how to create a image out of it ?

roy-akash commented 4 months ago

nevermind i followed the steps mentioned here : https://github.com/milvus-io/milvus/blob/master/DEVELOPMENT.md#build-image

let me test this image and get back

stale[bot] commented 3 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

sanjayk-github-dev commented 2 months ago

Hi, I'm also running into a similar issue on my M1 Mac. I ran the steps mentioned by @yellow-shine in one of the earlier comments.

Below is the tail output of make milvus before it fails. I would appreciate any pointers to fix this issue.

In member function ‘store’,
    inlined from ‘store’ at /usr/include/c++/12/atomic:104:20,
    inlined from ‘notify_one_relaxed’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/concurrent_monitor.h:293:53,
    inlined from ‘notify_one_relaxed’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/concurrent_monitor.h:280:10,
    inlined from ‘notify_one’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/concurrent_monitor.h:276:27,
    inlined from ‘execute’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/arena.cpp:669:47:
/usr/include/c++/12/bits/atomic_base.h:464:25: warning: ‘__atomic_store_1’ writing 1 byte into a region of size 0 overflows the destination [-Wstringop-overflow=]
  464 |         __atomic_store_n(&_M_i, __i, int(__m));
      |                         ^
make[4]: *** write jobserver: Bad file descriptor.  Stop.
make[4]: *** Waiting for unfinished jobs....
make[4]: *** write jobserver: Bad file descriptor.  Stop.
lto-wrapper: fatal error: make returned 2 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
gmake[3]: *** [src/tbb/CMakeFiles/tbb.dir/build.make:530: gnu_12.3_cxx11_64_release/libtbb.so.12.9] Error 1
gmake[3]: Leaving directory '/home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/build/Release'
gmake[2]: *** [CMakeFiles/Makefile2:134: src/tbb/CMakeFiles/tbb.dir/all] Error 2
gmake[2]: Leaving directory '/home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/build/Release'
gmake[1]: *** [Makefile:156: all] Error 2
gmake[1]: Leaving directory '/home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/build/Release'
onetbb/2021.9.0: 
onetbb/2021.9.0: ERROR: Package '1b60987548fe9206f368ea572a16971878dcb1c6' build failed
onetbb/2021.9.0: WARN: Build folder /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/build/Release
ERROR: onetbb/2021.9.0: Error in build() method, line 143
        cmake.build()
        ConanException: Error 2 while executing cmake --build "/home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/build/Release" '--' '-j10'
In member function ‘store’,
    inlined from ‘store’ at /usr/include/c++/12/atomic:104:20,
    inlined from ‘abort_all_relaxed’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/concurrent_monitor.h:430:53,
    inlined from ‘abort_all’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/concurrent_monitor.h:413:26,
    inlined from ‘destroy’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/concurrent_monitor.h:446:24,
    inlined from ‘__dt_base ’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/market_concurrent_monitor.h:102:16,
    inlined from ‘__dt_base ’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/market.cpp:81:1,
    inlined from ‘destroy’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/market.cpp:169:26,
    inlined from ‘acknowledge_close_connection’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/market.cpp:617:12,
    inlined from ‘remove_server_ref’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/private_server.cpp:172:51,
    inlined from ‘remove_server_ref’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/private_server.cpp:170:10,
    inlined from ‘run’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/private_server.cpp:281:32,
    inlined from ‘thread_routine’ at /home/milvus/.conan/data/onetbb/2021.9.0/_/_/build/1b60987548fe9206f368ea572a16971878dcb1c6/src/src/tbb/private_server.cpp:221:14:
/usr/include/c++/12/bits/atomic_base.h:464:25: warning: ‘__atomic_store_1’ writing 1 byte into a region of size 0 overflows the destination [-Wstringop-overflow=]
  464 |         __atomic_store_n(&_M_i, __i, int(__m));
      |                         ^
conan install failed
make: *** [Makefile:245: build-3rdparty] Error 1