milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
30.49k stars 2.92k forks source link

[Feature]: Please provide the arm64 dockerfile, we need to modify the container internal instructions【Kylin V10】 #27726

Open ZhangErling opened 1 year ago

ZhangErling commented 1 year ago

Is there an existing issue for this?

Is your feature request related to a problem? Please describe.

When I start the container with the following image, an exception is thrown:Unsupported system page size 7e8469f2a2db4cae5d73c743947361e docker pull milvusdb/milvus:2.3.0-latest linux/arm64/v8

OS:Kylin V10 SP1 Architecture:aarch64

PS: Kylin V10 ≈ CentOS 8

Describe the solution you'd like.

Add ENV USE_JEMALLOC no to the RUN make xxxx directive of the built Dockerfile so that redis is compiled without jemalloc. https://github.com/jemalloc/jemalloc/issues/467

Describe an alternate solution.

No response

Anything else? (Additional Context)

No response

xiaofan-luan commented 1 year ago

@ZhangErling so you are saying Kylin OS don't support jemalloc right?

ZhangErling commented 1 year ago

@ZhangErling so you are saying Kylin OS don't support jemalloc right?

I'm sorry. I don't know much about Kylin

This is what I think is a very credible source, and it says: In the arm64 environment, the centos pagesize is 64k, ubuntu pagesize is 4k, generally speaking, the image compiled under 64k can run in the environment less than or equal to 64k, but if the image compiled under 4k, It will not work with a pagesize greater than 4k. This is caused by jemalloc. You don't have this problem if you use libc when compiling build images.

https://bbs.huaweicloud.com/forum/thread-47123-1-1.html

While Kylin pagesize is 64k, milvus mirror pagesize is 4k, If he's right, I think that's the problem.

The quoted materials are in Chinese, which may cause your reading difficulties. Sorry, please feel free to contact me if you need any additional materials

xiaofan-luan commented 1 year ago

/assign @LoveEachDay maybe we should offer a rocky linux image?

where did the current tar compiled? ubuntu 22.04?

jiaoew1991 commented 1 year ago

@ZhangErling We already have an option here, but it does not open by default, feel free to compile your own Milvus version to use a larger page size. https://github.com/milvus-io/milvus/blob/84d05b939bedff778aa688062451819cbf17afa1/internal/core/thirdparty/jemalloc/CMakeLists.txt#L40-L44

ZhangErling commented 1 year ago

@ZhangErling We already have an option here, but it does not open by default, feel free to compile your own Milvus version to use a larger page size.

https://github.com/milvus-io/milvus/blob/84d05b939bedff778aa688062451819cbf17afa1/internal/core/thirdparty/jemalloc/CMakeLists.txt#L40-L44

really appreciate your help, We have abandoned the deployment of milvus on Kylin V10 due to our internal evaluation of the current cost of self-compiling milvus. The following information we found on compiling milvus from source code is hopefully useful for people who encounter the same problem

从零开始构建向量数据库:Milvus 的源码编译安装(Linux环境)

xiaohezi205 commented 11 months ago

加我1079361253微信 2.2.12版本arm64镜像免费送

ronghuaihai commented 11 months ago

we have same problem

image image

the page size is

image

Could you provide a image of standalone mivlus v2.3.3 on compliled on arm64 architecture operating system with 65536 pagesize?we will appreciate that then we can pull directly with “docker compose up -d” ,thanks very much。