getmetal / motorhead

🧠 Motorhead is a memory and information retrieval server for LLMs.
https://getmetal.io
Apache License 2.0
846 stars 79 forks source link

arm64 container? #44

Closed danielchalef closed 1 year ago

danielchalef commented 1 year ago

Do you have plans for building an arm64 container? It would significantly aid local development with motorhead as backend memory.

Czechh commented 1 year ago

yeah - I don't see why not! Will look into it.

Curious how it would significantly aid local development?

danielchalef commented 1 year ago

When developing an app using Motorhead memory, it would be useful to be able to run the full setup locally rather than using a remote staging Motorhead server. Many of us use MacBooks, hence the arm64 request. It might also be nice to be able to deploy Motorhead to cheap arm64 cloud instances.

On Thu, Apr 27, 2023 at 6:57 AM, Sergio Prada < @.*** > wrote:

yeah - I don't see why not! Will look into it.

Curious how it would significantly aid local development?

— Reply to this email directly, view it on GitHub ( https://github.com/getmetal/motorhead/issues/44#issuecomment-1525744634 ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AABAAZ62BZFOP5CO44M6YI3XDJ3MVANCNFSM6AAAAAAXNFN7HI ). You are receiving this because you authored the thread. Message ID: < @.***

Czechh commented 1 year ago

I develop using an M1 mackbook. And I'm using docker-compose to run Motorhead + Readisearch locally. I haven't tested the hosted image on it's own locally though.

danielchalef commented 1 year ago

Are you successfully using Rosetta emulation? I attempted to do so and got 500 errors on a number of endpoints. I did not dig too deep into what was happening as I assumed emulation was breaking.

On Thu, Apr 27 2023 at 8:51 AM, Sergio Prada < @.*** > wrote:

I develop using an M1 mackbook. And I'm using docker-compose to run Motorhead + Readisearch locally.

— Reply to this email directly, view it on GitHub ( https://github.com/getmetal/motorhead/issues/44#issuecomment-1525937702 ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AABAAZ24IZWBYEK3R47PBQLXDKIWXANCNFSM6AAAAAAXNFN7HI ). You are receiving this because you authored the thread. Message ID: <getmetal/motorhead/issues/44/1525937702 @ github. com>

Czechh commented 1 year ago

Would appreciate to see what you're seeing.

No rosetta needed. Just using docker-compose build && docker-compose up.

The image getting build in GH packages I haven't tested locally. But since you're building the docker image from scratch with the Dockerfile, it should be working for you if you run it in M1 or x86. This becomes an issue if you use the hosted image and it's not compatible, but building it from your computer should work.

So 5xx issues might be something else and would love to look into it.

danielchalef commented 1 year ago

I was using the prebuilt container on the GCR. Building the container (docker-compose build) fails with: tag v1.0.0 / 042b12f3ba2009416122e267fb7e26e2c179dbca

 175.8    Compiling futures v0.3.27
#0 176.0    Compiling actix-router v0.5.1
#0 176.7    Compiling flate2 v1.0.25
#0 177.5    Compiling brotli v3.3.4
#0 206.5 error: could not compile `brotli`
#0 206.5
#0 206.5 Caused by:
#0 206.5   process didn't exit successfully: `rustc --crate-name brotli /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/brotli-3.3.4/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg 'feature="alloc-stdlib"' --cfg 'feature="default"' --cfg 'feature="ffi-api"' --cfg 'feature="std"' -C metadata=5f833f24f65aa478 -C extra-filename=-5f833f24f65aa478 --out-dir /motorhead/target/release/deps -L dependency=/motorhead/target/release/deps --extern alloc_no_stdlib=/motorhead/target/release/deps/liballoc_no_stdlib-cfadf17c4a8b4968.rmeta --extern alloc_stdlib=/motorhead/target/release/deps/liballoc_stdlib-c8463f3a5152792c.rmeta --extern brotli_decompressor=/motorhead/target/release/deps/libbrotli_decompressor-daef67a5eb830569.rmeta --cap-lints allow` (signal: 9, SIGKILL: kill)
#0 206.5 warning: build failed, waiting for other jobs to finish...
------
failed to solve: executor failed running [/bin/sh -c cargo build --release]: exit code: 101
Czechh commented 1 year ago

Ok - I'll get an ARM64 image out soon. Thanks for the feedback!

danielchalef commented 1 year ago

Which container are you running on your MacBook M1?

On Thu, Apr 27 2023 at 11:38 AM, Sergio Prada < @.*** > wrote:

Ok - I'll get an ARM64 image out soon. Thanks for the feedback!

— Reply to this email directly, view it on GitHub ( https://github.com/getmetal/motorhead/issues/44#issuecomment-1526152049 ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AABAAZ27H3Z4YSENK7IL3DDXDK4J3ANCNFSM6AAAAAAXNFN7HI ). You are receiving this because you authored the thread. Message ID: <getmetal/motorhead/issues/44/1526152049 @ github. com>

Czechh commented 1 year ago

I'm using the projects docker-compose you can replicate by cloning this repo and running docker-compose build && docker-compose up.

danielchalef commented 1 year ago

For some reason, compiling using the repo Dockerfile failed. See above.

Czechh commented 1 year ago

Just used this docker-compose and it worked.

version: '3'
services:
  motorhead:
    image: ghcr.io/getmetal/motorhead:v1.0.1
    ports:
      - '8080:8080'
    links:
      - redis
    environment:
      PORT: 8080
      MOTORHEAD_LONG_TERM_MEMORY: 'true'
      REDIS_URL: 'redis://redis:6379'
    env_file:
      - .env

  redis:
    image: redis/redis-stack-server:latest
    ports:
      - '6379:6379'

Here with a M1 mac - not sure why this isn't working for you 🤔 - Here I didn't have to build any images.

platput commented 1 year ago

I am using M1 mac and it is working fine. I am running the docker containers on an Ubuntu arm64 VM. I used the docker-compose as well.

Czechh commented 1 year ago

I'll close this for now but let me know if it's still an issue @danielchalef