DuckY-Y
commented
1 week ago Alright this is quite interesting: I've got no idea why I had to use sudo to remove it, nor why the volume breaks it.
tesh@teshtesh:~/echo/marqo$ sudo docker run --restart unless-stopped --name marqo -it -p 8882:8882 -e "MARQO_MAX_DOC_BYT
ES=200000" marqoai/marqo:latest
docker: Error response from daemon: Conflict. The container name "/marqo" is already in use by container "0500beaa9a57f85f73938dceff24bd66422f0f5207f59bebbe6dc9d0f3a4c7ca". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
tesh@teshtesh:~/echo/marqo$ docker stop marqo
Error response from daemon: No such container: marqo
tesh@teshtesh:~/echo/marqo$ docker rm marqo
Error response from daemon: No such container: marqo
tesh@teshtesh:~/echo/marqo$ docker system prune -a
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all images without at least one container associated to them
- all build cache
Are you sure you want to continue? [y/N] y
Deleted Images:
untagged: marqoai/marqo:latest
untagged: marqoai/marqo@sha256:5d10b06ddb62665cc8ff57ffdedacecfad1660ac2a1a4235e87ab1c61a6c99ac
deleted: sha256:7c72260329e6a2e35b4ac15ea43a2f406d73790bb7608fe0a7bb58fbffa62972
deleted: sha256:8fe0b3bb1c3b7bc0036a8b1ad4bc0dd8f291d2db40596e7fd35e41f4def59f3f
deleted: sha256:9d23b2afd5cb9e915a2860f5eb3d9b1941464c60ee5135bc3c4e8eb618753a87
deleted: sha256:c5d97c85aade4862e21f3c87e2a9cb9b2ec919250a2c63a4093196ce0a79866d
deleted: sha256:d14dda496eabe09ed872509375143cd69e6e59146064ced681249a4f9706382a
deleted: sha256:015e8bef8822807d13f9ff1818b7c3fe4abda56cb6e64a78ba126ccd5d1624cf
deleted: sha256:bc85585b10cbc490f0ca3517cd54047352f23631d021339e63e161292b48c24f
deleted: sha256:e0989deaabec72ee6cfb5cae20a048a75cdd1de431ad2fcdb2259e19cb2e443b
deleted: sha256:841d1cdd5c74335c90915a6c86949fb4ee4b62fad7493b46a271d0e0ad3438b7
deleted: sha256:f7f9bcc29f0efbf484bcdd47a2c5df767c0f62e38d18e2211f3d7b3654b5fb0b
deleted: sha256:c277aebc6b7e9f5bb994b6ae77adf309c8b30b9045dfedb4a5565bfefe940691
deleted: sha256:6c34b849aeaeca6858f8afb0cb1b12fd1f90c9abd7f362db7593a2d44e0d940f
deleted: sha256:57e5642d63e332a0c106f2ce09691507e650c14a59bc581d0e36e942f929aebc
deleted: sha256:5054496fea8c678afcb66adb1a46947cf243901a6fc523bb45323ce716ed3d50
deleted: sha256:97bf8faa5fe984fa17fe7bd74f26bb182504bb7a24e411dc957efcebe7c7a645
deleted: sha256:11743e4defd3a6fc2aa8eda0856b5fab270fbc95fff3b91a507b896d8a8c52e6
deleted: sha256:7bd4e6645b5eb8eee49206422d76b0e2cdb01e815c3d1b82fd792c4f5eb10a6b
deleted: sha256:42700eeecb16d2f738af93b12d7e435e3fc7c834d464cf07d91d127a520c2e49
deleted: sha256:3e1520d4703898e2fc04106c2cbe13b9f83b448c58505fbb0aace5790b80a4d2
deleted: sha256:0945b96ec308a61b7293b23dd5016cfccb6463f64d2df670936e303a3401d478
deleted: sha256:a130314df5ced9df85970d096ca4d24d37e2d3c7ee6768b188326632aaef308f
Total reclaimed space: 9.703GB
tesh@teshtesh:~/echo/marqo$ sudo docker run --restart unless-stopped --name marqo -it -p 8882:8882 -e "MARQO_MAX_DOC_BYTES=200000" marqoai/marqo:latest
docker: Error response from daemon: Conflict. The container name "/marqo" is already in use by container "0500beaa9a57f85f73938dceff24bd66422f0f5207f59bebbe6dc9d0f3a4c7ca". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
tesh@teshtesh:~/echo/marqo$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
62d55c0007a6 ghcr.io/open-webui/open-webui:main "bash start.sh" 3 days ago Up 3 days (healthy) 0.0.0.0:3000->8080/tcp, :::3000->8080/tcp open-webui
ba0316eaef61 ollama/ollama:latest "/bin/ollama serve" 3 days ago Up 3 days 0.0.0.0:11434->11434/tcp, :::11434->11434/tcp ollama
ac44cf4e2db8 penpotapp/frontend:latest "/bin/bash /entrypoi…" 4 weeks ago Up 5 days 0.0.0.0:9001->80/tcp, :::9001->80/tcp taiga-docker-penpot-frontend-1
21221a13e849 penpotapp/backend:latest "/bin/bash run.sh" 4 weeks ago Up 5 days
taiga-docker-penpot-backend-1
1666c63232d9 sj26/mailcatcher:latest "mailcatcher --foreg…" 4 weeks ago Up 5 days 1025/tcp, 0.0.0.0:1080->1080/tcp, :::1080->1080/tcp taiga-docker-penpot-mailcatch-1
7fa9a1df8d83 postgres:15 "docker-entrypoint.s…" 4 weeks ago Up 5 days 5432/tcp taiga-docker-penpot-postgres-1
0d115dc00139 redis:7 "docker-entrypoint.s…" 4 weeks ago Up 5 days 6379/tcp taiga-docker-penpot-redis-1
77e3b4d9e251 penpotapp/exporter:latest "node app.js" 4 weeks ago Up 5 days
taiga-docker-penpot-exporter-1
2a9f10c43949 mongo-express "/sbin/tini -- /dock…" 5 weeks ago Up 5 days 0.0.0.0:8081->8081/tcp, :::8081->8081/tcp monger-mongo-express-1
608a840af444 mongo "docker-entrypoint.s…" 5 weeks ago Up 5 days 0.0.0.0:27017->27017/tcp, :::27017->27017/tcp monger-mongo-1
tesh@teshtesh:~/echo/marqo$ docker image list
REPOSITORY TAG IMAGE ID CREATED SIZE
ghcr.io/open-webui/open-webui main 95fa326eca5f 4 days ago 3.56GB
ollama/ollama latest 018439a5a146 5 days ago 465MB
postgres 15 e3e85a8d0b0f 5 weeks ago 440MB
mongo latest ff65a94ec485 7 weeks ago 795MB
mongo-express latest 0eec10060d2f 7 weeks ago 182MB
penpotapp/exporter latest 32028df821b9 2 months ago 1.23GB
penpotapp/frontend latest 4f343eac55bf 2 months ago 268MB
penpotapp/backend latest b05ca261add7 2 months ago 1.1GB
redis 7 9509c4dd19fb 2 months ago 116MB
sj26/mailcatcher latest 7ab5eb1bdf54 10 months ago 95.7MB
tesh@teshtesh:~/echo/marqo$ sudo docker rm marqo
Error response from daemon: cannot remove container "/marqo": container is running: stop the container before removing or force remove
tesh@teshtesh:~/echo/marqo$ sudo docker stop marqo
marqo
tesh@teshtesh:~/echo/marqo$ sudo docker rm marqo
marqo
tesh@teshtesh:~/echo/marqo$ sudo docker run --restart unless-stopped --name marqo -it -p 8882:8882 -e "MARQO_MAX_DOC_BYTES=200000" marqoai/marqo:latest
External vector store not configured. Using local vector store
Waiting for vector store to start
Marqo did not find an existing vector store. Setting up vector store...
Vector store is available. Vector store setup complete
Starting Marqo throttling
Called Marqo throttling start command
Marqo throttling is now running
/usr/local/lib/python3.8/site-packages/transformers/generation_utils.py:24: FutureWarning: Importing `GenerationMixin` from `src/transformers/generation_utils.py` is deprecated and will be removed in Transformers v5. Import as `from transformers import GenerationMixin` instead.
warnings.warn(
INFO:ModelsForStartup:pre-loading ['hf/e5-base-v2', 'open_clip/ViT-B-32/laion2b_s34b_b79k'] onto devices=['cpu']
INFO:marqo.tensor_search.index_meta_cache:Starting index cache refresh thread
###########################################################
###########################################################
###### STARTING DOWNLOAD OF MARQO ARTEFACTS################
###########################################################
###########################################################
INFO:CUDA device summary:Found devices [{'id': -1, 'name': ['cpu']}]
INFO:SetBestAvailableDevice:Best available device set to: cpu
loading for: model_name=hf/e5-base-v2 and properties={'name': 'intfloat/e5-base-v2', 'dimensions': 768, 'tokens': 512, 'type': 'hf', 'model_size': 0.438, 'text_query_prefix': 'query: ', 'text_chunk_prefix': 'passage: ', 'notes': ''}
INFO:marqo.tensor_search.index_meta_cache:Last index cache refresh at 1718612120.0507038
config.json: 100%|█████████████████████████████████████████████████████████████████████| 650/650 [00:00<00:00, 54.2kB/s]
model.safetensors: 100%|█████████████████████████████████████████████████████████████| 438M/438M [00:25<00:00, 17.3MB/s]
tokenizer_config.json: 100%|███████████████████████████████████████████████████████████| 314/314 [00:00<00:00, 44.3kB/s]
vocab.txt: 100%|█████████████████████████████████████████████████████████████████████| 232k/232k [00:00<00:00, 1.19MB/s]
tokenizer.json: 100%|████████████████████████████████████████████████████████████████| 711k/711k [00:00<00:00, 5.86MB/s]
special_tokens_map.json: 100%|█████████████████████████████████████████████████████████| 125/125 [00:00<00:00, 56.6kB/s]
INFO:marqo.s2_inference.s2_inference:loaded hf/e5-base-v2 on device cpu with normalization=True at time=2024-06-17 08:15:19.813664.
INFO:ModelsForStartup:hf/e5-base-v2 cpu run succesfully!
loading for: model_name=open_clip/ViT-B-32/laion2b_s34b_b79k and properties={'name': 'open_clip/ViT-B-32/laion2b_s34b_b79k', 'dimensions': 512, 'note': 'open_clip models', 'type': 'open_clip', 'pretrained': 'laion2b_s34b_b79k'}
open_clip_pytorch_model.bin: 100%|███████████████████████████████████████████████████| 605M/605M [00:23<00:00, 25.3MB/s]
INFO:marqo.s2_inference.s2_inference:loaded open_clip/ViT-B-32/laion2b_s34b_b79k on device cpu with normalization=True at time=2024-06-17 08:15:49.375963.
INFO:ModelsForStartup:open_clip/ViT-B-32/laion2b_s34b_b79k cpu run succesfully!
INFO:ModelsForStartup:0.027287817001342772 for hf/e5-base-v2 and cpu
INFO:ModelsForStartup:0.04055120944976807 for open_clip/ViT-B-32/laion2b_s34b_b79k and cpu
INFO:ModelsForStartup:completed loading models
INFO:marqo.connections:Took 1.476ms to connect to redis and load scripts.
INFO:CachePatchModels:completed prewarming patch models
###########################################################
###########################################################
###### !!COMPLETED SUCCESSFULLY!!! ################
###########################################################
###########################################################
Version: 2.8.1
__ __ ___ _ __ ___ ___ ___ ___ ______ ___ ___ ___ ____ ____ ___ ___ __
| |__| | / _]| | / ] / \ | | | / _] | | / \ | | | / || \ / \ / \ | |
| | | | / [_ | | / / | || _ _ | / [_ | || | | _ _ || o || D ) || || |
| | | || _]| |___ / / | O || \_/ || _] |_| |_|| O | | \_/ || || /| Q || O ||__|
| ` ' || [_ | / \_ | || | || [_ | | | | | | || _ || \| || | __
\ / | || \ || || | || | | | | | | | || | || . \ || || |
\_/\_/ |_____||_____|\____| \___/ |___|___||_____| |__| \___/ |___|___||__|__||__|\_|\__,_| \___/ |__|
_____ _ __ _
|_ _|__ _ __ ___ ___ _ __ ___ ___ __ _ _ __ ___| |__ / _| ___ _ __ | |__ _ _ _ __ ___ __ _ _ __ ___
| |/ _ \ '_ \/ __|/ _ \| '__| / __|/ _ \/ _` | '__/ __| '_ \ | |_ / _ \| '__| | '_ \| | | | '_ ` _ \ / _` | '_ \/ __|
| | __/ | | \__ \ (_) | | \__ \ __/ (_| | | | (__| | | | | _| (_) | | | | | | |_| | | | | | | (_| | | | \__ \
|_|\___|_| |_|___/\___/|_| |___/\___|\__,_|_| \___|_| |_| |_| \___/|_| |_| |_|\__,_|_| |_| |_|\__,_|_| |_|___/
INFO: Started server process [2052]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8882 (Press CTRL+C to quit)
INFO:marqo.tensor_search.index_meta_cache:Last index cache refresh at 1718612180.5601022
papa99do
farshidz
I'm not quite sure how this happened but perhaps a day after I added a bit more than 2k docs to the database I noticed that the docker was not in docker ps, so I booted it back up with the command and now it seems to be stuck on failing to startup the vector store db. Did something in-between updates fuck things up?
Ubuntu 22.04 the usual Anyways see logs appended below \/