search

dragonflydb / dragonfly

A modern replacement for Redis and Memcached
https://www.dragonflydb.io/
Other
25.47k stars 926 forks source link

"An attempt to execute an instruction failed. Exiting..." #3166

Closed perfectra1n closed 3 months ago

perfectra1n commented 4 months ago

Hey there, I don't see any other issues that are this specific issue.

I'm running vanilla Kubernetes on Debian 12 5.10.216-1 (2024-05-03) x86_64 GNU/Linux on Proxmox 8.2.2. However, when I try to run Dragonfly, I get the following output right before it crashes (after making a single query):

2024-05-15 17:28:53.395 I20240516 00:28:53.395092     1 init.cc:70] dragonfly running in opt mode.
2024-05-15 17:28:53.396 I20240516 00:28:53.395637     1 dfly_main.cc:642] Starting dragonfly df-v1.18.0-3dd6c4959c3056d6f007cc4001e0923302a566a2
2024-05-15 17:28:53.396 * Logs will be written to the first available of the following paths:
2024-05-15 17:28:53.396 /tmp/dragonfly.*
2024-05-15 17:28:53.396 ./dragonfly.*
2024-05-15 17:28:53.396 * For the available flags type dragonfly [--help | --helpfull]
2024-05-15 17:28:53.396 * Documentation can be found at: https://www.dragonflydb.io/docs
2024-05-15 17:28:53.396 I20240516 00:28:53.396409     1 dfly_main.cc:705] Max memory limit is: 512.00MiB
2024-05-15 17:28:53.401 I20240516 00:28:53.401196     8 uring_proactor.cc:182] IORing with 1024 entries, allocated 102720 bytes, cq_entries is 2048
2024-05-15 17:28:53.476 I20240516 00:28:53.475750     1 proactor_pool.cc:147] Running 2 io threads
2024-05-15 17:28:53.488 I20240516 00:28:53.488379     1 dfly_main.cc:271] Listening on admin socket any:9999
2024-05-15 17:28:53.491 I20240516 00:28:53.491189     1 server_family.cc:721] Host OS: Linux 5.10.0-29-amd64 x86_64 with 2 threads
2024-05-15 17:28:53.492 I20240516 00:28:53.492147     1 snapshot_storage.cc:108] Load snapshot: Searching for snapshot in directory: "/data"
2024-05-15 17:28:53.493 W20240516 00:28:53.492761     1 server_family.cc:814] Load snapshot: No snapshot found
2024-05-15 17:28:53.524 I20240516 00:28:53.524137     8 listener_interface.cc:101] sock[8] AcceptServer - listening on port 6379
2024-05-15 17:28:53.525 I20240516 00:28:53.525377     9 listener_interface.cc:101] sock[7] AcceptServer - listening on port 9999
2024-05-15 17:28:53.530 I20240516 00:28:53.530208     8 dflycmd.cc:587] Registered replica 10.233.81.15:6379
2024-05-15 17:28:53.556 I20240516 00:28:53.556521     8 dflycmd.cc:317] Started sync with replica 10.233.81.15:6379
2024-05-15 17:28:53.559 I20240516 00:28:53.558974     8 dflycmd.cc:355] Transitioned into stable sync with replica 10.233.81.15:6379
2024-05-15 17:28:53.983 I20240516 00:28:53.983150     9 dflycmd.cc:587] Registered replica 10.233.69.29:6379
2024-05-15 17:28:53.993 I20240516 00:28:53.993358     9 dflycmd.cc:317] Started sync with replica 10.233.69.29:6379
2024-05-15 17:28:53.996 I20240516 00:28:53.996198     9 dflycmd.cc:355] Transitioned into stable sync with replica 10.233.69.29:6379
2024-05-15 17:34:48.282 E20240516 00:34:48.281720     8 dfly_main.cc:560] An attempt to execute an instruction failed.The root cause might be an old hardware. Exiting...

My Dragonfly instance is as follows:

apiVersion: dragonflydb.io/v1alpha1
kind: Dragonfly
metadata:
  name: dragonfly
spec:
  image: ghcr.io/dragonflydb/dragonfly:v1.18.0
  replicas: 3
  args:
    - --proactor_threads=2
  resources:
    requests:
      cpu: 500m
    limits:
      memory: 2048Mi

Is there something else, or some command, that I could run to collect more logs to figure out why it's crashing? I don't think I'm doing anything too crazy lol

Pothulapati commented 3 months ago

I think this is more specific to Dragonfly binary and the container not being able to run in your environment. I think, Dragonfly repository makes more sense to discuss/debug this issue.

kostasrim commented 3 months ago

Hi @perfectra1n you are probably running on old hardware and you will need to compile dragonfly yourself.

Can you run sudo cat /proc/cpuinfo | grep avx and let me know what it outputs

See also #296

dobrac commented 3 months ago

Hi, I'm struggling with a similar issue. My virtual host on Proxmox was using x86-64-v2-AES. I've switched it to x86-64-v3 (at least), and it seems like it has solved the issue.

kostasrim commented 3 months ago

Hi @dobrac as explained above it's because you are running on a older architecture. You can either upgrade (as you did) to a newer machine or you can compile DF for that architecture (we got cmake flags for that). Both of them can work and it's a matter of preference :)

perfectra1n commented 3 months ago

Yep, this was my fault. I had imported my VMs into Proxmox from vSphere/ESXi and hadn't changed the CPU. I should've set them to host: image

Thanks for following up, I'll go ahead and close this out so that others that run into the same issue can at least find it :)