utah-scs / splinter

A low-latency, extensible, multi-tenant key-value store.
54 stars 12 forks source link

Sandstorm panics if compiled in debug mode #10

Open ethransom opened 6 years ago

ethransom commented 6 years ago

Steps to reproduce:

  1. Compile in debug mode
  2. Run sudo scripts/run-server and use a client with use_invoke = true to run sudo scripts/run-ycsb

Note: I'm not sure if there are other configurations that also trigger these errors, there may or may not be.

Sandstorm fails with two types of panics present in the log:

ethanr@sandstorm01:~/Sandstorm$ sudo scripts/run-server
INFO:server: Starting up Sandstorm server with config ServerConfig { mac_address: "3c:fd:fe:04:9f:c2", ip_address: "192.168.0.2", udp_port: 0, nic_pci: "0000:04:00.1", client_mac: "3c:fd:fe:04:b0:e2", client_ip: "192.168.0.1", num_tenants: 8, install_addr: "127.0.0.1:7700" }
INFO:server: Populating test data table and extensions...                                                                                                                                         INFO:server: Finished populating data and extensions                                                                                                                                              EAL: Detected 32 lcore(s)
EAL: Probing VFIO support...
Devname: "0000:04:00.1"
EAL: PCI device 0000:04:00.1 on NUMA socket 0
EAL:   probe driver: 8086:1572 net_i40e
INFO:server: Successfully added scheduler(TID 43978) with rx,tx,sibling queues (3, 3, 4) to core 13.
INFO:server: Successfully added scheduler(TID 43975) with rx,tx,sibling queues (0, 0, 1) to core 10.
INFO:server: Successfully added scheduler(TID 43979) with rx,tx,sibling queues (4, 4, 5) to core 14.
INFO:server: Successfully added scheduler(TID 43980) with rx,tx,sibling queues (5, 5, 6) to core 15.
INFO:server: Successfully added scheduler(TID 43977) with rx,tx,sibling queues (2, 2, 3) to core 12.
INFO:server: Successfully added scheduler(TID 43982) with rx,tx,sibling queues (7, 7, 0) to core 17.
INFO:server: Successfully added scheduler(TID 43981) with rx,tx,sibling queues (6, 6, 7) to core 16.
INFO:server: Successfully added scheduler(TID 43976) with rx,tx,sibling queues (1, 1, 2) to core 11.
thread 'sched-17' panicked at 'attempt to subtract with overflow', /users/ethanr/Sandstorm/net/framework/src/native/zcsi/mbuf.rs:91thread ':sched-109' panicked at '
attempt to subtract with overflownote: Run with `RUST_BACKTRACE=1` for a backtrace.
', /users/ethanr/Sandstorm/net/framework/src/native/zcsi/mbuf.rs:91:9
WARN:server: Detected misbehaving task 43975 on core 10.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: "SendError(..)"', libcore/result.rs:983:5
  1. attempt to subtract with overflow, from net/framework/src/native/zcsi/mbuf.rs:91
  2. called 'Result::unwrap()' on an 'Err' value: "SendError(..)", I believe the unwrap is coming from "net/framework/src/scheduler/context.rs" line 200

I'm trying to debug the panic issue (#9), and it would be nice to be able to work from a binary that has been built with debug info.