search

fengggli / comanche

comanche
Apache License 2.0
0 stars 1 forks source link

virtual machine using vagrant #5

Open fengggli opened 5 years ago

fengggli commented 5 years ago
  1. spdk provides a script to run nvme-enabled virtual machine in vagrant. See the instructions

Steps.

  1. install vbox and vbox extension instruction
  2. install vagrant instruction
  3. Follow the instruction [here] (https://spdk.io/doc/vagrant.html)

kernel

  1. needs to install kernel which has linux-headers provided by ubuntu. to compile kernel module 
    sudo apt-get install linux-generic
  2. to allocate hugepage, use spdk/scripts/setup.sh
fengggli commented 5 years ago

Example output

vagrant@vagrant:~/spdk$ sudo ./scripts/setup.sh       
0000:00:0e.0 (80ee 4e56): nvme -> uio_pci_generic
vagrant@vagrant:~/spdk$ sudo ./examples/nvme/hello_world/hello_world
Starting SPDK v18.10 / DPDK 18.08.0 initialization...
[ DPDK EAL parameters: hello_world -c 0x1 --file-prefix=spdk0 --base-virtaddr=0x200000000000 --proc-type=auto ]
EAL: Detected 2 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Auto-detected process type: PRIMARY
EAL: Multi-process socket /var/run/dpdk/spdk0/mp_socket
EAL: Probing VFIO support...
Initializing NVMe Controllers
EAL: PCI device 0000:00:0e.0 on NUMA socket 0
EAL:   probe driver: 80ee:4e56 spdk_nvme
Attaching to 0000:00:0e.0
Attached to 0000:00:0e.0
Using controller ORCL-VBOX-NVME-VER12 (VB1234-56789        ) with 1 namespaces.
  Namespace ID: 1 size: 1GB
Initialization complete.
INFO: using host memory buffer for IO
Hello world!

I haven't figured out how to enable vfio in this guest, perhaps it's not supported at all. In vagrant I could use ( see https://gist.github.com/tristanmorgan/3c7bd99d7bca81bd965b) vb.customize ["modifyvm", :id, "--chipset", "ich9"]

fengggli commented 5 years ago

error in the host i loaded vfio, and guest also tries to attach to the same device: in the host

[Jul 9 15:12] vboxpci 0000:20:00.0: PCIRAW_POWER_OFF
[  +0.000013] vboxpci 0000:20:00.0: detached from IOMMU
[  +0.167846] traps: EMT-0[4535] general protection ip:7f90e9bd01df sp:7f911451fdc0 error:0 in VBoxPciRawR3.so[7f90e9bcd000+c000]
[ +12.385184] vboxpci 0000:20:00.0: freeing irq 59
[  +0.000016] vboxpci 0000:20:00.0: vboxPciOsDevDeinit
[  +0.000122] vboxpci 0000:20:00.0: reattaching old host driver nvme
[  +0.000018] BUG: unable to handle kernel NULL pointer dereference at 0000000000000008
[  +0.000011] IP: set_root+0x26/0xc0
[  +0.000002] PGD 0 P4D 0 
[  +0.000004] Oops: 0000 [#1] SMP PTI
[  +0.000003] Modules linked in: vfio_pci vfio_virqfd vfio_iommu_type1 vfio xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 xt_tcpudp ebtable_filter ebtables ip6table_filter ip6_tables ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntra
[  +0.000049]  pcbc nls_iso8859_1 aesni_intel aes_x86_64 crypto_simd glue_helper cryptd joydev input_leds snd_hda_codec snd_hda_core nvidia_uvm(POE) snd_hwdep snd_pcm snd_timer intel_cstate snd intel_rapl_perf soundcore wmi ioatdma shpchp
[  +0.000041] CPU: 6 PID: 4535 Comm: EMT-0 Tainted: P          IOE    4.15.0-54-generic #58~16.04.1-Ubuntu
[  +0.000003] Hardware name: IBM System x3650 M4 : -[7915AC1]-/00Y8494, BIOS -[VVE124AUS-1.30]- 11/21/2012
[  +0.000004] RIP: 0010:set_root+0x26/0xc0
[  +0.000002] RSP: 0018:ffffa2c6ca11f848 EFLAGS: 00010202
[  +0.000003] RAX: ffff899a9a7f0000 RBX: ffffa2c6ca11f9a0 RCX: 0000000000000001
[  +0.000002] RDX: ffffffffbc4bbbe8 RSI: 0000000000000041 RDI: ffffa2c6ca11f9a0
[  +0.000002] RBP: ffffa2c6ca11f868 R08: ffff899ae1228220 R09: ffff899a84b1eaf8
[  +0.000003] R10: ffffa2c6ca11fac8 R11: 000000000000000a R12: 0000000000000000
[  +0.000002] R13: ffffa2c6ca11f9a0 R14: 0000000000000001 R15: ffff89925dbecc00
[  +0.000003] FS:  00007f9114520700(0000) GS:ffff899ae1200000(0000) knlGS:0000000000000000
[  +0.000003] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  +0.000002] CR2: 0000000000000008 CR3: 0000001018e0a001 CR4: 00000000000626e0
[  +0.000002] Call Trace:
[  +0.000006]  path_init+0x1d1/0x310
[  +0.000003]  path_openat+0x78/0x13f0
[  +0.000005]  ? __radix_tree_delete+0x8b/0xb0
[  +0.000003]  ? __radix_tree_delete+0x8b/0xb0
[  +0.000003]  do_filp_open+0x99/0x110
[  +0.000019]  ? strbufoutput+0x46/0xa0 [vboxdrv]
[  +0.000004]  ? vprintk_emit+0x2f9/0x3b0
[  +0.000004]  ? _cond_resched+0x1a/0x50
[  +0.000003]  ? kmem_cache_alloc+0x10f/0x1c0
[  +0.000003]  file_open_name+0xf3/0x130
[  +0.000002]  ? file_open_name+0xf3/0x130
[  +0.000003]  filp_open+0x33/0x60
[  +0.000003]  vboxPciFileOpen.constprop.3+0x16/0x70 [vboxpci]
[  +0.000003]  vboxPciOsDevDeinit+0x218/0x2d0 [vboxpci]
[  +0.000003]  vboxPciDevDeinit+0x4d/0x70 [vboxpci]
[  +0.000017]  ? supdrvDestroySession+0xed/0x400 [vboxdrv]
[  +0.000007]  ? supdrvSessionRelease+0x64/0x70 [vboxdrv]
[  +0.000007]  ? VBoxDrvLinuxClose+0x19/0x30 [vboxdrv]
[  +0.000003]  ? __fput+0xea/0x220
[  +0.000003]  ? ____fput+0xe/0x10
[  +0.000004]  ? task_work_run+0x8a/0xb0
[  +0.000003]  ? do_exit+0x2de/0xb50
[  +0.000003]  ? do_group_exit+0x43/0xb0
[  +0.000003]  ? get_signal+0x296/0x5c0
[  +0.000005]  ? do_signal+0x37/0x740
[  +0.000003]  ? wake_up_state+0x10/0x20
[  +0.000003]  ? signal_wake_up_state+0x25/0x30
[  +0.000002]  ? complete_signal+0x101/0x220
[  +0.000003]  ? send_signal+0x3e/0x80
[  +0.000004]  ? exit_to_usermode_loop+0x80/0xd0
[  +0.000003]  ? prepare_exit_to_usermode+0x66/0x90
[  +0.000003]  ? general_protection+0x2f/0x50
[  +0.000003]  ? retint_user+0x8/0x8

In the guest