TritonDataCenter / smartos-live

For more information, please see http://smartos.org/ For any questions that aren't answered there, please join the SmartOS discussion list: https://smartos.topicbox.com/groups/smartos-discuss
1.57k stars 246 forks source link

usbdevice qemu options block vm win10 #606

Open riggy2013 opened 8 years ago

riggy2013 commented 8 years ago

I can boot win10 x64 successfully.

Then I tried to add a usb ekey for my system. I got my usb vendor-id and product-id by prtconf. Then i added

qemu_opts="-usb -usbdevice host:vendor-id:product-id"

in my json and updated the vm.

However when I booted the vm again, the vm couldn't boot with error messages as:

WARNING: ip_interface_cleanup: cannot open /devices/pseudo/udp@0:udp: error 13.

I checked /zones/uuid/root/tmp, all log files are with length zero. vm.startvm.log only record the qemu command and no error. In qemu command, option of "-usb -usbdevice" is correctly added.

any idea?

riggy2013 commented 8 years ago

Here is vm.startvm.log:

# cat vm.startvm.log
+ [[ -x /startvm.zone ]]
+ exec /smartdc/bin/qemu-exec /smartdc/bin/qemu-system-x86_64 -m 4096 -name f97e8d77-293f-c50a-e147-908ed8158fee -uuid f97e8d77-293f-c50a-e147-908ed8158fee -cpu Nehalem -smp 2 -drive file=/dev/zvol/rdsk/zones/f97e8d77-293f-c50a-e147-908ed8158fee-disk0,if=virtio,index=0,media=disk,boot=on -boot order=cd -device virtio-net-pci,mac=c2:ca:32:8b:dd:43,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0 -net vnic,name=net0,vlan=0,ifname=net0 -smbios 'type=1,manufacturer=Joyent,product=SmartDC HVM,version=7.20160414T011743Z,serial=f97e8d77-293f-c50a-e147-908ed8158fee,uuid=f97e8d77-293f-c50a-e147-908ed8158fee,sku=001,family=Virtual Machine' -pidfile /tmp/vm.pid -vga std -chardev socket,id=qmp,path=/tmp/vm.qmp,server,nowait -qmp chardev:qmp -chardev socket,id=serial0,path=/tmp/vm.console,server,nowait -serial chardev:serial0 -chardev socket,id=serial1,path=/tmp/vm.ttyb,server,nowait -serial chardev:serial1 -usb -usbdevice host:1483:c007

vm.log has 1297b and soon be flushed (why?). here is the saved contents:

# cat t.log
== Zone privileges ==
file_read
file_write
net_rawaccess
proc_clock_highres
proc_exec
proc_lock_memory
sys_admin
sys_mount
=== ARGV ===
/smartdc/bin/qemu-exec
/smartdc/bin/qemu-system-x86_64
-m
4096
-name
f97e8d77-293f-c50a-e147-908ed8158fee
-uuid
f97e8d77-293f-c50a-e147-908ed8158fee
-cpu
Nehalem
-smp
2
-drive
file=/dev/zvol/rdsk/zones/f97e8d77-293f-c50a-e147-908ed8158fee-disk0,if=virtio,index=0,media=disk,boot=on
-boot
order=cd
-device
virtio-net-pci,mac=c2:ca:32:8b:dd:43,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0
-net
vnic,name=net0,vlan=0,ifname=net0
-smbios
type=1,manufacturer=Joyent,product=SmartDC HVM,version=7.20160414T011743Z,serial=f97e8d77-293f-c50a-e147-908ed8158fee,uuid=f97e8d77-293f-c50a-e147-908ed8158fee,sku=001,family=Virtual Machine
-pidfile
/tmp/vm.pid
-vga
std
-chardev
socket,id=qmp,path=/tmp/vm.qmp,server,nowait
-qmp
chardev:qmp
-chardev
socket,id=serial0,path=/tmp/vm.console,server,nowait
-serial
chardev:serial0
-chardev
socket,id=serial1,path=/tmp/vm.ttyb,server,nowait
-serial
chardev:serial1
-usb
-usbdevice
host:1483:c007
=== OUTPUT (2016-04-21T15:49:46Z) ===
qemu-system-x86_64: -net vnic,name=net0,vlan=0,ifname=net0: vnic dhcp disabled

Could not open option rom 'extboot.bin': No such file or directory
qemu: could not add USB device 'host:1483:c007'

I can assure vendor-id and product-id is corrected. here is the info in mac osx:

eKey:

  Product ID:   0xc007
  Vendor ID:    0x1483
  Version:  3.00
  Speed:    Up to 12 Mb/sec
  Manufacturer: MingwahAohan
  Location ID:  0x14200000 / 14
  Current Available (mA):   1000
  Current Required (mA):    100
  Extra Operating Current (mA): 0
riggy2013 commented 8 years ago

Anybody help?

Or give me some advice on debug procedure? Tools, logs, etc.

Thanks in advance!

rmustacc commented 8 years ago

On 4/25/16 7:25 , riggy2013 wrote:

Anybody help?

Or give me some advice on debug procedure? Tools, logs, etc.

Did you actually make that USB device available in the zone?

It sounds like you're using the qemu_extra_opts option to pass these into the zone? Is that correct? What does the actual JSON for the VM look like?

Note in general, we don't support USB passthrough, so I suspect you'll be doing a lot of digging. The vm.log where it mentions "qemu: could not add USB device 'host:1483:c007'" is almost certainly the problem. So if you actually put it into the zone, you'll need to next look at QEMU source code.

Let me know if you have additional questions or if that made sense.

riggy2013 commented 8 years ago

Hi, Robert,

Yes, I use qemu_extra_opts to pass the arguments to the zone. Here is my JSON file:

{
    "alias": "win10_64",
    "brand": "kvm",
    "vcpus": 2,
    "autoboot": false,
    "ram": 4096,
    "vnc_password": "password",
    "resolvers": ["8.8.8.8", "8.8.4.4"],
    "qemu_opts": "-usb -usbdevice host:1483:c007",
    "disks": [
    {
        "boot": true,
        "model": "virtio",
        "size": 81920
    }
    ],
    "nics": [
    {
        "nic_tag": "admin",
        "model": "virtio",
        "ip": "dhcp",
        "primary": 1
    }
    ]
}

The usb device is an e-key from a bank, which is plug-n-play in windows system w/o any driver. However, in smartos/solaris, it is not configured:

[root@homelab_dp /usbkey/vmcfg]# cfgadm -c configure usb0/1.2
cfgadm: Hardware specific failure: Cannot issue devctl to ap_id: /devices/pci@0,0/pci103c,330d@1a/hub@1:1.2
[root@homelab_dp /usbkey/vmcfg]# cfgadm usb0/1.2
Ap_Id                          Type         Receptacle   Occupant     Condition
usb0/1.2                       usb-input    connected    unconfigured ok
rmustacc commented 8 years ago

On 4/26/16 7:04 , riggy2013 wrote:

Hi, Robert,

Yes, I use qemu_extra_opts to pass the arguments to the zone. Here is my JSON file:

{
    "alias": "win10_64",
    "brand": "kvm",
    "vcpus": 2,
    "autoboot": false,
    "ram": 4096,
    "vnc_password": "password",
    "resolvers": ["8.8.8.8", "8.8.4.4"],
    "qemu_opts": "-usb -usbdevice host:1483:c007",
    "disks": [
    {
        "boot": true,
        "model": "virtio",
        "size": 81920
    }
    ],
    "nics": [
    {
        "nic_tag": "admin",
        "model": "virtio",
        "ip": "dhcp",
        "primary": 1
    }
    ]
}

The usb device is an e-key from a bank, which is plug-n-play in windows system w/o any driver. However, in smartos/solaris, it is not configured:

[root@homelab_dp /usbkey/vmcfg]# cfgadm -c configure usb0/1.2
cfgadm: Hardware specific failure: Cannot issue devctl to ap_id: /devices/pci@0,0/pci103c,330d@1a/hub@1:1.2
[root@homelab_dp /usbkey/vmcfg]# cfgadm usb0/1.2
Ap_Id                          Type         Receptacle   Occupant     Condition
usb0/1.2                       usb-input    connected    unconfigured ok

Regardless of whether we're configuring it, it appears you're not doing anything to make it visible to the zone. This means that even if we have it configured in the kernel (which to be honest, since I'm not sure how QEMU works with USB devices, I can't say if that's good or bad), it still wouldn't work because it's not visible in the zone. By default arbitrary devices don't show up in the zone.

As for it being configured, you may need to look at the USB ID or the USB class and consider what USB driver it should be attached to.

riggy2013 commented 8 years ago

Hi, Robert,

I'll say you are right. I changed to another USB device, a Toshiba USB key which can be recognized by smartos. The syndrome is the same. It seems the qemu_opts doesn't pass the info to the zone at all. Let me do more study on it.

Thanks,

David

rmustacc commented 8 years ago

On 4/26/16 7:43 , riggy2013 wrote:

Hi, Robert,

I'll say you are right. I changed to another USB device, a Toshiba USB key which can be recognized by smartos. The syndrome is the same. It seems the qemu_opts doesn't pass the info to the zone at all. Let me do more study on it.

The qemu_extra_opts field is just extra arbitrary arguments to QEMU, nothing more and nothing less. It's basically an escape hatch for doing unsupported things with QEMU.

You'll likely need to still pass in the device with things like zonecfg and co.

Robert

tomww commented 8 years ago

I've used for vbox this zoneconfig to pass the devices into the zone:

Beware, there is no locking of any type I believe. Avoid multiple zones access the same usb device.

riggy2013 commented 8 years ago

Hi, Tom,

Will u please give more details on how you do it?

Thanks,

David

On Wed, Apr 27, 2016 at 3:06 AM, tomww notifications@github.com wrote:

I've used for vbox this zoneconfig to pass the devices into the zone:

Beware, there is no locking of any type I believe. Avoid multiple zones access the same usb device.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/joyent/smartos-live/issues/606#issuecomment-214852101

riggy2013 commented 8 years ago

I read more on Internet. It seems smarts doesn't support usb passthrough or pci passthrough. And there is no intention to make it either.

Any one can confirm it?

I have to seek other options as freenas + virtualbox

rmustacc commented 8 years ago

On 4/28/16 6:54 , riggy2013 wrote:

I read more on Internet. It seems smarts doesn't support usb passthrough or pci passthrough. And there is no intention to make it either.

Any one can confirm it?

While we don't do PCI passthrough, USB may work. You'll need to use zonecfg to add the devices into the zone, which is a manual operation at this time.

jclulow commented 8 years ago

@riggy2013 It might help to take a look at Example 8: Using Pattern Matching in zonecfg(1M). I'm not sure what USB device it is that you're trying to pass through, or how it's represented in the system -- or even how QEMU looks for USB devices to pass through. Devices in /dev/usb might be a good place to look for the right nodes, but I'd also consult the QEMU documentation.

riggy2013 commented 8 years ago

I added my device thru zonecfg/device:

zonecfg -z f97e8d77-293f-c50a-e147-908ed8158fee info device
device:
    match: /dev/zvol/rdsk/zones/f97e8d77-293f-c50a-e147-908ed8158fee-disk0
    property: (name=boot,value="true")
    property: (name=model,value="virtio")
    property: (name=media,value="disk")
    property: (name=size,value="40960")
device:
    match: /dev/usb/mass-storage0

However it still doesn't show up in windows. Here is qemu log:

-drive
file=/dev/zvol/rdsk/zones/f97e8d77-293f-c50a-e147-908ed8158fee-disk0,if=virtio,index=0,media=disk,boot=on
-drive
file=/dev/usb/mass-storage0,if=undefined,index=1,media=disk
...
=== OUTPUT (2016-05-02T04:05:34Z) ===
qemu-system-x86_64: -net vnic,name=net0,vlan=0,ifname=net0: vnic dhcp disabled

qemu-system-x86_64: -drive file=/dev/usb/mass-storage0,if=undefined,index=1,media=disk: unsupported bus type 'undefined'

Thanks all for helping me.

BTW, the vm start logs: vm.log, vm.log.1 will quickly disappear. How do you guys debug vm boot issue?