search

GNS3 / gns3-server

GNS3 server
GNU General Public License v3.0
810 stars 263 forks source link

NX-OSv9000 doesn't boot with sata option #875

Closed adosztal closed 7 years ago

adosztal commented 7 years ago

Boot fails, goes for PXE boot:

Boot Failed. EFI Floppy
Boot Failed. EFI Floppy 1

iPXE 1.0.0+git-20131111.c3d1e78-2ubuntu1.1 -- Open Source Network Boot Firmware -- http://ipxe.org
Features: HTTP HTTPS DNS TFTP EFI Menu

net0: 00:53:18:d8:f3:00 using 82540em on PCI00:04.0 (open)
  [Link:up, TX:0 TXE:0 RX:0 RXE:0]
Configuring (net0 00:53:18:d8:f3:00).................. Error 0x040ee106 (http://ipxe.org/040ee106)

These are the disk related command line options from Cisco: -device ahci,id=ahci0,bus=pci.0 -drive file=/path_to_image/nxosv-final.7.0.3.I5.1.qcow2,if=none,id=drive-sata-disk0,id=drive-sata-disk0,format=qcow2 -device ide-drive,drive=drive-sata-disk0,bus=ahci0.0,id=drive-sata-disk0,bootindex=1

This is used for booting a node (right-click on node, Command line): -device ahci,id=ahci0,bus=pci.0 -drive file=/path_to_image/hda_disk.qcow2,if=none,id=drive-sata-disk0,index=0,media=disk -device ide-drive,drive=drive-sata-disk0,bus=ahci0.0

VM settings: nxosv9k-1 nxosv9k-2

PS: Low priority. The device boots well with hda set as IDE drive, it only takes more time.

grossmj commented 7 years ago

I can't reproduce the problem. I simply get a "No disk detected" when using SATA. However, I don't have the -bios and -smp 2 options using latest template. Maybe that is the reason?

grossmj commented 7 years ago

@adosztal where did you get bios.bin?

adosztal commented 7 years ago

https://sourceforge.net/projects/edk2/files/OVMF/ (direct link: https://sourceforge.net/projects/edk2/files/OVMF/OVMF-X64-r15214.zip/download)

cristian-ciobanu commented 7 years ago

I have tested this on ArchLinux and it boots ok using sata option. I'm using a newer ovmf bios file r18726 from the ArchLinux user repository https://aur.archlinux.org/packages/ovmf-git This is compiled using GIT sources from https://github.com/tianocore/edk2

grossmj commented 7 years ago

On Ubuntu 16.04 the boot fails, goes for PXE boot even when using IDE disk interface. I used https://sourceforge.net/projects/edk2/files/OVMF/OVMF-X64-r15214.zip/download

adosztal commented 7 years ago

Is the path to the BIOS file correct when uploading it using the "bios_image" property? The very same BIOS file boots for me.

grossmj commented 7 years ago

Here is the command line used by GNS3.

/usr/bin/qemu-system-x86_64 -name CiscoNX-OSv7.3.0-1 -m 3072M -smp cpus=2 -enable-kvm -machine smm=off -boot order=c -bios /home/grossmj/GNS3/images/QEMU/OVMF.fd -drive file=/home/grossmj/GNS3/projects/NX-OSv/project-files/qemu/51e65dfc-683f-457d-8ed2-2f75eabe3866/hda_disk.qcow2,if=ide,index=0,media=disk -uuid 51e65dfc-683f-457d-8ed2-2f75eabe3866 -serial telnet:127.0.0.1:5002,server,nowait -monitor tcp:127.0.0.1:50156,server,nowait -net none -device e1000,mac=00:52:21:38:66:00,netdev=gns3-0 -netdev socket,id=gns3-0,udp=127.0.0.1:10001,localaddr=0.0.0.0:10000 -nographic

I use the IDE interface and it still goes to PXE boot:

Boot Failed. EFI DVD/CDROM
Boot Failed. EFI Floppy
Boot Failed. EFI Floppy 1
Boot Failed. EFI Hard Drive

iPXE 1.0.0+git-20150424.a25a16d-1ubuntu1 -- Open Source Network Boot Firmware -- http://ipxe.org
Features: DNS HTTP HTTPS NFS TFTP EFI Menu

net0: 00:52:21:38:66:00 using 82540em on PCI00:03.0 (open)
  [Link:up, TX:0 TXE:0 RX:0 RXE:0]
Configuring (net0 00:52:21:38:66:00).................. Error 0x040ee186 (http://ipxe.org/040ee186)

Boot Failed. EFI Network                 

UEFI Interactive Shell v2.0
EDK II
UEFI v2.40 (EDK II, 0x00010000)
Mapping table
     BLK2: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)
     BLK9: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)
     BLK3: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)/HD(1,MBR,0x52F3A0EB,0x3F,0x1371)
     BLK4: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)/HD(2,MBR,0x52F3A0EB,0x13B0,0x147CC0)
     BLK7: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)/HD(3,MBR,0x52F3A0EB,0x149070,0xA3E60)
     BLK8: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)/HD(4,MBR,0x52F3A0EB,0x1ECED0,0x613110)
     BLK5: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)/HD(2,MBR,0x52F3A0EB,0x13B0,0x147CC0)/HD(1,MBR,0x00000000,0x13EF,0xA3E21)
     BLK6: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)/HD(2,MBR,0x52F3A0EB,0x13B0,0x147CC0)/HD(2,MBR,0x00000000,0xA524F,0xA3E21)
     BLK0: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x0)/Floppy(0x0)
     BLK1: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x0)/Floppy(0x1)
Press ESC in 1 seconds to skip startup.nsh or any other key to continue.
Shell>

It appears I can boot if I remove the bios image. So I am suspecting something is wrong with it.

Auto booting bootflash:/titanium-d1-kickstart.7.3.0.D1.1.bin bootflash:/titaniu
m-d1.7.3.0.D1.1.bin...
Booting kickstart image: bootflash:/titanium-d1-kickstart.7.3.0.D1.1.bin....
...............................................................................
........................Image verification OK

[    0.000000] Unknown boot option `ide_generic.probe_mask=0x0': ignoring
[    0.566796] pci 0000:00:01.0: PIIX3: Enabling Passive Release
INIT: version 2.86 booting
SCSI disk detected.
Enter system maintenance mode? (y/n) [n]: 
Checking all filesystems..n..r.retval=[0]
 done.
Trying to mount cdrom...
hostname: invalid option -- '-'
Try `hostname --help' for more information.
Renaming eth interfaces
  eobc_pseudo at eth0: no swap
no offload settings changed
Loading system software
Uncompressing system image: bootflash:/titanium-d1.7.3.0.D1.1.bin Sun Mar 5 16:58:23 UTC 2017
blogger: nothing to do.

..done Sun Mar 5 16:58:24 UTC 2017
INIT: Entering runlevel: 3
Mounting other filesystems:  [  OK  ]
Starting NFS statd: [  OK  ]

2017 Mar  5 16:58:30 switch %SYSLOG-2-SYSTEM_MSG : Syslogs wont be logged into logflash until logflash is online 

2017 Mar  5 16:58:38 switch %PKTMGR-2-STARTUP_FAILURE:  pktmgr [4081]  Failed to ERROR: Could not register MTS_OPC_VLAN_MGR_VLAN_CREATED opcode 0x5c1d, reason No such file or directory  during startup. error -7027844

2017 Mar  5 16:59:03 switch %MODULE-2-MOD_UNKNOWN: Module type [262] in slot 1 is not supported
cp: cannot stat `/isan/bin/cli-scripts/snapshot_dump.py': No such file or directory
cp: cannot stat `/isan/bin/cli-scripts/snapshot_ext.py': No such file or directory

2017 Mar  5 16:59:03 switch %MODULE-2-MOD_FAIL: Initialization of module 1 (Serial number: TM21386600B) failed
2017 Mar  5 16:59:03 switch %VDC_MGR-2-VDC_ONLINE: vdc 1 has come online 

User Access Verification
 login: 2017 Mar  5 16:59:06 switch %PLATFORM-2-CHASSIS_CLKMODOK: Chassis clock module A ok
2017 Mar  5 16:59:06 switch %PLATFORM-2-CHASSIS_CLKSRC: Current chassis clock source is clock-A

User Access Verification
 login: cisco
Password:
adosztal commented 7 years ago

You're booting the wrong image (NX-OSv, aka Titanium). UEFI is required by NX-OSv9000 (nxosv-final.7.0.3.I5.1.qcow2).

Note: it needs 8G RAM.

grossmj commented 7 years ago

Oh, how did I missed that? :(

Makes sense now, I will try with the correct image.

grossmj commented 7 years ago

With the correct image and IDE interface.

/usr/bin/qemu-system-x86_64 -name NXOSv9k -m 8092M -smp cpus=2 -enable-kvm -machine smm=off -boot order=c -bios /home/grossmj/GNS3/images/QEMU/OVMF.fd -drive file=/home/grossmj/GNS3/projects/NXOSv9000/project-files/qemu/be868d47-2a5e-4413-be25-b27b30c862ae/hda_disk.qcow2,if=ide,index=0,media=disk -uuid be868d47-2a5e-4413-be25-b27b30c862ae -serial telnet:127.0.0.1:5003,server,nowait -monitor tcp:127.0.0.1:38281,server,nowait -net none -device e1000,mac=00:54:c5:62:ae:00,netdev=gns3-0 -netdev socket,id=gns3-0,udp=127.0.0.1:10003,localaddr=0.0.0.0:10002 -nographic

I don't know what I missed this time. Looks like something happens on first boot but I always end up on a loader prompt :(

[    5.104215] [1488739105] Error: Could not find mtd devices 
mount: /dev/torissusda5 is not a valid block device
mount: /dev/torissusda5 is not a valid block device
mount: /dev/torissusda5 is not a valid block device
mount: /dev/torissusda5 is not a valid block device
umount: /torissusda5: not mounted
INIT: version 2.88 booting
Installing ata_piix module ... done.
Unsquashing rootfs ...
Installing isan procfs ... done.
Installing SSE module with card index 21099 ... done.
Creating SSE device node 248 ... done.
Loading I2C driver ... done.
Installing CCTRL driver for card_type 29 on a VM without NEED_GEM ... done.
18.14: Interrupt throttling disabled. No cctrl irq detected.
Loading IGB driver ... done.
Not Micron SSD...

Checking all filesystems.Warning: switch is starting up with default configuration

Installing SPROM driver ... IS_N9K done.
IS_SUP:; IS_SC:; IS_FC:; IS_LC:; IS_TOR:y done.
Installing pfmsvcs module with SPROM card index 21099 ... done.
Installing nvram module ... done.
Installing if_index module with port mode 6 ... done.
Installing RNI lcnd ... done
Installing LC netdev ... done
Installing psdev module ... done.
Installing veobc module ... done.
Inserting OBFL module ... done.
Making OBFL character devices
mounting plog for N9k!
Mounting OBFL pstore
Trying to mount cdrom...
Starting OpenBSD Secure Shell server: sshd ... done.
tune2fs 1.42.1 (17-Feb-2012)
Setting reserved blocks percentage to 0% (0 blocks)
flash_erase: error!: /dev/mtdrr
             error 19 (No such device)
flash_erase: error!: /dev/mtdoops
             error 19 (No such device)
Starting portmap daemon...
creating NFS state directory: done
starting 8 nfsd kernel threads: done
starting mountd: done
starting statd: done
Saving image for img-sync ...
Loading system software
Installing local RPMS
Patch Repository Setup completed successfully
Uncompressing system image: Sun Mar 5 18:38:52 UTC 2017
blogger: nothing to do.
Creating /dev/mcelog
Starting mcelog daemon
Overwriting dme stub lib
dme asap backend not explicitly enabled in platform.inf, removing it...
File /etc/shared/localtime does not exist.
INIT: Entering runlevel: 3
Running S93thirdparty-script...

Populating conf files for hybrid sysmgr ...
Starting hybrid sysmgr ...
inserting /isan/lib/modules/klm_cisco_nb.o ... done
Mar  5 18:39:07 %SYSMGR-2-LAST_CORE_BASIC_TRACE : PID 23515 with message sharedmemmgr(non-sysmgr) crashed, core will be saved .

Mar  5 18:39:12 %SYSMGR-3-CORE_OP_FAILED Core operation failed: send_msg_to_ccdmon: Could not send to CORE_DMON return -1 errno 32
Mar  5 18:39:23 %SYSMGR-3-CORE_OP_FAILED Core operation failed: send_msg_to_ccdmon: Could not send to CORE_DMON return -1 errno 32
Mar  5 18:39:24 %SYSMGR-2-HAP_FAILURE_SUP_RESET Service "event_manager" in vdc 1 has had a hap failure
[   67.730191] [1488739167] Restarting system.

Boot Failed. EFI DVD/CDROM
Boot Failed. EFI Floppy
Boot Failed. EFI Floppy 1
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(1,800,8000,3ff9057e-e931-40a4-4584-699cecf01024)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(2,8800,5147a,2e0a36f7-721d-4179-309f-e94b9e859790)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(3,5a000,a28f4,22a3a5ea-48a0-4521-f589-145002c2a32e)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(4,fd000,355712,662fab65-a470-4a5d-70b5-0d1f79ca70ed)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(5,452800,a28f4,5a07289e-1cb1-48b7-a687-d24fe98513b4)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(6,4f5800,a28f4,3391772f-460b-4a4b-5f8d-d122b01ec587)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(7,598800,261998,1ce2ee04-a294-47cf-7280-fc25766bcb63)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(1,0,0)/EndEntire
/ACPI(a0341d0,0)/PCI(0,1)/ACPI(60441d0,0)/EndEntire
/ACPI(a0341d0,0)/PCI(0,1)/ACPI(60441d0,1)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(1,800,8000,3ff9057e-e931-40a4-4584-699cecf01024)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(2,8800,5147a,2e0a36f7-721d-4179-309f-e94b9e859790)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(3,5a000,a28f4,22a3a5ea-48a0-4521-f589-145002c2a32e)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(4,fd000,355712,662fab65-a470-4a5d-70b5-0d1f79ca70ed)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(5,452800,a28f4,5a07289e-1cb1-48b7-a687-d24fe98513b4)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(6,4f5800,a28f4,3391772f-460b-4a4b-5f8d-d122b01ec587)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(0,0,0)/HD(7,598800,261998,1ce2ee04-a294-47cf-7280-fc25766bcb63)/EndEntire
/ACPI(a0341d0,0)/PCI(1,1)/ATAPI(1,0,0)/EndEntire
/ACPI(a0341d0,0)/PCI(0,1)/ACPI(60441d0,0)/EndEntire
/ACPI(a0341d0,0)/PCI(0,1)/ACPI(60441d0,1)/EndEntire
Total memory is 0xa0000 
Total memory is 0xbee3c000 
Sysconf checksum failed. Using default values
WARNING:  No BIOS Info found 
Sysconf checksum failed. Using default values
serial --speed=115200 --word=8 --stop=1 --parity=no

success 
Sysconf checksum failed. Using default values
Strlen bootflash: is 10 
Device bootflash: is Drive 0x80 Partition 3 
ATE0Q1&D2&C1S0=1
Trying to read config file /boot/grub/menu.lst.local from (hd0,4) 
 Filesystem type is ext2fs, partition type 0x83
Number of devices detected by BIOS is 1
Failed. Trying to read config file from (hd0,5) 
 Filesystem type is ext2fs, partition type 0x83
Number of devices detected by BIOS is 1
Failed. 

                Loader Version 5.00.9
Entering interactive mode

loader >
cristian-ciobanu commented 7 years ago

In the NX-OSv 9000 docs it's explained what needs to be done if image boots to the loader prompt. (http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/nx-osv/configuration/guide/b_NX-OSv_9000/b_NX-OSv_9000_chapter_010.html#reference_9D1A1EC3F2BF48A790DE489B6CDE69DF)

Tarik-H commented 7 years ago

Hi Jeremy were you able to boot N9kv via sata?

grossmj commented 7 years ago

@Tarik-H I never managed to boot via sata :(

Even by modifying the command line to match Cisco drive params.

/usr/bin/qemu-system-x86_64 -name CiscoNX-OSv9000 -m 8096M -smp cpus=1 -enable-kvm -machine smm=off -boot order=c -bios /home/grossmj/GNS3/images/QEMU/OVMF.fd -device ahci,id=ahci0,bus=pci.0 -drive file=/home/grossmj/GNS3/projects/NX-OS-test/project-files/qemu/28d9e622-8267-4952-9a77-184001c888e6/hda_disk.qcow2,if=none,id=drive-sata-disk0,format=qcow2 -device ide-drive,drive=drive-sata-disk0,bus=ahci0.0,id=drive-sata-disk0,bootindex=1 -uuid 28d9e622-8267-4952-9a77-184001c888e6 -serial telnet:127.0.0.1:5003,server,nowait -monitor tcp:127.0.0.1:44505,server,nowait -net none -device e1000,mac=00:ba:c0:88:e6:00,netdev=gns3-0 -netdev socket,id=gns3-0,udp=127.0.0.1:10001,localaddr=0.0.0.0:10000 -device e1000,mac=00:ba:c0:88:e6:01,netdev=gns3-1 -netdev socket,id=gns3-1,udp=127.0.0.1:10003,localaddr=0.0.0.0:10002 -device e1000,mac=00:ba:c0:88:e6:02,netdev=gns3-2 -netdev socket,id=gns3-2,udp=127.0.0.1:10005,localaddr=0.0.0.0:10004 -device e1000,mac=00:ba:c0:88:e6:03,netdev=gns3-3 -netdev socket,id=gns3-3,udp=127.0.0.1:10007,localaddr=0.0.0.0:10006 -device e1000,mac=00:ba:c0:88:e6:04,netdev=gns3-4 -netdev socket,id=gns3-4,udp=127.0.0.1:10009,localaddr=0.0.0.0:10008 -device e1000,mac=00:ba:c0:88:e6:05,netdev=gns3-5 -netdev socket,id=gns3-5,udp=127.0.0.1:10011,localaddr=0.0.0.0:10010 -device e1000,mac=00:ba:c0:88:e6:06,netdev=gns3-6 -netdev socket,id=gns3-6,udp=127.0.0.1:10013,localaddr=0.0.0.0:10012 -device e1000,mac=00:ba:c0:88:e6:07,netdev=gns3-7 -netdev socket,id=gns3-7,udp=127.0.0.1:10015,localaddr=0.0.0.0:10014 -device e1000,mac=00:ba:c0:88:e6:08,netdev=gns3-8 -netdev socket,id=gns3-8,udp=127.0.0.1:10017,localaddr=0.0.0.0:10016 -device e1000,mac=00:ba:c0:88:e6:09,netdev=gns3-9 -netdev socket,id=gns3-9,udp=127.0.0.1:10019,localaddr=0.0.0.0:10018 -nographic
cristian-ciobanu commented 7 years ago

The problem is that the version of the ovmf bios file r15214 you are using is too old. It was released 3 years ago feb 2014. At that time I didn't have SATA support. I saw a thread which had a patch to enable SATA controller on the ovmf bios image https://www.mail-archive.com/edk2-devel@lists.sourceforge.net/msg07864.html I have tested with recent releases compiled using GIT sources from https://github.com/tianocore/edk2 and it works fine. I also tried with the old bios image and it freezes and it boots into the UEFI shell.

grossmj commented 7 years ago

@cioby23 I didn't really look how you compile it but can you post the already compiled ovmf bios image? Yes this is lazyness ;) Thanks.

cristian-ciobanu commented 7 years ago

@grossmj I have attached the ovmf bios image I'm currently using. ovmf_x64.zip

grossmj commented 7 years ago

Thanks a lot. I am going to upload it to our sourceforge repository. Which version is it?

cristian-ciobanu commented 7 years ago

I think it's ovmf_x64-r18726. I saw also that Ubuntu provides ovmf package in their Xenial repository. http://packages.ubuntu.com/xenial/all/ovmf/download - release 18346

grossmj commented 7 years ago

I managed to boot via sate using your BIOS image. However, now it seems I cannot boot via ide or it just take so long :(

grossmj commented 7 years ago

@adosztal I have fixed the sata command line. It works provided you use a recent version of the BIOS image.

adosztal commented 7 years ago

Thanks, I'll update the template.

hslabbert-dw commented 7 years ago

The NX-OSv 9000 appliance info still references the r15214 image. Is the template fixing bits still to be done?

adosztal commented 7 years ago

Thanks for the heads up! I'll update the appliance soon.

adosztal commented 7 years ago

Login with admin, not root.

Btw there will be error messages, I think those appear because NX-OS tries to configure some features that are available in the hardware based models only.

JinwonPark commented 7 years ago

Oh....It was my fault. I don't know why I tried to root account.... I am sorry to have confused you. Thank you for your help 👍

I will delete the above article because It's too long

adosztal commented 7 years ago

No problem. :) Don't delete it, someone else might run into the same.