search

hellt / vrnetlab

Make VM-based Network OSes run in Containerlab
https://containerlab.dev
MIT License
117 stars 80 forks source link

Nokia IXR-X3 emulation failes with 'Unhandled IOM card type 1' #167

Closed sdktr closed 7 months ago

sdktr commented 7 months ago

While prepping for the addition of IXR-X1 and IXR-Xs to the supported pre-baked SROS types, I wasn't able to get a working IXR-X3 platform going. According to this vSIM docs, there is a card (cpm-ixr-x/imm36-qsfpdd) and mda (m36-qsfpdd) for this platform.

Testing with a raw variant reproduces the error:

    clab.type: >-
      cp: cpu=2 ram=6 chassis=ixr-x slot=A card=cpm-ixr-x/imm36-qsfpdd ___
      lc: cpu=2 ram=4 chassis=ixr-x slot=1 card=imm36-qsfpdd
docker logs -f clab-ml_8-x3
2024-01-04 09:44:55,196: launch    ' DEBUG    acting flags: username 'vrnetlab', password 'VR-netlab9', connection-mode 'tc', variant 'cp: cpu=2 ram=6 chassis=ixr-x slot=A card=cpm-ixr-x/imm36-qsfpdd ___ lc: cpu=2 ram=4 chassis=ixr-x slot=1 card=imm36-qsfpdd'
2024-01-04 09:44:55,196: launch    ' DEBUG    Environment variables: environ({'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'HOSTNAME': 'x3', 'TERM': 'xterm', 'CLAB_LABEL_CLAB_NODE_LAB_DIR': '/home/proxsys/netlab/lab8-bng/clab-ml_8/x3', 'CLAB_LABEL_CLAB_TOPO_FILE': '/home/proxsys/netlab/lab8-bng/clab.yml', 'CLAB_LABEL_CLAB_NODE_NAME': 'x3', 'CLAB_LABEL_CLAB_NODE_TYPE': 'cp: cpu=2 ram=6 chassis=ixr-x slot=A card=cpm-ixr-x/imm36-qsfpdd ___ lc: cpu=2 ram=4 chassis=ixr-x slot=1 card=imm36-qsfpdd', 'DOCKER_NET_V4_ADDR': '192.168.8.0/24', 'CLAB_LABEL_CONTAINERLAB': 'ml_8', 'CLAB_LABEL_CLAB_NODE_GROUP': '', 'LD_PRELOAD': '/usr/local/lib/faketime/libfaketime.so.1', 'CLAB_INTFS': '0', 'CONNECTION_MODE': 'tc', 'FAKETIME': '-40d', 'CLAB_LABEL_CLAB_NODE_KIND': 'vr-sros', 'DOCKER_NET_V6_ADDR': '', 'HOME': '/root', 'FAKETIME_SHARED': '/faketime_sem_1 /faketime_shm_1', 'LC_CTYPE': 'C.UTF-8'})
2024-01-04 09:44:55,197: launch    ' INFO     Parsed SR OS version: SROSVersion(version='23.10.R1', major=23, minor=10, patch=1)
2024-01-04 09:44:56,742: launch    ' INFO     License found
2024-01-04 09:44:56,742: launch    ' DEBUG    Raw SR OS variant: {'max_nics': 40, 'deployment_model': 'distributed', 'lcs': [{'cpu': '2', 'min_ram': '4', 'slot': '1', 'timos_line': 'chassis=ixr-x slot=1 card=imm36-qsfpdd'}], 'cp': {'cpu': '2', 'min_ram': '6', 'slot': 'A', 'timos_line': 'chassis=ixr-x slot=A card=cpm-ixr-x/imm36-qsfpdd'}}
2024-01-04 09:44:56,742: launch    ' INFO     SR OS Variant: cp: cpu=2 ram=6 chassis=ixr-x slot=A card=cpm-ixr-x/imm36-qsfpdd ___ lc: cpu=2 ram=4 chassis=ixr-x slot=1 card=imm36-qsfpdd
2024-01-04 09:44:56,742: launch    ' INFO     Number of NICs: 40
2024-01-04 09:44:56,742: launch    ' INFO     Configuration mode: model-driven
2024-01-04 09:44:56,742: launch    ' INFO     Creating br-mgmt bridge for management interface
2024-01-04 09:44:56,766: vrnetlab  ' DEBUG    Creating overlay disk image
2024-01-04 09:44:56,792: launch    ' INFO     License file found for UUID cb7be947-f755-4242-ac16-46c1e9083553 with start date None
2024-01-04 09:44:56,792: launch    ' INFO     Acting timos line: ['type=1,product=TIMOS:address=172.31.255.30/30@active address=200::1/127@active license-file=tftp://172.31.255.29/license.txt primary-config=tftp://172.31.255.29/config.txt system-base-mac=0C:00:e1:7c:51:00 chassis=ixr-x slot=A card=cpm-ixr-x/imm36-qsfpdd']
2024-01-04 09:44:56,792: vrnetlab  ' DEBUG    Creating overlay disk image
2024-01-04 09:44:56,816: launch    ' INFO     License file found for UUID cb7be947-f755-4242-ac16-46c1e9083553 with start date None
2024-01-04 09:44:56,821: vrnetlab  ' DEBUG    Starting vrnetlab SROS
2024-01-04 09:44:56,821: vrnetlab  ' DEBUG    VMs: [<__main__.SROS_cp object at 0x7f9069150d10>, <__main__.SROS_lc object at 0x7f9069150850>]
2024-01-04 09:44:56,821: vrnetlab  ' DEBUG    VM not started; starting!
2024-01-04 09:44:56,821: vrnetlab  ' INFO     Starting SROS_cp
2024-01-04 09:44:56,821: vrnetlab  ' DEBUG    qemu cmd: qemu-system-x86_64 -enable-kvm -display none -machine pc -monitor tcp:0.0.0.0:4000,server,nowait -m 6144 -serial telnet:0.0.0.0:5000,server,nowait -drive if=virtio,file=/sros-overlay-cp0.qcow2 -cpu host -smp 2 -uuid cb7be947-f755-4242-ac16-46c1e9083553 -smbios "type=1,product=TIMOS:address=172.31.255.30/30@active address=200::1/127@active license-file=tftp://172.31.255.29/license.txt primary-config=tftp://172.31.255.29/config.txt system-base-mac=0C:00:e1:7c:51:00 chassis=ixr-x slot=A card=cpm-ixr-x/imm36-qsfpdd" -device virtio-net-pci,netdev=br-mgmt,mac=0C:00:79:22:fc:00 -netdev bridge,br=br-mgmt,id=br-mgmt -device virtio-net-pci,netdev=vcp-int,mac=0C:00:52:c0:b1:01 -netdev tap,ifname=vcp-int,id=vcp-int,script=no,downscript=no
2024-01-04 09:45:02,838: launch    ' TRACE    OUTPUT: 
Total Memory: 6GB  Chassis Type: 0x0  Core Freq:        f MHz
TiMOS-V-23.10.R1 bootrom/i386 Nokia 7xxx ? Copyright (c) 2000-2023 Nokia.
All rights reserved. All use subject to applicable license agreements.
Built on Thu Oct 26 20:12:19 UTC 2023 by builder in /builds/2310B/R1/panos/main/sros
Looking for boot parameters.

2024-01-04 09:45:02,838: vrnetlab  ' DEBUG    VM not started; starting!
2024-01-04 09:45:02,838: vrnetlab  ' INFO     Starting SROS_lc
2024-01-04 09:45:02,838: vrnetlab  ' DEBUG    number of provisioned data plane interfaces is 0
2024-01-04 09:45:02,839: vrnetlab  ' DEBUG    qemu cmd: qemu-system-x86_64 -enable-kvm -display none -machine pc -monitor tcp:0.0.0.0:4001,server,nowait -m 4096 -serial telnet:0.0.0.0:5001,server,nowait -drive if=virtio,file=/sros-overlay-lc1.qcow2 -cpu host -smp 2 -uuid cb7be947-f755-4242-ac16-46c1e9083553 -smbios "type=1,product=TIMOS:chassis=ixr-x slot=1 card=imm36-qsfpdd" -device pci-bridge,chassis_nr=1,id=pci.1 -device pci-bridge,chassis_nr=2,id=pci.2 -device virtio-net-pci,netdev=mgmt,mac=0C:00:9c:96:98:00 -netdev user,id=mgmt,restrict=y -device virtio-net-pci,netdev=vfpc-int,mac=0C:00:55:87:51:00 -netdev tap,ifname=vfpc1-int,id=vfpc-int,script=no,downscript=no
2024-01-04 09:45:09,856: launch    ' TRACE    OUTPUT: 

Press any key to stop auto-boot...
 0
auto-booting...

boot device          : ata=0,2
unit number          : 0 
processor number     : 0 
host name            : 0.0.0.0
file name            : cf3:/timos/i386-boot.tim
ftp password (pw)    : *****
flags (f)            : 0x8 
target name (tn)     : VSR-I
startup script (s)   : pl=1
other (o)            : vsr-c#vsr_cpm,vsr_iom|158|0|0|0

Booting from ata=0,2:cf3:/timos/i386-boot.tim
Loading cf3:/timos/i386-boot.tim ...
Version L-23.10.R1, Thu Oct 26 20:12:19 UTC 2023 by builder in /builds/2310B/R1/panos/main/sros
text:(2487136-->9056756) + data:(439136-->2860252)
Starting at 0x4a000000...

(s) boot params: pl=1; setting TgtPf to 0x00000001

KVM based vcpu
Running in a KVM/QEMU virtual machine
ACPI: found 2 cores, 2 enabled

2024-01-04 09:45:15,863: launch    ' TRACE    OUTPUT: 
Total Memory: 6GB  Chassis Type: 0x13  Core Freq: 2596.992 MHz
TiMOS-L-23.10.R1 boot/i386 Nokia 7250 IXR Copyright (c) 2000-2023 Nokia.
All rights reserved. All use subject to applicable license agreements.
Built on Thu Oct 26 20:12:19 UTC 2023 by builder in /builds/2310B/R1/panos/main/sros

TiMOS BOOT LOADER
Time from clock is THU JAN 04 09:45:14 2024 UTC
Switching serial output to sync mode...
2024-01-04 09:45:18,868: launch    ' TRACE    OUTPUT:    done

Looking for cf3:/bof.cfg ... OK, reading

Contents of Boot Options File on cf3:
    primary-image    cf3:\timos\i386-both.tim
    primary-config   cf3:\config.cfg
    autonegotiate
    duplex           full
    speed            100
    wait             3
    persist          off

Contents of Boot Options File (including bios product data)
    primary-image    cf3:\timos\i386-both.tim
    primary-config   tftp://172.31.255.29/config.txt
    license-file     tftp://172.31.255.29/license.txt
    address          172.31.255.30/30 active
    address          200::1/127 active
    autonegotiate
    duplex           full
    speed            100
    wait             3
    persist          off
    system-base-mac  0c:00:e1:7c:51:00

Hit a key within 1 second to change boot parameters...    
2024-01-04 09:45:21,872: launch    ' TRACE    OUTPUT: 

Primary image location: cf3:\timos\
Loading image cf3:\timos\i386-cpm.tim
Version C-23.10.R1, Thu Oct 26 20:12:19 UTC 2023 by builder in /builds/2310B/R1/panos/main/sros
text:(76383072-->271531936)
2024-01-04 09:45:27,878: launch    ' TRACE    OUTPUT:  + data:(17969376-->148192336)
2024-01-04 09:45:30,882: launch    ' TRACE    OUTPUT: 
simStartupInfoBootLoaderInit, No chassis MAC address for slot 0
simStartupInfoBootLoaderInit, No chassis MAC address for slot 1
simStartupInfoBootLoaderInit, No chassis MAC address for slot 3
simStartupInfoBootLoaderInit, No chassis MAC address for slot 4
simStartupInfoBootLoaderInit, No chassis MAC address for slot 5
simStartupInfoBootLoaderInit, No chassis MAC address for slot 6
simStartupInfoBootLoaderInit, No chassis MAC address for slot 7
simStartupInfoBootLoaderInit, No chassis MAC address for slot 8
simStartupInfoBootLoaderInit, No chassis MAC address for slot 9
simStartupInfoBootLoaderInit, No chassis MAC address for slot 10
simStartupInfoBootLoaderInit, No chassis MAC address for slot 11
simStartupInfoBootLoaderInit, No chassis MAC address for slot 12
simStartupInfoBootLoaderInit, No chassis MAC address for slot 13
simStartupInfoBootLoaderInit, No chassis MAC address for slot 14
simStartupInfoBootLoaderInit, No chassis MAC address for slot 15
simStartupInfoBootLoaderInit, No chassis MAC address for slot 16
simStartupInfoBootLoaderInit, No chassis MAC address for slot 17
simStartupInfoBootLoaderInit, No chassis MAC address for slot 18
simStartupInfoBootLoaderInit, No chassis MAC address for slot 19
simStartupInfoBootLoaderInit, No chassis MAC address for slot 20
simStartupInfoBootLoaderInit, No chassis MAC address for slot 21
simStartupInfoBootLoaderInit, No chassis MAC address for slot 22
simStartupInfoBootLoaderInit, No chassis MAC address for slot 23
simStartupInfoBootLoaderInit, No chassis MAC address for slot 24
Initializing management port vio0 using IP addr 172.31.255.30.
Initial DNS resolving preference is ipv6-first
Initializing management port using fe80::e00:79ff:fe22:fc00/64
Initializing management port using 200::1/127

Current Time: THU JAN 04 09:45:27 UTC 2024
Loading license file for 'TiMOS-C-23.10.R1' from: tftp://172.31.255.29/license.txt ... OK
Executing TiMOS image at 0x150000

KVM based vcpu
Running in a KVM/QEMU virtual machine
ACPI: found 2 cores, 2 enabled

2024-01-04 09:45:42,896: launch    ' TRACE    OUTPUT: 
Total Memory: 6GB  Chassis Type: 0x13  Core Freq: 2596.992 MHz
TiMOS-C-23.10.R1 cpm/x86_64 Nokia 7250 IXR Copyright (c) 2000-2023 Nokia.
All rights reserved. All use subject to applicable license agreements.
Built on Thu Oct 26 20:12:19 UTC 2023 by builder in /builds/2310B/R1/panos/main/sros

   ___                    ___           ___           ___
  /\  \                  /\__\         /\  \         /\  \
  \:\  \        ___     /::|  |       /::\  \       /::\  \
   \:\  \      /\__\   /:|:|  |      /:/\:\  \     /:/\:\  \
   /::\  \    _\/__/  /:/|:|__|__   /:/  \:\  \   _\:\~\:\  \
  /:/\:\__\  /\__\   /:/ |::::\__\ /:/__/ \:\__\ /\ \:\ \:\__\
 /:/  \/__/ /:/  /   \/__/~~/:/  / \:\  \ /:/  / \:\ \:\ \/__/
/:/  /     /:/  /          /:/  /   \:\  /:/  /   \:\ \:\__\
\/__/      \/__/          /:/  /     \:\/:/  /     \:\/:/  /
                         /:/  /       \::/  /       \::/  /
                         \/__/         \/__/         \/__/

Running 64 bit architecture

Initializing VMM
  Virtual address sharing is disabled
Time from clock is THU JAN 04 09:45:40 2024 UTC

2024-01-04 09:45:51,906: launch    ' TRACE    OUTPUT: Initial DNS resolving preference is ipv6-first
SMP: 2 cores available

2024-01-04 09:45:54,910: launch    ' TRACE    OUTPUT: [023 s 01/04/24 09:45:53.527] A:tRootTask:CHMGR:chMgrGetDefaultIomCardType Unhandled IOM card type 1
[023 s 01/04/24 09:45:53.557] A:tRootTask:CHMGR:chMgrGetDefaultIomCardType Unhandled IOM card type 1
[023 s 01/04/24 09:45:53.660] A:jobsorter:CHMGR:chMgrGetDefaultIomCardType Unhandled IOM card type 1
[023 s 01/04/24 09:45:53.667] A:jobsorter:CHMGR:chMgrGetDefaultIomCardType Unhandled IOM card type 1
[023 s 01/04/24 09:45:53.707] A:tRootTask:CHMGR:chMgrGetDefaultIomCardType Unhandled IOM card type 1
[023 s 01/04/24 09:45:53.714] A:tRootTask:CHMGR:chMgrGetDefaultIomCardType Unhandled IOM card type 1
[023 s 01/04/24 09:45:53.723] A:tRootTask:CHMGR:chMgrGetDefaultIomCardType Unhandled IOM card type 1
[023 s 01/04/24 09:45:53.753] A:jobsorter:CHMGR:chMgrGetDefaultIomCardType Unhandled IOM card type 1
[023 s 01/04/24 09:45:53.761] A:jobsorter:CHMGR:chMgrGetDefaultIomCardType Unhandled IOM card type 1
023 s 01/04/24 09:45:53.786: 
tRootTask: Assertion "!response.hasErrors()" failed
   File: mgmt_core/db/session_mgmt/dbInterface.cpp  Line: 125  Function: initializeDatabase
   Message: "Failed to create default objects: 'root card' (db-mo-id 460) : {type mo, module CHMGR (15), error-code 1011 (The IOM card is automatically provisioned and cannot be unprovisioned in this system), MO 'DbKeyRef: `root card "1"`' (460), text ' '"
   Stack Trace:
   0x211f810, i386StackBackTrace (0x0211f810) + 0x0
   0x216dc89, stackBackTrace (0x0216dc80) + 0x9
   0x2306ac5, timosAssert (0x023068a0) + 0x225
   0x479dfba, mgmtCore::DbInterface::initializeDatabase(bool) (0x0479dd40) + 0x27a
   0x4aedb7f, mgmtCoreStart (0x04aedb60) + 0x1f
   0xaa81bcf, pnMainPrivate (0x0aa7f233) + 0x299c
   0xaa7ee19, pnMain (0x0aa7ee0d) + 0xc
   0xaacb999, usrRoot (0x0aacb560) + 0x439
   0xa92d2cf, vxTaskEntry (0x0a92d2b0) + 0x1f

2024-01-04 09:45:57,914: launch    ' TRACE    OUTPUT: 
***************************************************************

Disabling switch fabric and mgmt ethernet communications

***************************************************************

System Crash Dump Written - Please run the command
`admin tech-support` and contact your Nokia
support representative.

Rebooting...
Using preloaded VxWorks boot loader at 0x0000000000008000, size 0x0007D000, entrypoint 0x0000000000008010

KVM based vcpu
Running in a KVM/QEMU virtual machine
ACPI: found 2 cores, 2 enabled

2024-01-04 09:46:00,918: launch    ' TRACE    OUTPUT: 
Total Memory: 6GB  Chassis Type: 0x0  Core Freq:        f MHz
TiMOS-V-23.10.R1 bootrom/i386 Nokia 7xxx ? Copyright (c) 2000-2023 Nokia.
All rights reserved. All use subject to applicable license agreements.
Built on Thu Oct 26 20:12:19 UTC 2023 by builder in /builds/2310B/R1/panos/main/sros
Looking for boot parameters.

2024-01-04 09:46:03,922: launch    ' TRACE    OUTPUT:
sdktr commented 7 months ago

I found a first hint to the solution for this problem in this doc

The IXR-X3 has it's own chassis.

The vSIM boots when using this as the type:

    clab.type: >-
      cp: cpu=2 ram=6 chassis=ixr-x3 slot=A card=cpm-ixr-x/imm36-qsfpdd ___
      lc: cpu=2 ram=6 chassis=ixr-x3 slot=1 card=imm36-qsfpdd

After quite some time (5+ minutes?) the line card + mda show 'up'


A:admin@x3# show chassis
===============================================================================
System Information
===============================================================================
  Name                              : x3
  Type                              : 7250 IXR-x3
  Chassis Topology                  : Standalone
  Location                          : (Not Specified)
  Coordinates                       : (Not Specified)
  CLLI code                         :           
  Number of slots                   : 2
  Oper number of slots              : 2
  Num of faceplate ports/connectors : 36
  Num of physical ports             : 0
  Critical LED state                : Off
  Major LED state                   : Off
  Minor LED state                   : Off
  Over Temperature state            : OK
  Base MAC address                  : 0c:00:13:ff:cc:00

A:admin@x3# show card

===============================================================================
Card Summary
===============================================================================
Slot      Provisioned Type                         Admin Operational   Comments
              Equipped Type (if different)         State State         
-------------------------------------------------------------------------------
1         imm36-qsfpdd                             up    up            
A         cpm-ixr-x                                up    up/active     
===============================================================================

[/]

A:admin@x3# show mda

===============================================================================
MDA Summary
===============================================================================
Slot  Mda   Provisioned Type                            Admin     Operational
                Equipped Type (if different)            State     State
-------------------------------------------------------------------------------
1     1     m36-qsfpdd                                  up        up
===============================================================================

[/]