search

dhtech / fboss

Facebook Open Switching SystemSoftware for controlling network switches.
Other
1 stars 0 forks source link

QSFP IDProm failed as QSFP is not ready #1

Open bluecmd opened 6 years ago

bluecmd commented 6 years ago 
E1006 18:25:53.389362 28705 WedgeProductInfo.cpp:136] json parse error on line 0: expected json value
E1006 18:25:53.389615 28705 WedgeProductInfo.cpp:67] json parse error on line 0: expected json value
E1006 18:26:39.302598 28708 FunctionScheduler.cpp:508] Error running the scheduled function <refreshTransceivers>: N8facebook5fboss10FbossErrorE: QSFP IDProm failed as QSFP is not ready

When restarting qsfp_service:

E1006 18:27:01.224939 28763 WedgeProductInfo.cpp:67] json parse error on line 0: expected json value
terminate called after throwing an instance of 'facebook::fboss::FbossError'
  what():  QSFP IDProm failed as QSFP is not ready
*** Aborted at 1538850421 (unix time) try "date -d @1538850421" if you are using GNU date ***
PC: @     0x7f5e3ac46fff gsignal
*** SIGABRT (@0x705b) received by PID 28763 (TID 0x7f5e439c8a80) from PID 28763; stack trace: ***
    @     0x7f5e416860c0 (unknown)
    @     0x7f5e3ac46fff gsignal
    @     0x7f5e3ac4842a abort
    @     0x7f5e3b55f0ad __gnu_cxx::__verbose_terminate_handler()
    @     0x7f5e3b55d066 (unknown)
    @     0x7f5e3b55d0b1 std::terminate()
    @     0x7f5e3b55d2c9 __cxa_throw
    @     0x562fdc2f9fa0 facebook::fboss::QsfpModule::setQsfpIdprom()
    @     0x562fdc2fc7e1 facebook::fboss::QsfpModule::updateQsfpData()
    @     0x562fdc2fbc34 facebook::fboss::QsfpModule::refreshLocked()
    @     0x562fdc2fba87 facebook::fboss::QsfpModule::refresh()
    @     0x562fdc30d81a facebook::fboss::WedgeManager::initTransceiverMap()
    @     0x562fdc2f5d77 facebook::fboss::QsfpServiceHandler::init()
    @     0x562fdc2f2ea4 main
    @     0x7f5e3ac342e1 __libc_start_main
    @     0x562fdc2f221a _start
    @                0x0 (unknown)
bluecmd commented 6 years ago 
dhtech@wedge1:/usr/local/bin$ sudo /usr/local/bin/qsfp_service -alsologtostderr -v=2000 -mode=wedge 2>&1 | grep -v ": " | grep -v "CP2112" | grep -v 'EventBase' | grep -v 'AsyncSocket'
I1006 18:57:54.355195  1896 BaseWedgeI2CBus.cpp:93] selecting QSFP 1
I1006 18:57:54.382166  1896 BaseWedgeI2CBus.cpp:93] selecting QSFP 2
I1006 18:57:54.411170  1896 BaseWedgeI2CBus.cpp:93] selecting QSFP 3
I1006 18:57:54.438171  1896 BaseWedgeI2CBus.cpp:93] selecting QSFP 4
I1006 18:57:54.467165  1896 BaseWedgeI2CBus.cpp:93] selecting QSFP 5
I1006 18:57:54.496182  1896 BaseWedgeI2CBus.cpp:93] selecting QSFP 6
I1006 18:57:54.523181  1896 BaseWedgeI2CBus.cpp:93] selecting QSFP 7
I1006 18:57:54.558161  1896 BaseWedgeI2CBus.cpp:93] selecting QSFP 7
terminate called after throwing an instance of 'facebook::fboss::FbossError'
*** Aborted at 1538852274 (unix time) try "date -d @1538852274" if you are using GNU date ***
    @     0x7fc6d4cc80c0 (unknown)
    @     0x7fc6ce288fff gsignal
    @     0x7fc6ce28a42a abort
    @     0x7fc6ceba10ad __gnu_cxx::__verbose_terminate_handler()
    @     0x7fc6ceb9f066 (unknown)
    @     0x7fc6ceb9f0b1 std::terminate()
    @     0x7fc6ceb9f2c9 __cxa_throw
    @     0x5580fdffffa0 facebook::fboss::QsfpModule::setQsfpIdprom()
    @     0x5580fe0027e1 facebook::fboss::QsfpModule::updateQsfpData()
    @     0x5580fe001c34 facebook::fboss::QsfpModule::refreshLocked()
    @     0x5580fe001a87 facebook::fboss::QsfpModule::refresh()
    @     0x5580fe01381a facebook::fboss::WedgeManager::initTransceiverMap()
    @     0x5580fdffbd77 facebook::fboss::QsfpServiceHandler::init()
    @     0x5580fdff8ea4 main
    @     0x7fc6ce2762e1 __libc_start_main
    @     0x5580fdff821a _start
    @                0x0 (unknown)
bluecmd commented 6 years ago

This only happens on the 40 -> 4x10G cables. Might be related to port speed configuration?

bluecmd commented 6 years ago

With wedge_agent up and running and all ports set to "0" it still doesn't work.

Port 1
  ID: 0xff
  Status: 0xff 0xff
  Interrupt Flags:
    LOS: 0xff
    Fault: 0xff
    Temp: 0xff
    Vcc: 0xff
    Rx Power: 0xff 0xff
    Tx Power: 0xff 0xff
    Tx Bias: 0xff 0xff
    Reserved Set 4: 0xff 0xff
    Reserved Set 5: 0xff 0xff
    Vendor Defined: 0xff 0xff 0xff
  Temperature: -0.003906 C
  Supply Voltage: 6.553500 V
  Channel Data:      RX Power        TX Power         TX Bias
    Channel 1:       6.553500mW      6.553500mW    131.000000mA
    Channel 2:       6.553500mW      6.553500mW    131.000000mA
    Channel 3:       6.553500mW      6.553500mW    131.000000mA
    Channel 4:       6.553500mW      6.553500mW    131.000000mA
    Power measurement is unsupported
    Reported RX Power is OMA
  Power set:  0x05      Extended ID:  0x10      Ethernet Compliance:  0x08
  TX disable bits: 0x00
  Rate select is unsupported
  RX rate select bits: 0xff
  TX rate select bits: 0xff
  CDR support:  TX: unsupported RX: unsupported
  CDR bits: 0xff
  Connector: 0x23
  Spec compliance: 0x08 0x00 0x00 0x000x00 0x00 0x00 0x00
  Encoding: 0x05
  Nominal Bit Rate: 10300 MBps
  Ext rate select compliance: 0x00
  Length (SMF): 0 km
  Length (OM3): 0 m
  Length (OM2): 0 m
  Length (OM1): 0 m
  Length (Copper): 2 m
  Length (Copper dM): 9.0 m
  Length (Copper effective): 9.0 m
  Device Tech: 0xa0
  Ext Module: 0x00
  Wavelength tolerance: 0x00 0x00
  Max case temp: 70C
  CC_BASE: 0x42
  Options: 0x00 0x00 0x00 0xc0
  DOM Type: 0x00
  Enhanced Options: 0x00
  Reserved: 0x00
  CC_EXT: 0x4d
  Vendor Specific:
    00 00 08 f1 bd 64 83 b5  fa ce 0e 1d 5a 4c 3e 9c
    7a 41 cd 00 00 00 00 00  00 00 00 00 14 df 98 cf
  Vendor: Fiberstore
  Vendor OUI: ff:ff:ff
  Vendor PN: QSFP-4SFP10G-DAC
  Vendor Rev: 03
  Vendor SN: D8791918245
  Date Code: 20171116