Clearfog GT 8K device compatibility

[jblankfeld]

Hi, I am pleased to find resources like yours since I have been messing with GPON ONU sticks for a while. Thank you for your work. I currently have a Clearfog GT 8K and CarlitoxxPro v2.0 device (actually VSOL V2801F) and I struggle to make it work in the SFP port of the device. I found couple of linux patches to support it: https://patchwork.kernel.org/project/netdevbpf/patch/20201230154755.14746-2-pali@kernel.org/#23867771 but I can not get a robust "Link Up" at every boot.. So I fixed it with a TP-Link MC220L but his is not the best, I am looking for a GPON ONU stick that would work on linux. It looks like many problems were caused by a poort implementation of EEPROM in the device. It seems that you managed to plug DFP-34X-2C2 in a macchiatoBIN which is very similar to Clearfog GT 8K (same chip) and that chip might solve my issues. Is the stick SFP+ ? Did you get link up on your device when you first plugged it ? Thanks for your kind advice..

[Anime4000]

Hi @jblankfeld RTL9601CI have very janky EEPROM emulator, every Stick use RTL9601CI SoC have issue reading it.

doesn't mean broken EEPROM making host cannot link with Stick, like RB3011 just work fine

have you tried changing LAN_SDS_MODE? RTL960x has 3 mode for 1Gb and 2.5Gb. https://github.com/Anime4000/RTL960x/issues/27

need bring @stich86 from issue #17 he has same hardware as you

[stich86]

Hi @jblankfeld,

i've talked with Russel King (the owner of that patch), here is the full discussion:

https://lore.kernel.org/linux-arm-kernel/Yic6vFuEhS7iO7nq@shell.armlinux.org.uk/T/

At the moment the only way to make the stick recognized is to bring up ethernet port before insert it. It's a problem of the advertisement from the stick, that the SFP driver of the board doesn't recognize and doesn't bring up the SFP port.

It needs a full rewrite of the driver to avoid this behaviour, but for me was just a test to obtain 2.5GB (HiSGMII) link. At the moment my stick is moved into a Broadcom BC57810s card and there is not issue

[tdmadam]

@stich86 Have any of you tried changing the default BitRate Nominal from 1200MBd to 2500MBd. I have some Alcatel modules that support 2.5Gbps. In their EEPROM at offset 0x0C there is a value=19 I tried to change it on my V2801F. Reading at 0x0C returns the value 0C. After writing at 0x0C a new value=19 and a new checksum=3D at 0x3F the new BR reading shows correctly 19.

Everything goes back to the old values when I power down the module. The V2801F uses an emulated EEPROM which is probably written to Flash somewhere. It would be good to trace at which address it is located.
It is different with DFP-34X-2C2. The EEPROM can be read almost instantly at power-on. It can also be written without a problem and remembers the values after shutdown.

[Anime4000]

@tdmadam V2801F firmware has sfpapp, while other stick dont have including TWCGPON657 firmware... so I think sfpapp cause it... try delete sfpapp line in etc ?

[stich86]

@stich86

Have any of you tried changing the default BitRate Nominal from 1200MBd to 2500MBd. I have some Alcatel modules that support 2.5Gbps. In their EEPROM at offset 0x0C there is a value=19

I tried to change it on my V2801F. Reading at 0x0C returns the value 0C. After writing at 0x0C a new value=19 and a new checksum=3D at 0x3F the new BR reading shows correctly 19.

Everything goes back to the old values when I power down the module. The V2801F uses an emulated EEPROM which is probably written to Flash somewhere. It would be good to trace at which address it is located.

It is different with DFP-34X-2C2. The EEPROM can be read almost instantly at power-on. It can also be written without a problem and remembers the values after shutdown.

Can you share the command for DFP32? Thx

[tdmadam]

@Anime4000 Thanks, I will take a closer look at sfpapp.

@stich86 I'm not sure what I2C-enabled programmer you have.

#READ
i2cget -y 0 0x50 0x0C
i2cget -y 0 0x50 0x3F

#WRITE
i2cset -y 0 0x50 0x0C 0x19
i2cset -y 0 0x50 0x3F 0x??

Obviously it will easier if you have programmer that can read SFP module and fix the checksum. But there is also a way to do it. The pictures I posted here are from SFP Total wizard. Their SFP adaptors are expensive but you can download the software and load your own dump and then fix the cheksum in it. https://sfptotal.io/

If everything goes well ethtool should show 2500MB.

edit: I found this command very helpful in Ubuntu. It converts hexadecimal numbers in the text file to an actual binary file.

xxd -p -r sfpdump.ascii sfpdump.bin

[tdmadam]

This is a quick test I did. Ubuntu Live 22.04, DFP-34X-2C2 plugged into BCM57810 card without any bnx2x drivers

root@ubuntu:/home/ubuntu# ethtool -m enp1s0f0

 Identifier                                : 0x03 (SFP)
 Extended identifier                       : 0x04 (GBIC/SFP defined by 2-wire interface ID)
 Connector                                 : 0x01 (SC)
 Transceiver codes                         : 0x00 0x00 0x00 0x02 0x22 0x00 0x01 0x00 0x00
 Transceiver type                          : Ethernet: 1000BASE-LX
 Transceiver type                          : FC: intermediate distance (I)
 Transceiver type                          : FC: Longwave laser (LC)
 Transceiver type                          : FC: Single Mode (SM)
 Encoding                                  : 0x01 (8B/10B)
!BR, Nominal                               : 2500MBd
 Rate identifier                           : 0x00 (unspecified)
 Length (SMF,km)                           : 20km
 Length (SMF)                              : 20000m
 Length (50um)                             : 0m
 Length (62.5um)                           : 0m
 Length (Copper)                           : 0m
 Length (OM3)                              : 0m
 Laser wavelength                          : 1310nm
 Vendor name                               : OEM
 Vendor OUI                                : 00:00:00
 Vendor PN                                 : STICK
 Vendor rev                                : 
 Option values                             : 0x00 0x1a
 Option                                    : RX_LOS implemented
 Option                                    : TX_FAULT implemented
 Option                                    : TX_DISABLE implemented
 BR margin, max                            : 0%
 BR margin, min                            : 0%
 Vendor SN                                 : GPON22010163
 Date code                                 : 220117

I wonder what LAN_SDS_MODE will be needed when DFP-34X-2C2 EEPROM indicates that it supports 2500MB. When I find a spare hard drive and clean up my desk I might install the bnx2 drivers.

[stich86]

@Anime4000

Thanks, I will take a closer look at sfpapp.

@stich86

I'm not sure what I2C-enabled programmer you have.

#READ

i2cget -y 0 0x50 0x0C

i2cget -y 0 0x50 0x3F

#WRITE

i2cset -y 0 0x50 0x0C 0x19

i2cset -y 0 0x50 0x3F 0x??

Obviously it will easier if you have programmer that can read SFP module and fix the checksum.

But there is also a way to do it. The pictures I posted here are from SFP Total wizard. Their SFP adaptors are expensive but you can download the software and load your own dump and then fix the cheksum in it.

https://sfptotal.io/

If everything goes well ethtool should show 2500MB.

edit:

I found this command very helpful in Ubuntu. It converts hexadecimal numbers in the text file to an actual binary file.

xxd -p -r sfpdump.ascii sfpdump.bin

No I don't have any i2c program. But we can write the i2c parameters using diag app form the stick itself

This change should be good for ClearFog/MacchiatoBIN that use sfp native driver and doesn't need the quirk to force 2500BaseX

Is their SFP app free to use?

On the BCM57810s card you should need the patched driver because native one doesn't support 2500BaseX

[tdmadam]

Yes, I have seen i2c under diag, but could not find any information about its syntax. This would be a great solution for the V2801F if it could be made to work. SFP Total Wizard's most functions won't work because it requires their programmer, but some, like viewing binary files and manipulating them, do. Here's my binary file to try

DFP32A0.zip

[stich86]

Yes, I have seen i2c under diag, but could not find any information about its syntax. This would be a great solution for the V2801F if it could be made to work. SFP Total Wizard's most functions won't work because it requires their programmer, but some, like viewing binary files and manipulating them, do. Here's my binary file to try

DFP32A0.zip

there is another user from Unifi community that has managed to create some script to fix i2c broken EEPROM on CarlitoXX. He is also on our Discord channel (gaz), we can discuss there. I'll try in the next week to recovery my V2801F (need to desolder NAND and flash it again)

[tdmadam]

@stich86 Awesome! I have found what I need. I will test it later on V2801F.

[stich86]

@stich86 Awesome! I have found what I need. I will test it later on V2801F.

let me know, we are on Discord if you want to discuss live ;)

[tdmadam]

@stich86 Awesome! I have found what I need. I will test it later on V2801F.

let me know, we are on Discord if you want to discuss live ;)

Family time now but later I will join the channel :)

[tdmadam]

Here's a modified gazor's script to change BitRate value in the SFP EEPROM to 2500MBd. It works for EEPROM in V2801F and DFP-34X-2C2

# Author @gazor
# Small modifications by @tdmadam May2022
# This script will change BitRate value in the SFP EEPROM to 2500MBd
# cc_code at 0x3F is also corrected.

cd /tmp
cat << EOF > BRchg.sh
#!/bin/sh

patched=0
cc_base=0
i=0

# Direct cli commands used for BR
# RTK.0> i2c get port 1 dev 0x50 reg 0xc
# RTK.0> i2c set port 1 dev 0x50 reg 0xc data 0x19

while [[ \$i -lt 64 ]]; do 
  line=\`diag i2c get port 1 dev 0x50 reg \$i | grep dev:\`
  index=\`echo \$line | sed -n 's/dev: 0x50, reg: 0x\([0-9a-f]*\), data: 0x\([0-9a-f]*\)/0x\1/p'\`
  value=\`echo \$line | sed -n 's/dev: 0x50, reg: 0x\([0-9a-f]*\), data: 0x\([0-9a-f]*\)/0x\2/p'\`
  let index=index
  let value=value

  if [[ "\$index" -ne "\$i" ]]; then
    echo "I2C response reported incorrect index \$index, expected \$i"
    exit 1
  fi

  printf "EEPROM[0x%02X] = 0x%02X;\n" \$index \$value
  let newvalue=value

  # patch BR
  if [ "\$i" -eq "12" ] ; then
    printf "The current BitRate value is 0x%02X %4d00Mb\n"  \$value \$value
    let newvalue=0x19
    # To change it back newvalue=0x0C   
  fi

  # CC_BASE at index 63 covers ranges [0, 62]
  if [ \$i -lt 63 ]; then
    let cc_base=cc_base+newvalue # in case we patched
    let cc_base=cc_base%256
  elif [ \$i -eq 63 ]; then
    if [ \$value -eq \$cc_base ]; then
      printf "CC_BASE already correct 0x%X\n" \$cc_base
    else
      printf "Fixing CC_BASE from 0x%X to 0x%X\n" \$value \$cc_base
      let newvalue=cc_base
    fi
  fi

  # make change
  if [[ "\$newvalue" -ne "\$value" ]]; then
    command=\`printf "diag i2c set port 1 dev 0x50 reg 0x%02X data 0x%02X" \$index \$newvalue \`
    eval "\$command" > /dev/null 2>&1   
    patched=1    
  fi

    let i=i+1

done

line=\`diag i2c get port 1 dev 0x50 reg 0xc | grep dev:\`
value=\`echo \$line | sed -n 's/dev: 0x50, reg: 0x\([0-9a-f]*\), data: 0x\([0-9a-f]*\)/0x\2/p'\`
let value=value
 printf "The new BitRate value is 0x%02X %4d00Mb\n"  \$value \$value

EOF

chmod +x BRchg.sh
./BRchg.sh

[tdmadam]

Has anyone maybe tested this on Macchiatobin or Mikrotik to see if setting the BitRate ons SFP Stick from 1500Mb to 2500Mb makes any difference? I don't own a Mikrotik to try this, they are hard to buy now.

Small Form-factor Pluggable (SFP) Transceiver MultiSource Agreement (MSA) page. 35 https://members.snia.org/document/dl/26184

BR, nominal The nominal bit rate (BR, nominal) is specified in units of 100 Megabits per second, rounded off to the nearest 100 Megabits per second. The bit rate includes those bits necessary to encode and delimit the signal as well as those bits carrying data information. A value of 0 indicates that the bit rate is not specified and must be determined from the transceiver technology. The actual information transfer rate will depend on the encoding of the data, as defined by the encoding value.

[stich86]

Has anyone maybe tested this on Macchiatobin or Mikrotik to see if setting the BitRate ons SFP Stick from 1500Mb to 2500Mb makes any difference? I don't own a Mikrotik to try this, they are hard to buy now.

Small Form-factor Pluggable (SFP) Transceiver MultiSource Agreement (MSA) page. 35 https://members.snia.org/document/dl/26184

BR, nominal The nominal bit rate (BR, nominal) is specified in units of 100 Megabits per second, rounded off to the nearest 100 Megabits per second. The bit rate includes those bits necessary to encode and delimit the signal as well as those bits carrying data information. A value of 0 indicates that the bit rate is not specified and must be determined from the transceiver technology. The actual information transfer rate will depend on the encoding of the data, as defined by the encoding value.

not yet.. but for ODI stick, fix LAN_SDS_MODE to 4make it works without any issue.. at the moment auto-neg @ 2.5G is not working, only fixed speed

[jblankfeld]

@stich86 thanks for this discussion I tried the same hack as you did with los = 0, I also added the phylink set quirk and this worked for a while until one day I had no link again :( I guess I'm a bad kernel hacker. Have you managed to solve the link issue setting LAN_SDS_MODE=4 on the stick itself ?

[Headcrabed]

Has anyone maybe tested this on Macchiatobin or Mikrotik to see if setting the BitRate ons SFP Stick from 1500Mb to 2500Mb makes any difference? I don't own a Mikrotik to try this, they are hard to buy now.

Small Form-factor Pluggable (SFP) Transceiver MultiSource Agreement (MSA) page. 35 https://members.snia.org/document/dl/26184

BR, nominal The nominal bit rate (BR, nominal) is specified in units of 100 Megabits per second, rounded off to the nearest 100 Megabits per second. The bit rate includes those bits necessary to encode and delimit the signal as well as those bits carrying data information. A value of 0 indicates that the bit rate is not specified and must be determined from the transceiver technology. The actual information transfer rate will depend on the encoding of the data, as defined by the encoding value.

No, you shouldn't set that to 2500 if you use 2500basex : https://github.com/torvalds/linux/blob/cf1182944c7cc9f1c21a8a44e0d29abe12527412/drivers/net/phy/sfp-bus.c#L148

[Headcrabed]

More information here: https://github.com/Anime4000/RTL960x/discussions/250