opencomputeproject / oom

MIT License
66 stars 29 forks source link

Not able to find eeprom data from x86_64-accton_as5916_54xks-r0 platfrom #16

Closed bacharya closed 3 years ago

bacharya commented 5 years ago

SFP EEPROM file path on accton platform:

root@localhost:~# strace onlpdump -S 2>&1 | egrep 'open.*/sys/' open("/sys/devices/platform/as5916_54xks_sfp/module_present_1", O_RDONLY) = 3 open("/sys/devices/platform/as5916_54xks_sfp/module_eeprom_1", O_RDONLY) = 3

Thanks, Bhishma

donboll commented 5 years ago

Bhishma,

I’m not sure what the issue is here. But it looks like you’re exploring ground I’ve covered many times, so I’m interested…

The OOM github repository basically has a python package that understands SFP/QSFP/QSFP-DD EEPROMs. It can extract or write registers based on keys defined in the package. (You know all this.) There is also the optoe driver, which uses standard Linux i2c calls to implement a sysfs interface to the EEPROM. Simple file open/read/write/close calls are mapped directly to the EEPROM paged memory.

OOM has a shim layer that allows several ways to access EEPROM data (see oom/oomsysfsshim.py, lines 32-43 to see some of the preferred locations and names for the eeprom files). OOM particularly likes accessing EEPROMs via the optoe driver, but other approaches work as well. New shims can be created to access eeproms via other methods. See oom/oomjsonshim.py to see an alternative implementation. I have privately created several more shims for platforms that have very weird i2c infrastructure.

All of this is background. It turns out that the actual i2c implementation, and the use of Linux i2c infrastructure, is dependent on the hardware and the drivers/modules that the hardware vendor provides to the OS. Specifically, Edgecore/Accton, maker of the as5916xks, has provided the drivers/modules that drive this infrastructure, which are kept in the ONL github site and built for ONL on that switch. There are actually a couple dozen accton switches supported in ONL, and more than a dozen other switch vendors also represented there. Check here: https://github.com/opencomputeproject/OpenNetworkLinux/tree/master/packages/platforms/ to see all the vendors, and down each directory to find each switch from each vendor.

There is really no standardization of the methods to get to the EEPROM data. optoe is built into ONL and available to all the vendors, but many/most don’t use it. Accton uses it, but not on all platforms.

Bottom Line: This is not an issue that OOM or optoe can solve. Every platform is different. Some platforms work with OOM, some platforms support optoe, many don’t. In fact, on ONL optoe can often be configured even on platforms that don’t use it, as long as they have i2c devices for each transceiver. However, ‘out of the box’, OOM just can’t find the EEPROM data on many platforms.

But, I have seen several switches, especially accton switches. I can probably help. On the as5916_54xks platform, the actual driver that is accessing the EEPROM is here: https://github.com/opencomputeproject/OpenNetworkLinux/blob/master/packages/platforms/accton/x86-64/as5916-54xks/modules/builds/x86-64-accton-as5916-54xks-sfp.c

Examining it, I can see that it is using IPMI calls, not linux native i2c calls to access the EEPROMs (see line 1517 for the actual call to fetch the data). I’ve never seen that approach before! The comments from line 1491-1504 look like the key to how that data is being accessed. It looks like you can bypass that driver and go straight to ‘ipmitool’ to get the EEPROM data. See line 1583-1598 if you want to write to the EEPROM.

If you want to go through the driver, it looks from the code like there should be files named ‘moduleeeprom’, where ‘n’ is the port number. Ports 1-48 are SFP and 49-54 are QSFP. From the strace you provided, it looks like those files are in /sys/devices/platform/as5916_54xks_sfp. That makes sense from how linux builds the device tree files.

If all those guesses are right, you should be able to directly read and write to EEPROM for each port by accessing the moduleeeprom file for that port. Try:

cd /sys/devices/platform/as5916_54xks_sfp

od –c –N256 module_eeprom_1

od –c –N256 module_eeprom_54

At the least you should be able to spot the Vendor Name and the Serial Number in the output.

And, if that’s right, you can probably tweak the code in oom/oomsysfsshim.py to add this style of naming to its repertoire. All the changes should be in ‘paths_class’, line 24-44, and ‘class ports’, ‘def initports’, line 64-169. That code encapsulates where to find the EEPROM data, and how to name the port, and packs it all into the ports class so the rest of OOM doesn’t have to know.

I don’t have that particular switch, so I can’t actually test my guesses or develop the changes to OOM. But, if you want to try, I’m happy to answer questions and help debug issues.

Good luck.

Don

From: bacharya [mailto:notifications@github.com] Sent: Tuesday, October 22, 2019 1:59 AM To: opencomputeproject/oom oom@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: [opencomputeproject/oom] Not able to find eeprom data from x86_64-accton_as5916_54xks-r0 platfrom (#16)

SFP EEPROM file path on accton platform:

root@localhost:~# strace onlpdump -S 2>&1 | egrep 'open.*/sys/' open("/sys/devices/platform/as5916_54xks_sfp/module_present_1", O_RDONLY) = 3 open("/sys/devices/platform/as5916_54xks_sfp/module_eeprom_1", O_RDONLY) = 3

Thanks, Bhishma

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/opencomputeproject/oom/issues/16?email_source=notifications&email_token=AD6VCJ45B4EWVDEC6UJMJLDQP26EBA5CNFSM4JDMUAK2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HTN4VNQ , or unsubscribe https://github.com/notifications/unsubscribe-auth/AD6VCJ5OHT2PR2GUKXQJ5O3QP26EBANCNFSM4JDMUAKQ . https://github.com/notifications/beacon/AD6VCJYDW2KETPXCXYUI3T3QP26EBA5CNFSM4JDMUAK2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HTN4VNQ.gif

bacharya commented 5 years ago

Hi Don,

As you suggested I had made my changes in the file "oom/oomsysfsshim.py" and able to read eeprom files from "/sys/devices/platform/as5916_54xks_sfp" directory. How can bring my changes here.

Sample output: eeprom file location: root@localhost:~# cat /sys/devices/platform/as5916_54xks_sfp/module_eeprom_1 !g Fiberstore QSFP-4SFP10G-DACA ��LF172JU00611-2 170215 �740-030077 REV 01

sample dump using oom:

Port Name Vendor Type Part # Serial #

port1 Fiberstore SFP QSFP-4SFP10G-DAC F172JU00611-2
port2 Fiberstore SFP SFP-10G-AOC CAI2706060005-1

Thanks, Bhishma

donboll commented 5 years ago

I’m impressed! I assume the changes are very specific to the as5916_54xks. I’d rather not put something that specific into oomsysfsshim. But, as you can see, OOM is designed to have modular shims which can be loaded for particular environments. If you call your shim something specific to the as5916, I’ll commit it to the OOM repository. You might want to strip out the original oomsysfsshim stuff since it won’t be useful on the as5916, or you might want to leave it in.

Identify it as your contribution in the comments, and send it to me. I’ll check it in.

Beware, the world is full of switches that are different. You’re going to need a lot of custom shims :(.

Thanks

Don

From: bacharya [mailto:notifications@github.com] Sent: Thursday, October 24, 2019 10:39 PM To: opencomputeproject/oom oom@noreply.github.com Cc: donboll don@thebollingers.org; Comment comment@noreply.github.com Subject: Re: [opencomputeproject/oom] Not able to find eeprom data from x86_64-accton_as5916_54xks-r0 platfrom (#16)

Hi Don,

As you suggested I had made my changes in the file "oom/oomsysfsshim.py" and able to read eeprom files from "/sys/devices/platform/as5916_54xks_sfp" directory. How can bring my changes here.

Sample output: eeprom file location: root@localhost:~# cat /sys/devices/platform/as5916_54xks_sfp/module_eeprom_1 ��!��g� Fiberstore QSFP-4SFP10G-DACA ��L�F172JU00611-2 170215 �740-030077 REV 01

sample dump using oom:

Port Name Vendor Type Part # Serial #

port1 Fiberstore SFP QSFP-4SFP10G-DAC F172JU00611-2 port2 Fiberstore SFP SFP-10G-AOC CAI2706060005-1

Thanks, Bhishma

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/opencomputeproject/oom/issues/16?email_source=notifications&email_token=AD6VCJ5LTBKPNHVHGQTOT2DQQKA7RA5CNFSM4JDMUAK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECHHXYQ#issuecomment-546208738 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AD6VCJ6AGUQLXCJE5SEYADLQQKA7RANCNFSM4JDMUAKQ .

bacharya commented 5 years ago

Hi Don,

Thanks for your email. Please find attached diff.

If any comments please let me know.

Thanks, Bhishma

On Fri, Oct 25, 2019 at 10:57 PM donboll notifications@github.com wrote:

I’m impressed! I assume the changes are very specific to the as5916_54xks. I’d rather not put something that specific into oomsysfsshim. But, as you can see, OOM is designed to have modular shims which can be loaded for particular environments. If you call your shim something specific to the as5916, I’ll commit it to the OOM repository. You might want to strip out the original oomsysfsshim stuff since it won’t be useful on the as5916, or you might want to leave it in.

Identify it as your contribution in the comments, and send it to me. I’ll check it in.

Beware, the world is full of switches that are different. You’re going to need a lot of custom shims :(.

Thanks

Don

From: bacharya [mailto:notifications@github.com] Sent: Thursday, October 24, 2019 10:39 PM To: opencomputeproject/oom oom@noreply.github.com Cc: donboll don@thebollingers.org; Comment comment@noreply.github.com Subject: Re: [opencomputeproject/oom] Not able to find eeprom data from x86_64-accton_as5916_54xks-r0 platfrom (#16)

Hi Don,

As you suggested I had made my changes in the file "oom/oomsysfsshim.py" and able to read eeprom files from "/sys/devices/platform/as5916_54xks_sfp" directory. How can bring my changes here.

Sample output: eeprom file location: root@localhost:~# cat /sys/devices/platform/as5916_54xks_sfp/module_eeprom_1 ��!��g� Fiberstore QSFP-4SFP10G-DACA ��L�F172JU00611-2 170215 �740-030077 REV 01

sample dump using oom:

Port Name Vendor Type Part # Serial #

port1 Fiberstore SFP QSFP-4SFP10G-DAC F172JU00611-2 port2 Fiberstore SFP SFP-10G-AOC CAI2706060005-1

Thanks, Bhishma

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/opencomputeproject/oom/issues/16?email_source=notifications&email_token=AD6VCJ5LTBKPNHVHGQTOT2DQQKA7RA5CNFSM4JDMUAK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECHHXYQ#issuecomment-546208738> , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AD6VCJ6AGUQLXCJE5SEYADLQQKA7RANCNFSM4JDMUAKQ> .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/opencomputeproject/oom/issues/16?email_source=notifications&email_token=AEGGQWS4POH4ORB6DV2KYS3QQMT7XA5CNFSM4JDMUAK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECJAPIQ#issuecomment-546441122, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEGGQWT3EG6HCMQUHUBCU4DQQMT7XANCNFSM4JDMUAKQ .

donboll commented 5 years ago

Umm… no attachment. Maybe github strips attachments? Try sending it to me directly: don@thebollingers.org mailto:don@thebollingers.org .

Thanks

Don

From: bacharya [mailto:notifications@github.com] Sent: Sunday, October 27, 2019 1:28 AM To: opencomputeproject/oom oom@noreply.github.com Cc: donboll don@thebollingers.org; Comment comment@noreply.github.com Subject: Re: [opencomputeproject/oom] Not able to find eeprom data from x86_64-accton_as5916_54xks-r0 platfrom (#16)

Hi Don,

Thanks for your email. Please find attached diff.

If any comments please let me know.

Thanks, Bhishma

On Fri, Oct 25, 2019 at 10:57 PM donboll <notifications@github.com mailto:notifications@github.com > wrote:

I’m impressed! I assume the changes are very specific to the as5916_54xks. I’d rather not put something that specific into oomsysfsshim. But, as you can see, OOM is designed to have modular shims which can be loaded for particular environments. If you call your shim something specific to the as5916, I’ll commit it to the OOM repository. You might want to strip out the original oomsysfsshim stuff since it won’t be useful on the as5916, or you might want to leave it in.

Identify it as your contribution in the comments, and send it to me. I’ll check it in.

Beware, the world is full of switches that are different. You’re going to need a lot of custom shims :(.

Thanks

Don

From: bacharya [mailto:notifications@github.com] Sent: Thursday, October 24, 2019 10:39 PM To: opencomputeproject/oom <oom@noreply.github.com mailto:oom@noreply.github.com > Cc: donboll <don@thebollingers.org mailto:don@thebollingers.org >; Comment <comment@noreply.github.com mailto:comment@noreply.github.com > Subject: Re: [opencomputeproject/oom] Not able to find eeprom data from x86_64-accton_as5916_54xks-r0 platfrom (#16)

Hi Don,

As you suggested I had made my changes in the file "oom/oomsysfsshim.py" and able to read eeprom files from "/sys/devices/platform/as5916_54xks_sfp" directory. How can bring my changes here.

Sample output: eeprom file location: root@localhost:~# cat /sys/devices/platform/as5916_54xks_sfp/module_eeprom_1 ��!��g� Fiberstore QSFP-4SFP10G-DACA ��L�F172JU00611-2 170215 �740-030077 REV 01

sample dump using oom:

Port Name Vendor Type Part # Serial #

port1 Fiberstore SFP QSFP-4SFP10G-DAC F172JU00611-2 port2 Fiberstore SFP SFP-10G-AOC CAI2706060005-1

Thanks, Bhishma

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/opencomputeproject/oom/issues/16?email_source=notifications https://github.com/opencomputeproject/oom/issues/16?email_source=notifications&email_token=AD6VCJ5LTBKPNHVHGQTOT2DQQKA7RA5CNFSM4JDMUAK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECHHXYQ#issuecomment-546208738 &email_token=AD6VCJ5LTBKPNHVHGQTOT2DQQKA7RA5CNFSM4JDMUAK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECHHXYQ#issuecomment-546208738> , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AD6VCJ6AGUQLXCJE5SEYADLQQKA7RANCNFSM4JDMUAKQ> .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/opencomputeproject/oom/issues/16?email_source=notifications https://github.com/opencomputeproject/oom/issues/16?email_source=notifications&email_token=AEGGQWS4POH4ORB6DV2KYS3QQMT7XA5CNFSM4JDMUAK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECJAPIQ#issuecomment-546441122 &email_token=AEGGQWS4POH4ORB6DV2KYS3QQMT7XA5CNFSM4JDMUAK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECJAPIQ#issuecomment-546441122>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEGGQWT3EG6HCMQUHUBCU4DQQMT7XANCNFSM4JDMUAKQ .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/opencomputeproject/oom/issues/16?email_source=notifications&email_token=AD6VCJZCGOTSS4EF2TPNXJTQQVGJHA5CNFSM4JDMUAK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECKY7GI#issuecomment-546672537 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AD6VCJ76AUASTSM4HBPYWIDQQVGJHANCNFSM4JDMUAKQ . https://github.com/notifications/beacon/AD6VCJ7H4DNRWTBG2R4HWPLQQVGJHA5CNFSM4JDMUAK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECKY7GI.gif

donboll commented 3 years ago

Fixed as part of the Python3 fixes, this platform is now explicitly supported in oom/oomsysfsshim.py.