Original issue 695 created by arlake228 on 2013-02-19T19:01:34.000Z:
Communication with end site follows. They saw issues with ixgbe driver, and DYNES team rebuilt for their use. Future pSPT releases should do the same:
-=-=-=-=-=-=-=-=-=-=-=-=-=-
My recollection is that ixgbe just requires a rebuild with the new kernel.
It should just be a matter of rebuilding the most recent srpm when we
rebuild the kernels.
Cheers,
Aaron
On 2/19/13 7:35 AM, "Jason Zurawski" <zurawski@internet2.edu> wrote:
Gang;
See attached message regarding some testing (via the DYNES effort) of the
pSPT on Dell R610 hardware that requires the ixgbe driver for the 10G
NIC. Since this may come back to bite everyone regarding support on this
hardware, would it make sense to take some of the work that Ben provided
below, and get that rolled into a future RC?
Thanks;
-jason
Begin forwarded message:
From: Benjeman Meekhof <bmeekhof@umich.edu>
Subject: Re: 10G driver problem on FRGP perfSONAR machine
Date: February 18, 2013 1:42:00 PM EST
To: Pete Siemsen <siemsen@ucar.edu>
Cc: Jason Zurawski <zurawski@internet2.edu>, Shawn McKee
<smckee@umich.edu>, Brian Cashman <bsc@internet2.edu>, John Hernandez
<jph@ucar.edu>
I'm sorry, I misread the SPEC file. You do not need to do any manual
copying of .ko files. The rpm install process does that. We do still
need to build an RPM appropriate to the kernel on these machines.
-Ben
On Mon, Feb 18, 2013 at 1:29 PM, Benjeman Meekhof <bmeekhof@umich.edu>
wrote:
Hi Pete,
I apologize in advance that this is more steps than I would like. I
may be able to take care of some for you with more information but
we'll start here.
I've built an ixgbe source RPM (SRPM) which incorporates a patch to
enable the driver to use any brand of SFP module. I've tested it can
build on an el5 type system. Without knowing what enterprise version
and what kernel we are dealing with, you can do the following to
create a binary RPM for your system. You or someone else here can
tell me those variables and I'll see if I can build one specific to
your perfSonar system to save steps 1-4 for you.
Since the kernel packages ixgbe.ko our package would conflict if
we named it ixgbe.ko. We have to manually over-write the old one
(this path is an example, it is /lib/modules/<your running kernel>:
mv /lib/modules/2.6.18-308.16.1.el5/kernel/drivers/net/ixgbe/ixgbe.ko
/lib/modules/2.6.18-308.16.1.el5/kernel/drivers/net/ixgbe/ixgbe.ko.old
mv
/lib/modules/2.6.18-308.16.1.el5/kernel/drivers/net/ixgbe/ixgbe.ko.new
/lib/modules/2.6.18308.16.1.el5/kernel/drivers/net/ixgbe/ixgbe.ko
Sorry it's such a pain. We might actually be better off doing what I
didn't want to do and building a special perfSonar kernel based on the
stock one but incorporating this driver.
Or...buy the right SFPs in the future...?
Let me know if I can assist in any way.
regards,
Ben
On Mon, Feb 18, 2013 at 11:41 AM, Benjeman Meekhof
<bmeekhof@umich.edu> wrote:
We have run into this problem before and as a result compiled an
updated driver for the NIC. Unfortunately I think we only produced a
kernel with the updated module and not a module package. We run this
kernel on our DYNES FDT machines. I would guess we don't want to run
it on the perfsonar boxes since we don't produce the perfsonar
distribution and don't want to be maintaining our own flavor of it.
Let me see about producing an ixgbe driver package (RPM) that has the
patched driver that you can install on your perfsonar box. We will
surely have this problem on other boxes.
-Ben
On Fri, Feb 15, 2013 at 8:03 PM, <siemsen@ucar.edu> wrote:
Thanks for the help with the fan issue. I have a new problem. We
racked one of the FRGP servers in its home in Denver today (we'll do
the other on later). We couldn't get the 10G NIC to work. It got
link, but we got this in /var/log/messages:
Feb 15 15:12:40 perfsonar-bw kernel: ixgbe 0000:04:00.0: failed to
load because an unsupported SFP+ module type was detected.
Feb 15 15:12:40 perfsonar-bw kernel: ixgbe 0000:04:00.0: Reload the
driver after installing a supported module.
This is using the Dell SR SFP that you sent us with the machines.
Have you guys seen this? Perhaps we need to recompile a driver or
give the driver options or something?
We considered working around this by connecting the server to our
Juniper router using a Juniper twinax cable. Unfortunately, the
twinax is in Cheyenne, so that option can't be tested until someone
in our crew happens to make a trip to Cheyenne. That happens about
once a week or so. What do you think of this idea?
Original issue 695 created by arlake228 on 2013-02-19T19:01:34.000Z:
Communication with end site follows. They saw issues with ixgbe driver, and DYNES team rebuilt for their use. Future pSPT releases should do the same:
-=-=-=-=-=-=-=-=-=-=-=-=-=-
My recollection is that ixgbe just requires a rebuild with the new kernel. It should just be a matter of rebuilding the most recent srpm when we rebuild the kernels.
Cheers, Aaron
On 2/19/13 7:35 AM, "Jason Zurawski" <zurawski@internet2.edu> wrote:
Gang;
See attached message regarding some testing (via the DYNES effort) of the pSPT on Dell R610 hardware that requires the ixgbe driver for the 10G NIC. Since this may come back to bite everyone regarding support on this hardware, would it make sense to take some of the work that Ben provided below, and get that rolled into a future RC?
Thanks;
-jason
Begin forwarded message:
From: Benjeman Meekhof <bmeekhof@umich.edu> Subject: Re: 10G driver problem on FRGP perfSONAR machine Date: February 18, 2013 1:42:00 PM EST To: Pete Siemsen <siemsen@ucar.edu> Cc: Jason Zurawski <zurawski@internet2.edu>, Shawn McKee <smckee@umich.edu>, Brian Cashman <bsc@internet2.edu>, John Hernandez <jph@ucar.edu>
I'm sorry, I misread the SPEC file. You do not need to do any manual copying of .ko files. The rpm install process does that. We do still need to build an RPM appropriate to the kernel on these machines.
-Ben
On Mon, Feb 18, 2013 at 1:29 PM, Benjeman Meekhof <bmeekhof@umich.edu> wrote: Hi Pete,
I apologize in advance that this is more steps than I would like. I may be able to take care of some for you with more information but we'll start here.
I've built an ixgbe source RPM (SRPM) which incorporates a patch to enable the driver to use any brand of SFP module. I've tested it can build on an el5 type system. Without knowing what enterprise version and what kernel we are dealing with, you can do the following to create a binary RPM for your system. You or someone else here can tell me those variables and I'll see if I can build one specific to your perfSonar system to save steps 1-4 for you.
Sorry it's such a pain. We might actually be better off doing what I didn't want to do and building a special perfSonar kernel based on the stock one but incorporating this driver.
Or...buy the right SFPs in the future...?
Let me know if I can assist in any way.
regards, Ben
On Mon, Feb 18, 2013 at 11:41 AM, Benjeman Meekhof <bmeekhof@umich.edu> wrote: We have run into this problem before and as a result compiled an updated driver for the NIC. Unfortunately I think we only produced a kernel with the updated module and not a module package. We run this kernel on our DYNES FDT machines. I would guess we don't want to run it on the perfsonar boxes since we don't produce the perfsonar distribution and don't want to be maintaining our own flavor of it.
Let me see about producing an ixgbe driver package (RPM) that has the patched driver that you can install on your perfsonar box. We will surely have this problem on other boxes.
-Ben
On Fri, Feb 15, 2013 at 8:03 PM, <siemsen@ucar.edu> wrote: Thanks for the help with the fan issue. I have a new problem. We racked one of the FRGP servers in its home in Denver today (we'll do the other on later). We couldn't get the 10G NIC to work. It got link, but we got this in /var/log/messages:
Feb 15 15:12:40 perfsonar-bw kernel: ixgbe 0000:04:00.0: failed to load because an unsupported SFP+ module type was detected. Feb 15 15:12:40 perfsonar-bw kernel: ixgbe 0000:04:00.0: Reload the driver after installing a supported module.
This is using the Dell SR SFP that you sent us with the machines.
Have you guys seen this? Perhaps we need to recompile a driver or give the driver options or something?
We considered working around this by connecting the server to our Juniper router using a Juniper twinax cable. Unfortunately, the twinax is in Cheyenne, so that option can't be tested until someone in our crew happens to make a trip to Cheyenne. That happens about once a week or so. What do you think of this idea?
-- Pete