Closed ivan-w closed 7 years ago
mcisho
commented
7 years ago Afaik, QETH does not work, and never has worked, with a virtual machine. The magic combination of CHSC replies from Hercules to VM that results in the correct CHSC replies from VM to a virtual machine is still a mystery.
ivan-w
commented
7 years ago On 2/16/2017 1:30 PM, mcisho wrote:
Afaik, QETH does not work, and never has worked, with a virtual machine. The magic combination of CHSC replies from Hercules to VM that results in the correct CHSC replies from VM to a virtual machine is still a mystery.
Not sure...
I didn't have any problem attaching the QETH adapter to a CP VSWITCH, and the QETH responded correctly (I asked for a L2 Uplink, and TAP was opened as expected). The linux bridge did recognize the the interface.
My issue here, at this point, before it actually works - is that it kinda bogs down hercules to the point where all I/Os (I suspect mostly socket oriented I/Os) are suddenly crippling down to a crawl. (My dasds are all shared because my setup is a 2 Member z/VM 6.3 SSI Cluster with shared DASDs - therefore all DASD I/O go through TCP sockets).
I looked at qeth.c, and it all seems to be there...
--Ivan
mcisho
commented
7 years ago There's a lot of qeth.c that is there, but it's definitely not all there.
QETH should, imho, always use a TAP, and if the guest wants to use L3, QETH should deal with it, just like a real OSA does. Always using a TAP means QETH needs to handle ARP and Neighbor Discovery. QETH also needs to handle multiple data devices, as z/OS requires two data device, one for IPv4 and the other for IPv6. I'm not sure QETH will ever work with virtual machines, due to the lack of documentation of CHSC requests/responses. In fact CHSC is the stumbling block to QETH working well with anything other than a Linux guest.
ivan-w
commented
7 years ago On 2/16/2017 8:01 PM, mcisho wrote:
There's a lot of qeth.c that is there, but it's definitely not all there.
QETH should, imho, always use a TAP, and if the guest wants to use L3, QETH should deal with it, just like a real OSA does. Always using a TAP means QETH needs to handle ARP and Neighbor Discovery. QETH also needs to handle multiple data devices, as z/OS requires two data device, one for IPv4 and the other for IPv6. I'm not sure QETH will ever work with virtual machines, due to the lack of documentation of CHSC requests/responses. In fact CHSC is the stumbling block to QETH working well with anything other than a Linux guest.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hercules-390/hyperion/issues/195#issuecomment-280425498, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjMWx6lkK9KKfnpXw7yZgn8tWJRXlIIks5rdJzsgaJpZM4MCcml.
Not sure...
Apparently The implementation of the CHSC instruction seems complete enough that z/VM seems happy with attaching an hercules OSA defined QETH device to a VSWITCH (using the DTCVSW* virtual machines who will bridge between the QETH interface and the virtual switch).
CHSC (Channel Subsystem Call) only deals with configuration and discovery of capabilities - NOT sending/receiving L2 frames. The QETH interface is properly reported as being on a QDIO CHPID (which is what really matters) and when asked properly, the proper TAP (not TUN) device is used (indicating the CHSC worked).
However, once it is activated, anything socket related in hercules slows down to a crawl.... Note : If the underlying TAP device is down, there are no performance issues - hence my intuition that the QETH code may be holding a lock that is not necessary. I try making a couple changes (like reducing the QETH select timeout from 50000 usec to 50 usec... it is better - but still slower than without the QETH interface active).
And another note : ARP (IPv4) and Neighbor discovery (IPv6) on a Layer 2 connection to a VSWITCH will be handled by the z/VM TCP/IP stack.. Offload handling of ARP and ND are only relevant on a Layer 3 interface.
--Ivan
ivan-w
commented
7 years ago On 2/16/2017 8:01 PM, mcisho wrote:
There's a lot of qeth.c that is there, but it's definitely not all there.
QETH should, imho, always use a TAP,
Nope.. If a QETH is configured by the guest to be a layer 3 QETH, a TUN is used.
and if the guest wants to use L3, QETH should deal with it, just like a real OSA does.
If a QETH is configured to be a layer 3 interface, ARP & such are handled by the adapter (or in our case by using a TUN device and having the host OS IP layer handle the ARP, ND, routing and such)
Always using a TAP means QETH needs to handle ARP and Neighbor Discovery.
Nope - it is then handled by the IP stack running in the OS running under hercules.
QETH also needs to handle multiple data devices, as z/OS requires two data device, one for IPv4 and the other for IPv6.
For a Layer 2 QETH it is just a matter of forwarding the frame. The Ethertype portion of the Ethernet frame allows a stack to determine the use of the frame (ARP, IPv4, or IPv6)...
I'm not sure QETH will ever work with virtual machines, due to the lack of documentation of CHSC requests/responses. In fact CHSC is the stumbling block to QETH working well with anything other than a Linux guest.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hercules-390/hyperion/issues/195#issuecomment-280425498, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjMWx6lkK9KKfnpXw7yZgn8tWJRXlIIks5rdJzsgaJpZM4MCcml.
Going further..
I have configured a VSWITCH on my z/VM instance, using a QETH device defined in hercules over pre-established TAP instance connected to a bridge (virtual switch) on my host system.
The z/VM has a Virtual NIC coupled to the VSWITCH, a DEVICE, LINK and HOME statement associated with the Virtual NIC.
Looking at the traffic on the TAP, I DO see ARP requests coming from the z/VM IP stack.. However, the z/VM IP stack doesn't seem to get the response.
13:59:20.090818 ARP, Request who-has 192.168.1.1 tell 192.168.1.126, length 28 13:59:20.092358 ARP, Reply 192.168.1.1 is-at b8:26:6c:59:9e:80, length 46 [... Repeated multiple times ...]
Here is the setup :
[z/VM - TCPIP - VNIC - VSWITCH - DTCVSWX(*) - QETH ] [ Hercules QETH - tapX ] [ Linux Tapx - brX - Physical Eth ] [ Phsyical network - router à 192.168.1.1 ]
(*) the DTCVSWx virtual machines are service virtual machines that handle the actual I/O between a CP Virtual switch and a QETH adapter.
So the QETH implementation seems to be capable enough to SEND Layer 2 frames out from a z/VM OS using a VSWITCH... However :
--Ivan
ivan-w
commented
7 years ago On 2/16/2017 8:01 PM, mcisho wrote:
There's a lot of qeth.c that is there, but it's definitely not all there.
QETH should, imho, always use a TAP, and if the guest wants to use L3, QETH should deal with it, just like a real OSA does. Always using a TAP means QETH needs to handle ARP and Neighbor Discovery. QETH also needs to handle multiple data devices, as z/OS requires two data device, one for IPv4 and the other for IPv6. I'm not sure QETH will ever work with virtual machines, due to the lack of documentation of CHSC requests/responses. In fact CHSC is the stumbling block to QETH working well with anything other than a Linux guest.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hercules-390/hyperion/issues/195#issuecomment-280425498, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjMWx6lkK9KKfnpXw7yZgn8tWJRXlIIks5rdJzsgaJpZM4MCcml.
Further info :
Q VSWITCH DETAIL shows :
Uplink Port: State: Ready PMTUD setting: EXTERNAL PMTUD value: 8992 Trace Pages: 8 RDEV: 0A00.P00 VDEV: 0609 Controller: DTCVSW1 ACTIVE Uplink Port Connection: RX Packets: 0 Discarded: 88 Errors: 0 TX Packets: 165 Discarded: 0 Errors: 0 RX Bytes: 0 TX Bytes: 8118 Device: 0609 Unit: 000 Role: DATA Port: 2049
So the switch DID receive 88 packets from the QETH... But discarded them (for reasons unknown yet).
--Ivan
jphartmann
commented
7 years ago " the DTCVSWx virtual machines are service virtual machines that handle the actual I/O between a CP Virtual switch and a QETH adapter."
Not quite so. The traffic is handled entirely by CP. The virtual machines configures this. It is a standard VM TCPIP stack, but with a specialised configuration file.
Have you looked at the MAC addresses? I think Q NIC DET will show them.
ivan-w
commented
7 years ago On 2/17/2017 3:23 PM, John P. Hartmann wrote:
" the DTCVSWx virtual machines are service virtual machines that handle the actual I/O between a CP Virtual switch and a QETH adapter."
Not quite so. The traffic is handled entirely by CP. The virtual machines configures this. It is a standard VM TCPIP stack, but with a specialised configuration file.
Have you looked at the MAC addresses? I think Q NIC DET will show them.
I don't agree...
The actual QETH devices are attached to the DTCVSWx machines when the virtual switch is defined. The Controlling virtual machines actually handle the I/O to the real (sort to speak) QETH adapter.
16:06:09 Q VSWITCH 16:06:09 VSWITCH SYSTEM VSW01 Type: QDIO Connected: 0 Maxconn: INFINITE 16:06:09 PERSISTENT RESTRICTED ETHERNET Accounting: OFF 16:06:09 USERBASED LOCAL 16:06:09 VLAN Unaware 16:06:09 MAC address: 02-00-01-00-00-01 MAC Protection: Unspecified 16:06:09 IPTimeout: 5 QueueStorage: 8 16:06:09 Isolation Status: OFF VEPA Status: OFF 16:06:09 Uplink Port: 16:06:09 State: Ready 16:06:09 PMTUD setting: EXTERNAL PMTUD value: 8992 Trace Pages: 8 16:06:09 RDEV: 0A00.P00 VDEV: 0600 Controller: DTCVSW2 ACTIVE
16:06:16 Q A00-A02 16:06:16 OSA 0A00 ATTACHED TO DTCVSW2 0600 DEVTYPE OSA CHPID 0A OSD 16:06:17 OSA 0A01 ATTACHED TO DTCVSW2 0601 DEVTYPE OSA CHPID 0A OSD 16:06:17 OSA 0A02 ATTACHED TO DTCVSW2 0602 DEVTYPE OSA CHPID 0A OSD
The switch configuration in SYSTEM CONFIG is :
DEFINE VSWITCH VSW01 QDIO UPLINK RDEV A00 ETHERNET
(The attach for A00-A02 is done by CP when the VSWITCH is defined).
And... when TCPIP is logged on is started, after the following commands are issued :
DEF NIC A00 QDIO (done in the CP Directory) In TCPIP's PROFILE EXEC :
SET VSWITCH VSW01 GRANT TCPIP COUPLE A00 SYSTEM VSW01
16:07:23 Q VSWITCH DET 16:07:23 VSWITCH SYSTEM VSW01 Type: QDIO Connected: 1 Maxconn: INFINITE 16:07:23 PERSISTENT RESTRICTED ETHERNET Accounting: OFF 16:07:23 USERBASED LOCAL 16:07:23 VLAN Unaware 16:07:23 MAC address: 02-00-01-00-00-01 MAC Protection: Unspecified 16:07:23 IPTimeout: 5 QueueStorage: 8 16:07:23 Isolation Status: OFF VEPA Status: OFF 16:07:23 Uplink Port: 16:07:23 State: Ready 16:07:23 PMTUD setting: EXTERNAL PMTUD value: 8992 Trace Pages: 8 16:07:23 RDEV: 0A00.P00 VDEV: 0600 Controller: DTCVSW2 ACTIVE 16:07:23 Uplink Port Connection: 16:07:23 RX Packets: 0 Discarded: 1 Errors: 0 16:07:23 TX Packets: 6 Discarded: 0 Errors: 0 16:07:23 RX Bytes: 0 TX Bytes: 484 16:07:23 Device: 0600 Unit: 000 Role: DATA Port: 2049 16:07:23 Partner Switch Capabilities: Unavailable 16:07:23 Adapter Connections: Connected: 1 16:07:23 Adapter Owner: TCPIP NIC: 0A00.P00 Name: OSA1 Type: QDIO 16:07:23 RX Packets: 0 Discarded: 0 Errors: 0 16:07:23 TX Packets: 4 Discarded: 0 Errors: 0 16:07:23 RX Bytes: 0 TX Bytes: 292 16:07:23 Device: 0A00 Unit: 000 Role: DATA Port: 0001 16:08:32 Options: Ethernet Broadcast 16:08:32 Unicast MAC Addresses: 16:08:32 02-00-01-00-00-02 16:08:32 Multicast MAC Addresses: 16:08:32 01-00-5E-00-00-01 16:08:32 33-33-00-00-00-01 16:08:32 33-33-FF-00-00-02 hercules.cnf : A00.3 QETH ifname tap2 Linux :
bridge name bridge id STP enabled interfaces br0 8000.000c29f8e34e no eth0 tap0 tap1 tap2 tap3
I DO see traffic going from the TCPIP->VSWITCH->DTC->QETH->hercules->tap2 adapter, but the traffic from tap2 to hercules (and then QETH.. etc..) seem to be dropped.
I see the traffic coming from the z/VM TCPIP VM stack using tcpdump on the tap2 interface... I see traffic going TO that device... But it is getting dropped...
... But mostly - when setting QETH debug I see : ...
15:05:03 HHC03991D 0:0A00 QETH: Adapter Interrupt 15:05:03 15:05:03 HHC03991D 0:0A00 QETH: Input dropped (No available buffers) 15:05:03
And a lot of those...
--Ivan
ivan-w
commented
7 years ago On 2/17/2017 3:23 PM, John P. Hartmann wrote:
" the DTCVSWx virtual machines are service virtual machines that handle the actual I/O between a CP Virtual switch and a QETH adapter."
Not quite so. The traffic is handled entirely by CP. The virtual machines configures this. It is a standard VM TCPIP stack, but with a specialised configuration file.
Have you looked at the MAC addresses? I think Q NIC DET will show them.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hercules-390/hyperion/issues/195#issuecomment-280662806, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjMW5jFbitvJDJWfQrvohHZIZDCBZXwks5rda1_gaJpZM4MCcml.
Oh... And one other thing...
When I tried using the QETH tri-devices directly from the TCPIP virtual machine without going through a VSWITCH, that didn't go well... (The interface wouldn't even start).
--Ivan
PS : My issue is not having an IPv4 connectivity... Getting LCS to work was easy enough ;)
ivan-w
commented
7 years ago On 2/17/2017 3:23 PM, John P. Hartmann wrote:
" the DTCVSWx virtual machines are service virtual machines that handle the actual I/O between a CP Virtual switch and a QETH adapter."
Not quite so. The traffic is handled entirely by CP. The virtual machines configures this. It is a standard VM TCPIP stack, but with a specialised configuration file.
Have you looked at the MAC addresses? I think Q NIC DET will show them.
Anyway, that wasn't the point !
Once the QETH device is activated (even if it doesn't work well), operations on hercules become VERY sluggish... (even anything that has NOTHING to do with the QETH adapter).
--Ivan
ivan-w
commented
7 years ago More on my QETH performance issue (not related whether it works or not.)
When an OSA adapter is active, it seems (per looking at a bunch of PTT traces and whatnot), when channel.c presents an I/O interrupt, the CPU(s) are not awoken.
The I/O interrupt occurs when the CPU is awoken by something else (like a timer interrupt, or a SYSG... SYSG isn't affected).... On z/VM, this means it can take an indefinite amount of time (I've seen latencies of around 1 second).
19:11:17 HHC90021I ../../src/hyperion/cpu.c:1514 19:10:55.700131 00007ff830d53700 wait before 00007ff8330daea0 00007ff82c001998 ...
*** Present I/O Interrupt
19:11:17 HHC90021I ../../src/hyperion/channel.c:3824 19:10:56.360728 00007ff830f56700 lock before 0000000001b6f040 0000000000000000 19:11:17 HHC90021I ../../src/hyperion/channel.c:3824 19:10:56.360728 00007ff830f56700 lock after 0000000001b6f040 0000000000000000 0
Attention message (tracing is enabled on the console
19:11:17 HHC90021I ../../src/hyperion/logmsg.c:512 19:10:56.360740 00007ff830f56700 lock before 00007ff83330c158 0000000000000000 19:11:17 HHC90021I ../../src/hyperion/logmsg.c:512 19:10:56.360740 00007ff830f56700 lock after 00007ff83330c158 0000000000000000 0 19:11:17 HHC90021I ../../src/hyperion/logmsg.c:516 19:10:56.360740 00007ff830f56700 unlock 00007ff83330c158 0000000000000000 0 19:11:17 HHC90021I ../../src/hyperion/logger.c:685 19:10:56.360810 00007ff830f56700 lock before 00007ff83330c108 0000000000000000 19:11:17 HHC90021I ../../src/hyperion/logger.c:685 19:10:56.360810 00007ff830f56700 lock after 00007ff83330c108 0000000000000000 0 19:11:17 HHC90021I ../../src/hyperion/logger.c:686 19:10:56.360811 00007ff830f56700 broadcast 0000000000000000 00007ff83330c120 0 19:11:17 HHC90021I ../../src/hyperion/logger.c:687 19:10:56.360811 00007ff830f56700 unlock 00007ff83330c108 0000000000000000 0
*** Later on... (1.1+ seconds later...)
19:11:17 HHC90021I ../../src/hyperion/clock.c:939 19:10:57.488085 00007ff830c52700 lock before 00007ff8330da1c8 0000000000000000 19:11:17 HHC90021I ../../src/hyperion/clock.c:939 19:10:57.488085 00007ff830c52700 lock after 00007ff8330da1c8 0000000000000000 0 19:11:17 HHC90021I ../../src/hyperion/clock.c:953 19:10:57.488085 00007ff830c52700 unlock 00007ff8330da1c8 0000000000000000 0 19:11:17 HHC90021I ../../src/hyperion/hinlines.h:582 19:10:57.488086 00007ff830c52700 lock before 00007ff8330daea0 0000000000000000 19:11:17 HHC90021I ../../src/hyperion/hinlines.h:582 19:10:57.488086 00007ff830c52700 lock after 00007ff8330daea0 0000000000000000 0
*** Clock awakens CPU
19:11:17 HHC90021I ../../src/hyperion/hinlines.h:506 19:10:57.488100 00007ff830c52700 signal 0000000000000000 00007ff82c001998 0 19:11:17 HHC90021I ../../src/hyperion/hinlines.h:607 19:10:57.488100 00007ff830c52700 unlock 00007ff8330daea0 0000000000000000 0
*** CPU awakens..
19:11:17 HHC90021I ../../src/hyperion/cpu.c:1514 19:10:57.488183 00007ff830d53700 wait after 00007ff8330daea0 00007ff82c001998 0
*** And finally (after a bunch of thing occur and get dequeued)...
19:11:17 HHC90021I ../../src/hyperion/cpu.c:1063 19:10:57.490053 00007ff830d53700 *IOINT 0000000000010013 0000000000e57720 00000000
Looking to see what occurs in qeth/qdio that prevents awakening CPUs when an I/O interrupt occurs - even on something totally unrelated.
Also confirmed ... If I have a single CPU and it is running a continuous non-wait PSW (running a rexx loop in CMS), the slowdown no longer shows.
--Ivan
On 2/17/2017 3:23 PM, John P. Hartmann wrote:
" the DTCVSWx virtual machines are service virtual machines that handle the actual I/O between a CP Virtual switch and a QETH adapter."
Not quite so. The traffic is handled entirely by CP. The virtual machines configures this. It is a standard VM TCPIP stack, but with a specialised configuration file.
Have you looked at the MAC addresses? I think Q NIC DET will show them.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hercules-390/hyperion/issues/195#issuecomment-280662806, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjMW5jFbitvJDJWfQrvohHZIZDCBZXwks5rda1_gaJpZM4MCcml.
ivan-w
commented
7 years ago Found the reason (although not the root cause) of the problem.
When the OSA adapter is active, at some point, an I/O interrupt is made pending to the CPU. But the condition is never cleared.
Therefore, when an another I/O interrupt presents itself on another device and is presented to the CPU(s), the CPU is not woken up because an interrupt is already pending.
That's what shows in ON_IC_IOPENDING. When the OSA adapter is active, there is always a pending interrupt and WAKEUP_CPU is never invoked for I/O interrupts. However, if the CPU wakes up for another reason (or simply because it is not in a wait state), the CPU detects a regular (non QDIO) I/O interrupt is pending, presents the interrupt, thus allowing the TSCH to be issued to dequeue the SCSW and the I/O to proceed.
It may be related to QDIO thin interrupts, but I am not proficient enough with QDIO to figure it out... Yet ;)
--Ivan
On 2/17/2017 3:23 PM, John P. Hartmann wrote:
" the DTCVSWx virtual machines are service virtual machines that handle the actual I/O between a CP Virtual switch and a QETH adapter."
Not quite so. The traffic is handled entirely by CP. The virtual machines configures this. It is a standard VM TCPIP stack, but with a specialised configuration file.
Have you looked at the MAC addresses? I think Q NIC DET will show them.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hercules-390/hyperion/issues/195#issuecomment-280662806, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjMW5jFbitvJDJWfQrvohHZIZDCBZXwks5rda1_gaJpZM4MCcml.
ivan-w
commented
7 years ago I think I fixed it ;)
The issue is that in channel.c, when the I/O interrupt is prepared to be presented (present_io_interrupt() doesn't actually perform the PSW switch) - it indicates the device now needs a TSCH to clear the interrupt (dev->tschpending). This apparently does not apply for QDIO Thin Interrupts (the actual point of QDIO Thin Interrupts). Therefore, the interrupt was never off the subchannel and it couldn't present any other interrupt to indicate the change in the QDIO Queue/Buffer status. Furthermore since there was already an interrupt pending clearing, the channel code wouldn't present any further interrupts.
With this change, I managed to have the z/VM TCPIP stack talk to a CP VSWITCH and to the outside world.
**** Sample output ***
netstat devlink
VM TCP/IP Netstat Level 630 TCP/IP Server Name: TCPIP
Device OSAQDIO1 Type: OSD Status: Ready
Queue size: 0 CPU: 0 Address: 0A00 Port name: OSA1
Link ETH0 Type: QDIOETHERNET Port number: 0
Transport Type: Ethernet MAC: 02-00-01-00-00-05
Speed: 1000000000
BytesIn: 2404 BytesOut: 732
Forwarding: Enabled MTU: 8992 IPv6: Enabled
IPv4 Path MTU Discovery: Disabled
IPv4 VIPA ARP
IPv6 VIPA ND
Multicast Group Members
224.0.0.1 1
FF02::1:FF00:5 1
FF02::1 1 Ready; T=0.05/0.07 12:18:21
ping 192.168.1.1
Ping Level 630: Pinging host 192.168.1.1. (My internet router at home)
Enter #CP EXT to interrupt.
PING: Ping #1 response took 0.010 seconds. Successes so far 1.
Ready; T=0.11/0.24 12:18:31
q vswitch det
VSWITCH SYSTEM VSW01 Type: QDIO Connected: 1 Maxconn: INFINITE
PERSISTENT RESTRICTED ETHERNET Accounting: OFF
USERBASED LOCAL
VLAN Unaware
MAC address: 02-00-01-00-00-01 MAC Protection: Unspecified
IPTimeout: 5 QueueStorage: 8
Isolation Status: OFF VEPA Status: OFF
Uplink Port:
State: Ready
PMTUD setting: EXTERNAL PMTUD value: 8992 Trace Pages: 8
RDEV: 0A00.P00 VDEV: 0600 Controller: DTCVSW2 ACTIVE
Uplink Port Connection:
RX Packets: 25 Discarded: 483 Errors: 0
TX Packets: 25 Discarded: 0 Errors: 0
RX Bytes: 1978 TX Bytes: 1850
Device: 0600 Unit: 000 Role: DATA Port: 2049
Partner Switch Capabilities: Unavailable
Adapter Connections: Connected: 1
Adapter Owner: TCPIP NIC: 0A00.P00 Name: OSA1 Type: QDIO
RX Packets: 46 Discarded: 0 Errors: 0
TX Packets: 11 Discarded: 0 Errors: 0
RX Bytes: 6582 TX Bytes: 850
Device: 0A00 Unit: 000 Role: DATA Port: 0004
Options: Ethernet Broadcast
Unicast MAC Addresses:
02-00-01-00-00-05 IP: 192.168.1.126
Multicast MAC Addresses:
01-00-5E-00-00-01
33-33-00-00-00-01
33-33-FF-00-00-05
Ready; T=0.01/0.01 12:19:32
netstat home
VM TCP/IP Netstat Level 630 TCP/IP Server Name: TCPIP
IPv4 Home address entries:
Address Subnet Mask Link VSWITCH
192.168.1.126 255.255.255.0 ETH0
IPv6 Home address entries:
Link: ETH0
Address: FE80::1FF:FE00:5
Flags: Autoconfigured
DAD State: Address Passed DAD
Origin: Link-local
Link: ETH0
Address: 2A01:CB19:678:2500::1FF:FE00:5
Flags: Autoconfigured
DAD State: Interface ID Passed DAD
Origin: Received Router Advertisement
Ready; T=0.05/0.08 12:21:58
ping ns3.nic.fr
Ping Level 630: Pinging host NS3.NIC.FR (2001:660:3006:1::1:1).
Enter #CP EXT to interrupt.
PING: Ping #1 response took 0.039 seconds. Successes so far 1.
Ready; T=0.12/0.20 12:22:34
So I have Layer 2 working under z/VM... It's both IPv4 and IPv6 capable....
I have a couple of adjustments to apply before I commit a patch.
--Ivan
GKevinGodwin610
commented
4 years ago Hi @ivan-w ,
Sorry to re-open a VERY old thread, but I am having a real issue around getting QETH on Z/VM 5.3 Eval working with Hercules on a Ubuntu 18.04 linux machine.
I can get the V/VM to ipl without errors etc and I have the hercules.conf setup right... I think...
The hercules (v 4.3) console shows this output when ipl is run:
HHC03800I 0:0A01 QETH: Adapter mode set to Layer 2
HHC00901I 0:0A01 QETH: Interface tap1, type TAP opened
HHC03997I 0:0A01 QETH: tap1: using MAC address 0E:D6:F5:DA:58:E8
HHC03997I 0:0A01 QETH: tap1: using MTU 1500
and on the linux console the ifconfig shows the following:
tap1: flags=579<UP,BROADCAST,RUNNING,ALLMULTI> mtu 1500 inet6 fe80::cd6:f5ff:feda:58e8 prefixlen 64 scopeid 0x20 ether 0e:d6:f5:da:58:e8 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 185 bytes 24512 (24.5 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
I then have both my physical ethernet port and the tap1 in a bridge as below:
bridge name bridge id STP enabled interfaces bridge0 8000.0ed6f5da58e8 no enp5s0 tap1
But in Z/VM I cannot ping anything on the LAN and from the LAN I cannot ping the IP from within the same LAN that I have configured on the Z/VM.
I am NOT a Z/VM expert at all and this is the reason I am trying to get this working... so I can learn how to use Z/VM.
In Z/VM the Q VSWITCH DET output show this:
q vswitch det DMSDCS1083E Saved segment CMSPIPES does not exist DMSDCS1083E Saved segment CMSPIPES does not exist DMSDCS1083E Saved segment CMSVMLIB does not exist DMSACC724I 191 replaces A (191) Ready; T=0.21/0.29 06:31:27 06:31:27 VSWITCH SYSTEM VSW1 Type: VSWITCH Connected: 1 Maxconn: INFINITE 06:31:27 PERSISTENT RESTRICTED ETHERNET Accounting: OFF 06:31:27 VLAN Unaware 06:31:27 MAC address: 02-00-00-00-00-01 06:31:27 State: Ready 06:31:27 IPTimeout: 5 QueueStorage: 8 06:31:27 RDEV: 0A00 VDEV: 0A00 Controller: DTCVSW2 06:31:27 VSWITCH Connection: 06:31:27 RX Packets: 0 Discarded: 388 Errors: 0 06:31:27 TX Packets: 0 Discarded: 0 Errors: 0 06:31:27 RX Bytes: 0 TX Bytes: 0 06:31:27 Device: 0A02 Unit: 002 Role: DATA Port: 0001 Index: 000 1 06:31:27 Adapter Connections: 06:31:27 Adapter Owner: TCPIP NIC: 0900 Name: UNASSIGNED 06:31:27 RX Packets: 0 Discarded: 0 Errors: 0 06:31:27 TX Packets: 0 Discarded: 0 Errors: 0 06:31:27 RX Bytes: 0 TX Bytes: 0 06:31:27 Device: 0902 Unit: 002 Role: DATA Port: 0065 Index: Non e Ready; T=0.01/0.01 06:31:27
and finally, the ifconfig -all output is:
ifconfig -all LNK2NET inet addr: 192.168.1.120 mask: 255.255.255.0 DOWN MTU: 1500 vdev: 0900 type: QDIO ETHERNET portname: UNASSIGNED ipv4 router type: NONROUTER ipv6: DISABLED cpu: 0 forwarding: ENABLED RX bytes: 0 TX bytes: 0 Ready; T=0.19/0.23 06:33:25
Any advise, guidance or suggestions would be really welcome.
Thanks.
Kevin
Fish-Git
commented
4 years ago The hercules (v 4.3) console ...
Hi Kevin!
If you are using "Hercules 4.3", then you are very likely using SDL Hyperion, not regular Hercules Hyperion.
Please issue the version command on the Hercules panel. If it reports a version string similar to the following:
HHC01413I Hercules version 4.3.9999.9975-SDL-gf99cae09-modified (4.3.9999.9975)then you are definitely using SDL Hyperion 4.x Hercules and not regular Hercules 4.x Hyperion (which is a completely different Hercules).
If that is the case (and I highly suspect it is), then you have reported your problem IN THE WRONG PLACE! :)
You need to report issues (problems) with SDL Hyperion 4.x Hercules here:
Please create a new GitHub Issue using the URL above explaining your problem and hopefully someone with Linux networking experience will be able to help you. (I'm a Windows person and know nothing about Linux networking.)
Hope that helps!
mcisho
commented
4 years ago This is not a Hercules issue, it is a VM configuration issue and really should be opened on an appropriate VM list.
However, in the mean time the main problem is that LNK2NET is DOWN, when it should be UP. You should logon and start your TCPIP virtual machine, and look for any messages that might indicate what is wrong.
Shouldn't the VSWITCH have a PORTNAME?
Ian
Under linux....
Under z/VM 6.3 - when a QDIO/QETH triple subchannel set (tried in Layer 2 mode) is activated (using CREATE VSWITCH UPLINK ETHERNET RDEV
The QETH is created as follows :
0A00.3 QETH ifname tapX
tapX is predefined and bound to a bridge using brctl.
The tapX interface indicated by the ifname is opened (so everything looks fine) as shown by devlist and brctl showstp brX shows the interface is open and forwarding frames.
However, all I/O on ALL devices in hercules take an inordinate amount of time after that (anything before the activation would take sub millisecond of time now takes about a second)... Operations for the system become very sluggish (Logging in a user which prior to the activation of the QDIO/QETH interface took less than a second can take up to a minute after activation)...
Even the hercules console becomes sluggish (even with panrate fast - it kind of jumps all over the place).
Looking at the host CPU activity doesn't show any loop or unusual CPU tie up.
I suspect a lock issue - maybe a lock held by the QETH module while waiting for a frame or a timeout.
I will try to get a PTT trace on that to see if anything odd shows up.