search

SDL-Hercules-390 / hyperion

The SDL Hercules 4.x Hyperion version of the System/370, ESA/390, and z/Architecture Emulator
Other
240 stars 90 forks source link

Multiple OSA IP support Ping to other-subnet VIPA expires in transit #204

Closed rgschmi closed 5 years ago

rgschmi commented 5 years ago

Environment is Windows 7, Hercules with multiple IP support and CTCI-WIN 3.7

My TCPIP config has an OSA with address 192.168.20.12 and a VIPA with address 10.0.0.2. I'm running OSPF, which shows my router as it's neighbor and that it is advertising the VIPA address. However when I ping the VIPA, the ping expires in transit:

EZZ7833I INTERFACE CONFIGURATION
IP ADDRESS      AREA             COST RTRNS TRDLY PRI HELLO  DEAD DB_E*
192.168.20.13   0.0.0.20            1     5     1   1    10    40    40
192.168.20.12   0.0.0.20            1     5     1   1    10    40    40
10.0.0.2        0.0.0.20            1   N/A   N/A N/A   N/A   N/A   N/A

ADVERTISED VIPA ROUTES
10.0.0.2       /255.255.255.255       <== advertising the VIPA
D TCPIP,TCPIP,OMPR,OSPF,NEIGHBOR
EZZ7851I NEIGHBOR SUMMARY 317
NEIGHBOR ADDR   NEIGHBOR ID     STATE  LSRXL DBSUM LSREQ HSUP IFC
192.168.20.100  10.0.0.100        128      0     0     0  OFF LNK3000   <== neighbor router OK
C:\Users\HP>ping 10.0.0.2

Pinging 10.0.0.2 with 32 bytes of data:
Reply from 192.168.20.1: TTL expired in transit.
Reply from 192.168.20.1: TTL expired in transit.
Reply from 192.168.20.1: TTL expired in transit.
Reply from 192.168.20.1: TTL expired in transit.

Ping statistics for 10.0.0.2:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

NOTE: This issue is closely related to Issue #203.

rgschmi commented 5 years ago

I understand the Hercules OSA function a lot better now. I will try the VIPA changes. Now I understand that IP addresses have to be in the OSA OAT in order to be forwarded to z/OS. I can live with that, and yes, I feel DAMN lucky that OSA works as well as it does, especially the multiple address support. I will quit pushing the limits of qeth, and I am happy with the function that is there, now that I know the limitations.

I appreciate the time and effort everyone has put into this and apologize for taking up so much of it.

Fish-Git commented 5 years ago

*** FOLLOWUP / ADDENDUM ***

I have been conversing with Bob offline regarding this issue and we've discovered a few important things that need to be mentioned.

The first thing that needs to be mentioned is that I was wrong about multiple VIPA definitions. Multiple VIPA definitions are allowed and subsequent definitions do not "override" previous definitions:

Fish wrote:

Bob wrote:

While I am perfectly happy keep #204 closed, I have to respectfully disagree with your conclusion for several reasons.

First, I can add many VIPAs to the TCPIP stack dynamically, and each same-subnet one caused a Register guest IP address message on the Hercules panel. In addition I could ping each of the addresses. I even deleted and re-added a VIPA and got the proper Hercules panel messages.

Okay, I guess I was wrong then: the second/subsequent VIPA definition does NOT override (the) previous one(s). I guess the issue is with whether or not the VIPA is within the same SUBNET as the primary OSA adapter or not, as you explain in your next sentence below:

However, if I tried to add a cross-subnet VIPA, there was no Hercules message and the ping failed. After that failure, I could still add more same-subnet VIPAs.

So it's only DIFFERENT SUBNET VIPAs that fail. SAME SUBNET VIPAs seem to work.

And the REASON different subnet VIPAs fail (whereas same subnet VIPAs work) is due to z/OS not issuing its "Register IP Address" request to Hercules.

Which of course is NOT the fault of Hercules or CTCI-WIN. That is to say, the "fault" is with z/OS. If z/OS would issue the "Register IP Address" request for different subnet VIPAs, then they would work (just like they do for same subnet VIPAs which it does issue a "Register IP Address" request for).

So like I said: the "fault" is with z/OS, not Hercules or CTCI-WIN.

And ALSO like I said, the logical next question then becomes: WHY isn't z/OS issuing a "Register IP Address" request for VIPAs that are in a different subnet? Does IBM mention that anywhere? Is it valid to define a VIPA for a given OSA that's in a completely different subnet as the OSA itself?

  The second, perhaps most important thing that needs to be mentioned, is that we have discovered the real reason why different subnet VIPAs were not working (i.e we have discovered the reason why z/OS wasn't issuing any "Register guest IP address" request to Hercules for different subnet VIPAs):

Fish wrote:

And ALSO like I said, the logical next question then becomes: WHY isn't z/OS issuing a "Register IP Address" request for VIPAs that are in a different subnet? Does IBM mention that anywhere?

Yep!

Is it valid to define a VIPA for a given OSA that's in a completely different subnet as the OSA itself? I'm doubting it.

It doesn't say anything about being valid or invalid, but it DOES mention about VIPA IP Addresses being within the same or different subnet.

Because if it was valid, then I would think z/OS would issue the "Register IP Address" request. The simple fact that z/OS is NOT issuing any "Register IP Address" request for different subnet VIPAs proves it.

Which I still believe is true (even though IBM doesn't explicitly say so, it only implies it).

The manual is: SG24-8360-00 "zOS V2R2 - Communications Server - TCPIP Implementation - Volume 1 - Base Functions, Connectivity, and Routing".

In section "6.4.3 Multiple VLANs configuration guidelines" on page 298 ("ARP processing"):

"ARP processing"

"In QDIO mode, the OSA performs all Address Resolution
Protocol (ARP) processing for IPv4.  The z/OS stack informs
the OSA of the IP addresses for which it should perform ARP
processing.  Because the z/OS stack also supports config-
urations where ARPs flow for VIPAs (which one might see
on some flat network configurations by using static routing),
the stack also informs the OSA of the VIPAs for which it
should perform ARP processing.  OSA sends gratuitous ARPs
for these IP addresses during interface takeover scenarios
to provide fault tolerance."

"If the OSA is defined by using DEVICE/LINK statements,
then the stack informs OSA to perform ARP processing for
all VIPAs in the home list, which can result in numerous
unnecessary gratuitous ARPs for VIPAs in an interface
takeover scenario.  However, if you use the IPv4 INTERFACE
statement for IPAQENET, and a subnet mask is configured
(non-0 num_mask_bits) on the IPADDR parameter of the
INTERFACE statement, the stack informs OSA to perform ARP
processing for a VIPA only if the VIPA is configured in
the same subnet as the OSA."

The last sentence above is KEY!

"... the stack informs OSA to perform ARP processing
for a VIPA only if the VIPA is configured in the same
subnet as the OSA."

So while it does not mention anything about it being invalid, it DOES mention that it purposely does NOT issue a "Register IP Address" request for VIPAs that are not in the same subnet!

Which we already know since we can see clear evidence of that.

And as I explained earlier, if neither Hercules nor CTCI-WIN is ever told about a given IP Address (via the "Register IP Address" request we expect), then how and the heck can Hercules or CTCI-WIN be blamed for not responding to pings sent to those IP addresses?! Answer: IT CAN'T!

The "fault" (if you want to call it that) lies with z/OS: when you define a VIPA that's in a different subnet as the OSA, it purposely DOES NOT tell Hercules about it (nor CTCI-WIN as a result).

And THAT'S the problem.

(I guess different-subnet VIPAs need to be managed "manually"??)

  Finally, after writing that to Bob, he wrote back with confirmation of the above, resulting in (not surprisingly) pings to his different subnet VIPAs now suddenly working!:

Bob wrote:

I have not seen that key statement in the IP Configuration Reference (SC27-3651-00) or IP Configuration Guide (SC27-3650-00), but it's damn well true! The text I copied was from the Configuration Guide, Chapter 2, around page 66 or 67.

I removed the mask from IPADDR on the INTERFACE statement and now I can ping 10.0.0.2! I gotta go back to the above manuals and see if I missed something. That's a gross error on my part if I did! The sad part is that I have all four volumes of the TCPIP Implementation series downloaded and use them for reference a lot :( And I've been doing this for a LONG time.

HHC03997I 0:3001 OSA: tun0: not using MAC address 02:00:5e:a3:be:84
HHC03997I 0:3001 OSA: tun0: not using IP address 192.168.20.12
HHC03997I 0:3001 OSA: tun0: not using MTU 1500
HHC03997I 0:3001 OSA: tun0: using MAC address 02:00:5e:a3:be:84
HHC03997I 0:3001 OSA: tun0: using IP address 192.168.20.12
HHC03997I 0:3001 OSA: tun0: using MTU 1500
HHC03997I 0:3001 OSA: tun0: using drive MAC address 96:7a:59:e5:d2:bf
HHC03997I 0:3001 OSA: tun0: using drive IP address fe80::967a:59ff:fee5:d2bf
HHC03805I 0:3001 OSA: tun0: Register guest IP address 192.168.20.12
HHC03805I 0:3001 OSA: tun0: Register guest IP address 10.0.0.2

We can safely put #204 to bed...

So this issue is now OFFICIALLY and completely closed!   :)