sas server running on mainframe and connecting through saspy in windows

[hrharish111]

Describe the bug On Windows SAS workspace server we connected to SASApp running on port 8591 and successfully able to establish connection. However on Mainframe SAS workspace server, Port 8591 is not available and hence we tried to connect to port 8601 on which this SAS server is running.

below is code to

import saspy
import os,sys
sas = saspy.SASsession(cfgfile="sascfg_personal.py")
print(sas)

below is how my configuration looks in sascfg_personal.py

iomcomzos = {
    'java' : 'C:/Program Files/Eclipse Adoptium/jdk-8.0.332.9-hotspot/bin/java.exe',
    'autoexec' : 'options filesystem=hfs;',
    'iomhost': 'xxxxxxxxxxx',
    'iomport' : 8601,
    'provider': 'sas.iomprovider',
   }

For that it failed and below is the screen print of the error which we are getting.

Screenshots

[tomweber-sas]

Hey, I can help you with this. First, the traceback and the config definition don't match. The traceback shows you're using the COM access method, which isn't supported for connecting to MVS, nor really supported fully for anything. IOM is what you need to be using, which is what the config def has, even though it has the provider key in it. The java key would take precedence, but the traceback shows it using COM.

So, let's start with removing the the provider key. and running the test to see what we get when actually using IOM as needed. Can you tell me what version of SASPy you have? I have to believe the traceback was from a test not having java specified. I see you added the hfs option, so that looks good. The port is whatever port is configured, so it not being the default value isn't a problem. As long as it's the port that the object spawner is listening on for workspace server requests.

As for the encoding, we'll see see about that next when we see what's happening using the IOM access method. I'll need to double check something on that; it's been a while since I added support for connecting to MVS, so I'll need to check something.

Let get a fresh run with IOM and see what we get! Thanks, Tom

[hrharish111]

Hello @tomweber-sas,

Thank for very quick response, and I wanted to let you know that I have followed your suggestion and commented out the provider key. I ran the test and here are the results:

code snippet for sascfg_personal.py

iomcomzos = {
    'java' : 'C:/Program Files/Eclipse Adoptium/jdk-8.0.332.9-hotspot/bin/java.exe',
    'autoexec' : 'options filesystem=hfs;',
    'iomhost': 'xxxxxxxxxxxxxx',
    'iomport' : 8601,
    # 'provider': 'sas.iomprovider',
    }

Below is the screenshot to show port and dns is accessible from terminal

Also, as you requested, I have attached the saspy version for your reference.

Please let me know if there is anything else you need or if you have any further instructions.

Thanks again!

[tomweber-sas]

Hey, that's better. It's using IOM but getting an IOM error trying to make that connection: Invalid physical name. So, I'm guessing the iomhost name being provided isn't what it needs. I don't know that tool you show above is showing or doing. so I can't tell what that implies. Can you try the fully qualified name, or the IP of your MVS system? And, are you sure the port is the port for workspace servers?

The other thing, would be to install the current version of saspy, since what you have is about 7 months old. I don't know it there's anything that could be a problem or not with that, but by running the current code, I can know what it's doing. I have connected to my MVS system, and can debug if needed with the current code.

the best way it to simply issue:

pip uninstall -y saspy pip install saspy

and you'll have the latest.

>>> sas = saspy.SASsession(results='text', cfgname='zos'); sas
SAS Connection established. Subprocess id is 1178080

Access Method         = IOM
SAS Config name       = zos
SAS Config file       = /opt/tom/github/saspy/saspy/sascfg_personal.py
WORK Path             = /tmp/SAS_util00010501075B_DEVA/SAS_util00020501075B_DEVA/
SAS Version           = 9.04.01M7P08052020
SASPy Version         = 5.2.3
Teach me SAS          = False
Batch                 = False
Results               = text
SAS Session Encoding  = open_ed-1047
Python Encoding value = cp1047
SAS process Pid value = BCI5DEM

>>> df = sas.sd2df('cars','sashelp')
>>> df
      Make                    Model   Type  Origin DriveTrain     MSRP  Invoice  EngineSize  Cylinders  Horsepower  MPG_City  MPG_Highway  Weight  Wheelbase  Length
0    Acura           RSX Type S 2dr  Sedan    Asia      Front  23820.0  21761.0         2.0        4.0       200.0      24.0         31.0  2778.0      101.0   172.0
1    Acura                  TSX 4dr  Sedan    Asia      Front  26990.0  24647.0         2.4        4.0       200.0      22.0         29.0  3230.0      105.0   183.0
2    Acura                   TL 4dr  Sedan    Asia      Front  33195.0  30299.0         3.2        6.0       270.0      20.0         28.0  3575.0      108.0   186.0
3    Acura               3.5 RL 4dr  Sedan    Asia      Front  43755.0  39014.0         3.5        6.0       225.0      18.0         24.0  3880.0      115.0   197.0
4    Acura  3.5 RL w/Navigation 4dr  Sedan    Asia      Front  46100.0  41100.0         3.5        6.0       225.0      18.0         24.0  3893.0      115.0   197.0
..     ...                      ...    ...     ...        ...      ...      ...         ...        ...         ...       ...          ...     ...        ...     ...
423  Volvo  C70 HPT convertible 2dr  Sedan  Europe      Front  42565.0  40083.0         2.3        5.0       242.0      20.0         26.0  3450.0      105.0   186.0
424  Volvo               S80 T6 4dr  Sedan  Europe      Front  45210.0  42573.0         2.9        6.0       268.0      19.0         26.0  3653.0      110.0   190.0
425  Volvo                  XC90 T6    SUV  Europe        All  41250.0  38851.0         2.9        6.0       268.0      15.0         20.0  4638.0      113.0   189.0
426  Volvo                      V40  Wagon  Europe      Front  26135.0  24641.0         1.9        4.0       170.0      22.0         29.0  2822.0      101.0   180.0
427  Volvo                     XC70  Wagon  Europe        All  35145.0  33112.0         2.5        5.0       208.0      20.0         27.0  3823.0      109.0   186.0

[428 rows x 15 columns]
>>> wcars = sas.df2sd(df)
>>> wcars
Libref  = WORK
Table   = _df
Dsopts  = {}
Results = text

>>> wcars.head()

                                                           The SAS System                       11:49 Wednesday, August 16, 2023   1

                                                                                                          M
                                                              D                      E         H          P
                                                              r                      n    C    o          G           W
                                                              i                      g    y    r     M    _           h
                                                              v               I      i    l    s     P    H           e
                                                      O       e               n      n    i    e     G    i     W     e     L
                              M                       r       T               v      e    n    p     _    g     e     l     e
             M                o                T      i       r       M       o      S    d    o     C    h     i     b     n
      O      a                d                y      g       a       S       i      i    e    w     i    w     g     a     g
      b      k                e                p      i       i       R       c      z    r    e     t    a     h     s     t
      s      e                l                e      n       n       P       e      e    s    r     y    y     t     e     h

       1   Acura   RSX Type S 2dr            Sedan   Asia   Front   23820   21761   2.0   4   200   24   31   2778   101   172
       2   Acura   TSX 4dr                   Sedan   Asia   Front   26990   24647   2.4   4   200   22   29   3230   105   183
       3   Acura   TL 4dr                    Sedan   Asia   Front   33195   30299   3.2   6   270   20   28   3575   108   186
       4   Acura   3.5 RL 4dr                Sedan   Asia   Front   43755   39014   3.5   6   225   18   24   3880   115   197
       5   Acura   3.5 RL w/Navigation 4dr   Sedan   Asia   Front   46100   41100   3.5   6   225   18   24   3893   115   197

>>>

[hrharish111]

Hello @tomweber-sas

I have attached a screenshot of SAS EG that displays my hostname and port. I have been using the same hostname and port in SASpy, but unfortunately, I keep encountering an "Invalid physical name" error when I use IOM. I have tried troubleshooting on my own, but I haven't been successful in resolving the issue.

secondly, do we have any another alternative to find right dns name and port using sas-eg.

Third point is our Metaserver is in windows and there are multiple workspace server, mvsu is one among them which is in mainframe

Thank you for your time and assistance. I look forward to hearing back from you.

[tomweber-sas]

OK, I was just able to configure my MVS system in EG on my pc, so I can validate what I was going to tell you to try, and it worked as expected, so this ought to help. In the config doc is a link to the way to get the iomhost and iomport, and I just verified that worked for me connecting to my mainframe. See this doc

In my server properties, in EG, it looks like what you have except my hostname is a fully qualified name (I typed it in, so it's what I specified). But the host and port are correct for me and resolvable. Your hostname of 4 uppercase letters seems suspicious to me. I'm not sure if that's some kind of alias reference or something else set up in EG that I'm not familiar with. What you could also try is using nslookup hostname from a command prompt on your client to see if the name resolves or not with DNS from that client machine. I still think this is the issue, that the name isn't being resolved.

Hopefully, the output from the proc iomoperate's will have the fully qualified name which is then what would be needed.

[tomweber-sas]

Also, if you have access to SASMC (SAS Management Console), that's just an easier way to look at the metadata configuration, than using the proc. You can navigate to the severs and see their properties. So, if you can connect to the metadata server w/ that, it's easy to navigate and find the info. Or the proc will get you the info too.

[hrharish111]

nslookup is resolving below is the screenshot to show result

We also tried executing to find proc type object and workspace details from sas eg and I got error the same it attached below

please navigate us to find what is missing

Thanking you in advance

[tomweber-sas]

Hey @hrharish111 nslookup appears to resolve the name, so that's good. Your proc iomoperate calls above show you're using that same port as you're trying to use to get a workspace server, and given the message, it seems you connected to one! The proc iomoperate needs to connect to your metadata server; they are querying metadata. SASPy connects to the Object spawner to get a Workspace server created and connected to it. So, the port of OMR is what you need for that proc.

I haven't found that error being an IOM connection error yet either, and I haven't seen that error before. So I'm still not sure what may cause it. Though, as it seems like you have the correct host/port for a workspace server (from the error of your proc calls), there is one TS note I saw that may be a clue. Seems a bit of a long shot, but I wonder if it might have something to do with this. http://support.sas.com/kb/35/225.html It makes me wonder if the configuration of your workspace server may have something making it fail to start, like in that ticket.

It seems, can't be sure, but from what I see above, it seems like you have the correct host and port, and it's something else that's not working. That ticket above about ODS failing depending upon how you allocate an hfs file to the server, make me think of one thing to just try. If we try connecting, setting the results to 'text' then SASPy won't be trying to use the ODS system like it would normally. It's a quick thing to try, though I'm not confident it's the problem. Can you just try the following when connecting and see if it makes any difference; use your existing SASsession(...) but add resutls='text' and see if it makes a difference:

sas = saspy.SASsession(results='text')
sas

I can use ODS on my system. But again, there could be things different between our systems. Here's from a notebook showing ods output, FWTW.

[tomweber-sas]

Also, just to help confirm, can you try the proc iomoperate's with the port for the Metadata server (or look at it w/ SMC), just to follow up on that? I expect to see the host and port you're using, but just to be sure ...

[tomweber-sas]

OK, I think I'm closing in on this one. It may very well be that the work library for your system isn't an hfs filesystem directory. That's a setting that submitting 'options filesystem=hfs;' after the connection won't change. It still need to be changed for other things, but at the minimum, work has to be an HFS dir to begin with (same with USER if that's allocated), which has to be set up before trying to connect. Can you run this from your EG session that connected to this MVS system:

libname work list;
proc options option=filesystem; run;

I think WORK is a bound library and not an hfs directory, which I think (looking at my java code), might actually cause that failure. I don't have a deployment to test that out with. The one I have that I'm connecting to already has work configured to HFS. But I'm guessing yours has a bound library instead. If so, can you test out pointing WORK at HFS and see if we can connect?

[tomweber-sas]

And to be clear about If so, can you test out pointing WORK at HFS and see if we can connect?, that's something an admin would need to do, not an end user, as it's part of the configuration of the Workspace server. Either in metadata or in a config file or even in a DD card in JCL.

[hrharish111]

I have run the above proc code i.e

libname work list;
proc options option=filesystem; run;

[tomweber-sas]

Yes, there we are. So that why it's failing. The requirements for this to work are simply that WORK, and USER if assigned, need to be HFS paths, not bound libraries. And, that you set filesystem =hfs (which can be done after the server is up, while WORK/USER must be done before, of course). See https://sassoftware.github.io/saspy/configuration.html#remote-to-mvs-sas.

So, are you admins able to define another server to use with those settings? If so, the encoding would then be the next thing to think about with that server definition. Since this is the IOM access method, most transcoding is done in the IOM client itself, but there is some that I still need to do in Python. Thus the part about the limited native python EBCDIC encodings (in that same part of the doc).

[hrharish111]

In the meantime, could you please share the sample output for WORK/USER?

libname work list;
pro options option= filesystem; run; 

I would like to provide a reference to our SAS admin for configuration purposes.

I was curious about the possibility of using the Python EBCDIC library (ref https://pypl.org/project/ebcdic/) for handling complete transcoding. However, you mentioned something about managing it from Python. Could you please provide us with an overview of this approach?

[tomweber-sas]

Sure, I'll provide output below, though the path is shown in the posts I provided above (WORK Path). Both UNIX System Services (USS) and Hierarchical File System (HFS) have been around on MVS for at least 25 years; that's when I wrote the interface to it for SAS I/O (SAS files access through the BASE Engine). It's the Unix environment and the Unix Filesystem that are integrated w/ MVS.

As for encoding, that link provided is a bad link, but yes, that actual PyPI ebcdic package seems to provide many of the EBCDIC encodings for Python. Looking back, I don't see what encoding your MVS system is running in. Ours happens to be open_ed-1047 and that's one that's available in that package; looking at it's doc. Since access to MVS uses the IOM access method, most transcoding through the IOM API, done in the Java client. So, I don't have to transcode everything myself between SAS and Python, like I do with the STDIO Access Method. However, for data transfer, I do need to transcode certain characters, like LF (line feed) which is different on EBCDIC (0x15) compared to ASCII encodings (0x10). So, having a matching Python encoding to the SAS encoding is still needed.

Here's a run on my system, for reference, with the WORK and Filesystem info:

tom64-7> python3
Python 3.9.12 (main, Apr  5 2022, 06:56:58)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import saspy
>>> sas = saspy.SASsession(cfgname='zos', authkey='mvs', results='text')
SAS Connection established. Subprocess id is 1270337

>>> sas
Access Method         = IOM
SAS Config name       = zos
SAS Config file       = /opt/tom/github/saspy/saspy/sascfg_personal.py
WORK Path             = /tmp/SAS_util0001030107BC_DEVA/SAS_util0002030107BC_DEVA/
SAS Version           = 9.04.01M7P08052020
SASPy Version         = 5.3.0
Teach me SAS          = False
Batch                 = False
Results               = text
SAS Session Encoding  = open_ed-1047
Python Encoding value = cp1047
SAS process Pid value = SASTPW

>>> sas.submitLOG('''
... libname work list;
... proc options option= filesystem; run;
... ''')

5                                                          The SAS System                           09:48 Wednesday, August 23, 2023

24
25
26         libname work list;
NOTE: Libref=   WORK
      Scope=    IOM ROOT COMP ENV
      Engine=   V9
      Access=   TEMP
      Physical Name= /tmp/SAS_util0001030107BC_DEVA/SAS_util0002030107BC_DEVA
      Owner= SASTPW
      Group= SYS1
      File Size= 8192
      Device number (dev)= 11
      Dir serial number (ino)= 44932
27         proc options option= filesystem; run;

    SAS (r) Proprietary Software Release 9.4  TS1M7

 FILESYSTEM=MVS    Specifies the default file system used when the filename is ambiguous.
NOTE: The PROCEDURE OPTIONS used 0.00 CPU seconds and 35103K.

NOTE: The address space has used a maximum of 1084K below the line and 49008K above the line.

28
29
30
31

6                                                          The SAS System                           09:48 Wednesday, August 23, 2023

32
>>> sas.submitLOG('''
... options filesystem=hfs;
... proc options option= filesystem; run;
... ''')

7                                                          The SAS System                           09:48 Wednesday, August 23, 2023

35
36
37         options filesystem=hfs;
38         proc options option= filesystem; run;

    SAS (r) Proprietary Software Release 9.4  TS1M7

 FILESYSTEM=HFS    Specifies the default file system used when the filename is ambiguous.
NOTE: The PROCEDURE OPTIONS used 0.00 CPU seconds and 35103K.

NOTE: The address space has used a maximum of 1084K below the line and 49008K above the line.

39
40
41
42

8                                                          The SAS System                           09:48 Wednesday, August 23, 2023

43
>>>

[tomweber-sas]

As a reference, the ZOS Companion document: https://go.documentation.sas.com/api/docsets/hosto390/9.4/content/hosto390.pdf has information on assigning WORK to UFS (they changed from referring to the file system as HFS to UFS, talked about on page 7). The WORK library info starts on page 26. Hopefully your systems programmers or SAS admins can define a server to try out using UFS. As for the encoding, just use one that has a python equivalent so it matches.

Let me know what else I can do, Tom

[tomweber-sas]

I see in the track you asked about zFS instead of HFS. I don't have that to try out, but looking it up, it says it's a posix style filesystem and that it's the successor to hfs, so I would expect that ought to work. I just can't test it out to prove it. Tom

[hrharish111]

Thank @tomweber-sas for your support now we have made a change to our physical name path. We have transitioned from using the previous path to a new Unix path. The updated physical name path is now: /u/sas/physicalname/. however now the error message got changed I have attached the error screenshot below for your reference

[tomweber-sas]

Hey @hrharish111, glad you were able to get that set up. The error above is: The launch of the server process failed with and unknown status code which, I haven't seen before. That error is from the IOM client, on the SASPy side, requesting the workspace server. There's no other info on the SASPy side to look at to know why this happened. I would like to confirm your config file to see there's nothing unusual there (I don't expect so). Can you connect to this workspace server from EG? The object spawner log for your connection attempt would be what we would need to see to get more info on why the workspace server could not be launched. A first guess would be perhaps a permission problem with the new library of something related to that; workspace server run under the id of the user, not a server id like stored process servers, so user permissions could be an issue. But without seeing the object spawner log, which should have more info, that's just a guess. Can you attain the spawner log for one of these attempts? And, again, can you connect to this w/ EG? Thanks, Tom

[tomweber-sas]

Were you able to see why the workspace server couldn't start? Something to do with permissions on the dir you pointed WORK at? Or something else; was the Object Spawner log helpful? I see the TS track is now closed also. Did they have any idea of the problem starting the server?

[hrharish111]

we finally have access to the workspace server. It turns out the issue was with the permissions on the path we provided in the SAS config for the physical name. My user didn't have the necessary authorization.

Now that the problem has been resolved, we should be able to proceed with our work on the workspace server without any further issues

I wanted to express my gratitude for your support in resolving this matter. Your assistance is greatly appreciated.

Thank you once again.

[tomweber-sas]

Hey, that's great! I thought permission might be the issue with that. Glad it was easy to address. Which encoding did you end up with? One that was in that package on PyPI? Or did you just use utf-8? Just curious. Glad you're up and running, let me know if you need anything else!

Thanks! Tom