sassoftware / saspy

A Python interface module to the SAS System. It works with Linux, Windows, and Mainframe SAS as well as with SAS in Viya.
https://sassoftware.github.io/saspy
Other
366 stars 149 forks source link

SASsession using stdio on linux hangs waiting for keyboard input #568

Closed weg-jr closed 5 months ago

weg-jr commented 7 months ago

Describe the bug I am having an issue very similar to issue #343 except I am using a terminal and have no autoexec. I am using sas9.4 and saspy 3.7.2 on python 3.11.5. As soon as I initialize a sas session using using saspy the terminal freezes like its trying to read text directly from the terminal keyboard with no other input. This happens when running python or in a script, both cases will halt the program and wait for keyboard input before continuing. The only way to escape this is to ctrl+c which gives the sas termination prompt and then crashes saspy. A __STDERR__ file then appears in my working directory. ctrl+d or submitting endsas; does not stop this connection.

If I run the sample sas call

/apps/SAS/v9.4/SASHome/SASFoundation/9.4/bin/sas_en -fullstimer -nodms -stdio -terminal -nosyntaxcheck -pagesize MAX 

directly from the terminal I get the same behavior, except submitting an endsas statement or using ctrl+d will terminate the session and create the __STDERR__ file in my home directory. This time the file contains the log of whatever I "ran" at the terminal prompt. the return code is 0 for these cases. and I dont see any other file in my working directory.

calling just sas -nodms will give me an interactive sas session.

My sascfg_personal file is pretty basic with only the STDIO connection that points to sas_en. I tried sas_u8 but it has the same behavior.

sascfg_personal:

SAS_config_names=['default']

SAS_config_options = {'lock_down': False,
                      'verbose'  : True,
                      'prompt'   : True
                     }

SAS_output_options = {'output' : 'html5'}       # not required unless changing any of the default

default  = {'saspath'  : '/apps/SAS/v9.4/SASHome/SASFoundation/9.4/bin/sas_en',
        'options'  : ["-fullstimer"]
            }

To Reproduce ( indentifying locations edited with *** as appropriate, )

$ python
Python 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import saspy
>>> printt(saspy.list_configs())>>> print(saspy)
<module 'saspy' from '******/lib/python3.11/site-packages/saspy/__init__.py'>
>>> print(saspy.SAScfg)
******/lib/python3.11/site-packages/saspy/sascfg_personal.py
>>> print(saspy.list_configs())
['******/lib/python3.11/site-packages/saspy/sascfg_personal.py']
>>> sas = saspy.SASsession()
Using SAS Config named: default
(here is where sas takes over the python session)
data _null_; length x $ 4096; 
                   x = resolve('%sysfunc(pathname(work))');  put 'WORKPATH=' x 'WORKPATHEND=';
                   x = resolve('&SYSENCODING');              put 'ENCODING=' x 'ENCODINGEND=';
                   x = resolve('&SYSVLONG4');                put 'SYSVLONG=' x 'SYSVLONGEND=';
                  x = resolve('&SYSJOBID');                 put 'SYSJOBID=' x 'SYSJOBIDEND=';
                  x = resolve('&SYSSCP');                     put 'SYSSCP=' x 'SYSSCPEND=';
               run;
^CException caught!
(back to python)
Processing interrupt
Attn handler Query is

Select:
  C. Cancel the dialog
  T. Terminate the SAS System

Please enter your Response: t
Secondary Query is:

Press 
  Y to terminate this SAS session, 
  N to continue.

Please enter your Response: y
SAS Connection failed. No connection established. Double check your settings in sascfg_personal.py file.

Attempted to run program /apps/SAS/v9.4/SASHome/SASFoundation/9.4/bin/sas_en with the following parameters:['/apps/SAS/v9.4/SASHome/SASFoundation/9.4/bin/sas_en', '-fullstimer', '-nodms', '-stdio', '-terminal', '-nosyntaxcheck', '-pagesize', 'MAX', '']

Try running the following command (where saspy is running) manually to see if you can get more information on what went wrong:
/apps/SAS/v9.4/SASHome/SASFoundation/9.4/bin/sas_en -fullstimer -nodms -stdio -terminal -nosyntaxcheck -pagesize MAX  

No SAS process attached. SAS process has terminated unexpectedly.
Invalid response from SAS on inital submission. printing the SASLOG as diagnostic

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "****/site-packages/saspy/sasbase.py", line 567, in __init__
    self.pyenc = sas_encoding_mapping[self.sascei]
                 ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'No SAS process attached. SAS process has terminated unexpectedly.'

I'm putting the data null statement that you ask for from the other issue. I got it from running the sas command in the traceback directly. Again this doesnt return any output, I need to read the __STDERR__ log file.

WORKPATH=/saswork/SAS_work*** WORKPATHEND=
ENCODING=latin1 ENCODINGEND=
SYSVLONG=9.04.01M8P01182023 SYSVLONGEND=
SYSJOBID=34530 SYSJOBIDEND=
SYSSCP=LIN X64 SYSSCPEND=

Expected behavior I can establish a sas session and interact with it in python instead of having my terminal hijacked by the sas input prompt

Additional context One other potential issue occurs if I select cancel instead of terminate at the sas termination prompt after pressing ctrl+c. Might be an edge case, but we are already in the middle of a traceback so it probably doesn't matter much.

^CException caught!
Processing interrupt
Attn handler Query is

Select:
  C. Cancel the dialog
  T. Terminate the SAS System

Please enter your Response: C
Traceback (most recent call last):
  File "******/lib/python3.11/site-packages/saspy/sasiostdio.py", line 840, in submit
    lst = self.stdout.read1(4096)
          ^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/sipp/dev/jacob393/sand/pyproj/work/2022/saspy_test.py", line 3, in <module>
    sas = saspy.SASsession()
          ^^^^^^^^^^^^^^^^^^
  File "******/lib/python3.11/site-packages/saspy/sasbase.py", line 522, in __init__
    self._io = SASsessionSTDIO(sascfgname=self.sascfg.name, sb=self, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "******/lib/python3.11/site-packages/saspy/sasiostdio.py", line 231, in __init__
    self._startsas()
  File "******/lib/python3.11/site-packages/saspy/sasiostdio.py", line 423, in _startsas
    ll = self.submit("options svgtitle='svgtitle'; options validvarname=any validmemname=extend; ods graphics on;", "text")
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "******/lib/python3.11/site-packages/saspy/sasiostdio.py", line 902, in submit
    ll = self._breakprompt(logcodeo)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "******/lib/python3.11/site-packages/saspy/sasiostdio.py", line 1010, in _breakprompt
    if (response == 'C' or response == 'c') and query.count("C. Cancel") >= 1:
                                                ^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument should be integer or bytes-like object, not 'str'
SAS Connection terminated. Subprocess id was 11083
tomweber-sas commented 7 months ago

Looking at the other issue, I don't see any understanding of what was happening or any resolution. So, what you're describing is simply doing the following, just hangs at that point?

import saspy
sas = saspy.SASsession()
(hung)?

Thanks for showing your config, that looks fine, and running that datastep - that was just to see what kind of SAS session was trying to be connected to. Can you try changing the 'prompt' : True to False in the SAS_config_options in your config file and see if that makes a difference? Prompt should work in a python shell, but I don't have any idea at the moment why it would just hang there. You're on the same linux machine, just a local linux SAS install, right?

Can you see what's happening with the processes from another os shell and see if SAS came up, or if either SAS or Python are consuming cpu or anything? Do you have a SAS config file where the log is redirected or something out of the ordinary like that? And, what do you mean when you said that trying to just run SAS the same way from a shell does the same thing and create the __STDERR__ file in my home directory? I don't know what that is, but make me wonder if you have some configuration (SAS configuration) that's causing this.

Not to say 'it works for me', just showing the same thing I think you're showing, which is as close as I can see:

tom64-7> python3
Python 3.9.12 (main, Apr  5 2022, 06:56:58)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import saspy
>>> sas = saspy.SASsession(cfgname='tdi'); sas
SAS Connection established. Subprocess id is 752851

Access Method         = STDIO
SAS Config name       = tdi
SAS Config file       = /opt/tom/github/saspy/saspy/sascfg_personal.py
WORK Path             = /usr/tmp/SAS_work13E7000B7CD3_tom64-7/
SAS Version           = 9.04.01M8P01182023
SASPy Version         = 5.4.4
Teach me SAS          = False
Batch                 = False
Results               = Pandas
SAS Session Encoding  = utf-8
Python Encoding value = utf_8
SAS process Pid value = 752851

>>> sas.sascfg.SAScfg.tdi
{'saspath': '/tdi/mva-v940m8/usrlibsas/laxno/SASFoundation/9.4/bin/sas_u8', 'options': ['-encoding', 'utf8']}
>>>

I don't know of anything that could be prompting, so I'm thinking it's something with your SAS configuration redirecting output or something, at the moment. Need more info on that.

Tom

weg-jr commented 7 months ago

Looking at the other issue, I don't see any understanding of what was happening or any resolution. So, what you're describing is simply doing the following, just hangs at that point?

that is correct

Can you try changing the 'prompt' : True to False in the SAS_config_options in your config file and see if that makes a difference?

That does not, once it hangs and I send sigint the python error message changes to KeyboardInterrupt: Interupt handling is disabled due to prompting being disabled.

You're on the same linux machine, just a local linux SAS install, right?

yes im ssh'd into a server which is then running local python with a local sas install. there is no sas grid or anything fancy.

Can you see what's happening with the processes from another os shell and see if SAS came up, or if either SAS or Python are consuming cpu or anything?

the sas process gets spawned and does not get terminated even after the error until I type exit() in which I get a message that subprocess was closed.

Do you have a SAS config file where the log is redirected or something out of the ordinary like that?

I dont think so. there are a couple cfg files but I glanced through the /apps/SAS/v9.4/SASHome/SASFoundation/9.4/sasv9.cfg file and didnt see anything. it works as expected when I run in batch mode and if I call just sas -nodbms

And, what do you mean when you said that trying to just run SAS the same way from a shell does the same thing and create the __STDERR__ file in my home directory?

executing the sas string "-fullstimer -nodms -stdio -terminal -nosyntaxcheck -pagesize MAX" directly from my terminal.

It looks like this file is what would would be sent through stderr but it is creating a physical file. I never use interactive sas so I dont know what expected behavior is. I asked a question regarding that sas command on the sas forums and people thought -nodms -stdio -terminal was not a valid combination of options. the sas help docs dont really explain it well so I'm not sure what the expected behavior is when running that line of code. I had someone else test sas with those arguments on a seperate server and in both cases it hangs waiting for input unless I submit a sig code or type "endsas;" before it executes.

create the STDERR file in my home directory

This was a miscommunication on my part, the file gets created in the CWD that I execute python in. It happened to be my home directory for one test.

Not to say 'it works for me', just showing the same thing I think you're showing, which is as close as I can see:

right before here in your example is where it hangs, I dont get SAS Connection estableshed but when python exits() I can see the process ID

I don't know of anything that could be prompting, so I'm thinking it's something with your SAS configuration redirecting output or something, at the moment. Need more info on that.

I have never had to deal with these config files before so I dont know how sas works. I will try to get an administrator to verify which of the config files is actually used when I start an interactive sas session. I have an autoexec that loads some environmental variables in my home directory but I removed it for debugging this. The only logging related vars in the cfg are -LOGAPPLNAME and -LOGCONFIGLOC which points to a xml file. The config looks to be pretty boilerplate.

Thanks,

tomweber-sas commented 7 months ago

Hey, thanks for all of that! Yes, the prompting thing isn't the problem, just ruling things out. I have to believe that saspy isn't getting the log back from SAS and that's why it's 'hung' not completing. We can try that out to see. Would you be up for a Teams meeting, so I can see what you're trying and diagnose things? I think that would be pretty quick. I can send a link/meeting invite. If not, I can walk you through trying something with the debugger so I can then get a confirmation on if it's not getting the log that's the problem. If so, then we can look at config files or se if there's something causing that.

weg-jr commented 7 months ago

I very much appreciate the offer but I can't take a teams call. It sounds like you suspect the issue is on the sas end rather than with how pysas is handling the piped info? I dont want to drag you into solving somebody else's unrelated problem if that is the case.

Part of my trouble debugging is not knowing what I should be expecting. Can you give me an idea of your expected input and output when you run sas -fullstimer -nodms -stdio -terminal -nosyntaxcheck -pagesize MAX? I might be being dumb here but is it supposed to pipe the log, lst, and dataset to the receiving process? I'm assuming log gets sent through stder and lst/data both get sent through stdout?

If I do something like this: echo 'proc print data=sashelp.class(obs=3);run;' | sas -fullstimer -nodms -stdio -terminal -nosyntaxcheck -pagesize MAX

This is the output I get:

                                                           The SAS System                     14:32 Wednesday, November 15, 2023   1

                                          Obs     Name      Sex    Age    Height    Weight

                                            1    Alfred      M      14     69.0      112.5
                                            2    Alice       F      13     56.5       84.0
                                            3    Barbara     F      13     65.3       98.0

Does this match what you get from that sas command?

Again, thanks for your time.

tomweber-sas commented 7 months ago

@weg-jr I understand. One thing I'd like is for you to install the current version of SASPy. Yours is over a couple years old. It's possible this is something that's fixed, though I haven't looked through every change log since then to try to see. Either way, if it doesn't help, at least I can provide you some tweaks in the code to try as debugging, or just debug it live if you're python savvy, then I can tell if it's something like I'm thinking. Can you upgrade to current, 5.4.4 from your 3.7.2 version? easiest is

pip uninstall -y saspy
pip install saspy

Thanks! Tom

tomweber-sas commented 7 months ago

and, as for just running what saspy runs, from the command prompt, running SAS with all those options works but is a little weird. You have to hit enter just to get a prompt and again after you run things, but it works. It's not how you would run linemode for real if you were doing that, but I'm not running in a shell, I'm running programmatically and yes, those options are correct. Been working for years and endless users. Here's the equivalent on my system running it that way:

tom64-7> /tdi/mva-v940m8/usrlibsas/laxno/SASFoundation/9.4/bin/sas_u8 -encoding utf8 -nodms -stdio -terminal -nosyntaxcheck -pagesize MAX
1                                                          The SAS System                   Wednesday, November 15, 2023 04:39:00 PM

NOTE: Unable to open SASUSER.REGSTRY. WORK.REGSTRY will be opened instead.
NOTE: All registry changes will be lost at the end of the session.

WARNING: Unable to copy SASUSER registry to WORK registry. Because of this, you will not see registry customizations during this
         session.
NOTE: Copyright (c) 2023 by SAS Institute Inc., Cary, NC, USA.
NOTE: SAS (r) Proprietary Software 9.4 (TS1M8 MBCS3170)
      Licensed to SAS Institute Inc., Site 1.
NOTE: This session is executing on the Linux 4.18.0-513.5.1.el8_9.x86_64 (LIN X64) platform.

NOTE: Analytical products:

      SAS/STAT 15.3
      SAS/ETS 15.3
      SAS/OR 15.3
      SAS/IML 15.3
      SAS/QC 15.3

NOTE: Additional host information:

 Linux LIN X64 4.18.0-513.5.1.el8_9.x86_64 #1 SMP Fri Sep 29 05:21:10 EDT 2023 x86_64 CentOS Linux release 8.3.2011

You are running SAS 9. Some SAS 8 files will be automatically converted
by the V9 engine; others are incompatible.  Please see
http://support.sas.com/rnd/migration/planning/platform/64bit.html

PROC MIGRATE will preserve current SAS file attributes and is
recommended for converting all your SAS libraries from any
SAS 8 release to SAS 9.  For details and examples, please see
http://support.sas.com/rnd/migration/index.html

This message is contained in the SAS news file, and is presented upon
initialization.  Edit the file "news" in the "misc/base" directory to
display site-specific news and information in the program log.
The command line option "-nonews" will prevent this display.

NOTE: SAS initialization used:
      real time           0.17 seconds
      cpu time            0.07 seconds

1    data a; x=1; run;

2    data a; x=1; run;
NOTE: The data set WORK.A has 1 observations and 1 variables.
NOTE: DATA statement used (Total process time):
      real time           0.03 seconds
      cpu time            0.02 seconds

3    proc print;run;

4    proc print;run;                                                           The SAS System               Wednesday, November 15, 2023 04:39:00 PM   1

                                                              Obs    x

                                                               1     1
NOTE: There were 1 observations read from the data set WORK.A.
NOTE: The PROCEDURE PRINT printed page 1.
NOTE: PROCEDURE PRINT used (Total process time):
      real time           0.12 seconds
      cpu time            0.06 seconds

5
NOTE: SAS Institute Inc., SAS Campus Drive, Cary, NC USA 27513-2414
NOTE: The SAS System used:
      real time           18.58 seconds
      cpu time            0.16 seconds

tom64-7>
weg-jr commented 7 months ago

Thanks for the output, that was very helpful. That is not what I get when running so I'm sure that's going to be part of the issue.

I updated to 5.4.4 and have the same results.

tomweber-sas commented 7 months ago

Ok, well at least we're on the same page now. Can I give you a module with some print statements in it to swap out so I can tell what saspy is getting from SAS (which I'm guessing is nothing, but that's what we need to see). Now that you have the current code, that will be easy. Or, if you know how to use pdb, we could use that to get the same.

tomweber-sas commented 7 months ago

If you can just add these 2 print statements to your sasiostdio.py file in your deployment, then we can see if I'm getting the log from SAS or not. Here's the diff so yo see where they go, and the output I get from it, just running SASsession():

tom64-7> git diff
diff --git a/saspy/sasiostdio.py b/saspy/sasiostdio.py
index 4c6f2e0..1d94716 100755
--- a/saspy/sasiostdio.py
+++ b/saspy/sasiostdio.py
@@ -940,6 +940,7 @@ Will use HTML5 for this SASsession.""")
                while True:
                   try:
                      lst = self.stdout.get_nowait()
+                     print("LST:", lst)
                   except Empty:
                      lst = b''

@@ -954,6 +955,7 @@ Will use HTML5 for this SASsession.""")
                while True:
                   try:
                      log = self.stderr.get_nowait()
+                     print("LOG:", log)
                   except Empty:
                      log = b''
tom64-7> python3
Python 3.9.12 (main, Apr  5 2022, 06:56:58)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import saspy
>>> sas = saspy.SASsession(cfgname='tdi')
LOG: b'1                                                          The SAS System                    Thursday, November 16, 2023 09:23:00 AM\n\n'
LOG: b'NOTE: Copyright (c) 2023 by SAS Institute Inc., Cary, NC, USA. \nNOTE: SAS (r) Proprietary Software 9.4 (TS1M8 MBCS3170) \n      Licensed to SAS Institute Inc., Site 1.\nNOTE: This session is executing on the Linux 4.18.0-513.5.1.el8_9.x86_64 (LIN X64) platform.\n\n\n'
LOG: b'\nNOTE: Analytical products:\n      \n      SAS/STAT 15.3\n      SAS/ETS 15.3\n      SAS/OR 15.3\n      SAS/IML 15.3\n      SAS/QC 15.3\n'
LOG: b'\nNOTE: Additional host information:\n\n Linux LIN X64 4.18.0-513.5.1.el8_9.x86_64 #1 SMP Fri Sep 29 05:21:10 EDT 2023 x86_64 CentOS Linux release 8.3.2011 \n'
LOG: b'\nYou are running SAS 9. Some SAS 8 files will be automatically converted \nby the V9 engine; others are incompatible.  Please see \nhttp://support.sas.com/rnd/migration/planning/platform/64bit.html\n\nPROC MIGRATE will preserve current SAS file attributes and is \nrecommended for converting all your SAS libraries from any \nSAS 8 release to SAS 9.  For details and examples, please see\nhttp://support.sas.com/rnd/migration/index.html\n\n\nThis message is contained in the SAS news file, and is presented upon\ninitialization.  Edit the file "news" in the "misc/base" directory to\ndisplay site-specific news and information in the program log.\nThe command line option "-nonews" will prevent this display.\n\n\n\n'
LOG: b'\nNOTE: SAS initialization used:\n      real time           0.19 seconds\n      cpu time            0.07 seconds\n      '
LOG: b'\n'
LOG: b'\nNOTE: AUTOEXEC processing beginning; file is /opt/tom/github/saspy/saspy/autoexec.sas.\n'
LOG: b'\nNOTE: Libref USER was successfully assigned as follows: \n      Engine:        V9 \n      Physical Name: /opt/tom/github/saspy/saspy/user'
LOG: b'\n\nNOTE: AUTOEXEC processing completed.\n'
LOG: b'\n1    ;*\';*";*/;'
LOG: b"\n2    options svgtitle='svgtitle'; options validvarname=any validmemname=extend; ods graphics on;\n"
LOG: b'3    '
LOG: b'\n4    ;*\';*";*/;%put %upcase(e3969440a681a2408885998500000001);\nE3969440A681A2408885998500000001'
SAS Connection established. Subprocess id is 13268

LOG: b'\n5    ;*\';*";*/;'
LOG: b'\n6    data _null_; length x $ 4096; file STDERR;'
LOG: b"\n7                   x = resolve('%sysfunc(pathname(work))');  put 'WORKPATH=' x 'WORKPATHEND=';"
LOG: b"\n8                   x = resolve('&SYSENCODING');              put 'ENCODING=' x 'ENCODINGEND=';"
LOG: b"\n9                   x = resolve('&SYSVLONG4');                put 'SYSVLONG=' x 'SYSVLONGEND=';\n10                  x = resolve('&SYSJOBID');                 put 'SYSJOBID=' x 'SYSJOBIDEND=';\n11                  x = resolve('&SYSSCP');                     put 'SYSSCP=' x 'SYSSCPEND=';\n12               run;"
LOG: b'\nNOTE: The file STDERR is:\n      Pipe command="<standard error>"\n'
LOG: b'WORKPATH=/usr/tmp/SAS_work3A36000033D4_tom64-7 WORKPATHEND=\n'
LOG: b'ENCODING=utf-8 ENCODINGEND=\nSYSVLONG=9.04.01M8P01182023 SYSVLONGEND=\nSYSJOBID=13268 SYSJOBIDEND=\nSYSSCP=LIN X64 SYSSCPEND=\n\n\nNOTE: 5 records were written to the file STDERR.\n      The minimum record length was 25.\n      The maximum record length was 59.\nNOTE: DATA statement used (Total process time):\n      real time           0.02 seconds\n      cpu time            0.01 seconds\n      \n13   \n14   '
LOG: b'\n15   ;*\';*";*/;%put %upcase(e3969440a681a2408885998500000002);'
LOG: b'\nE3969440A681A2408885998500000002'
LOG: b'\n16   ;*\';*";*/;'
LOG: b"\n17   data _null_; file STDERR; put %upcase('col0REG=');"
LOG: b'\nNOTE: The file STDERR is:\n      Pipe command="<standard error>"\n'
LOG: b'COL0REG=\n\n'
LOG: b'\nNOTE: 1 record was written to the file STDERR.\n      The minimum record length was 8.\n      The maximum record length was 8.\nNOTE: DATA statement used (Total process time):\n      real time           0.00 seconds\n      cpu time            0.01 seconds\n      '
LOG: b"\n18                                  data _null_;\n18 !                                             put %upcase('col0LOG=');run;"
LOG: b'\nCOL0LOG='
LOG: b'\nNOTE: DATA statement used (Total process time):\n      real time           0.00 seconds\n      cpu time            0.00 seconds\n      '
LOG: b'\n19   '
LOG: b'\n20   ;*\';*";*/;%put %upcase(e3969440a681a2408885998500000003);'
LOG: b'\nE3969440A681A2408885998500000003'
>>> sas
Access Method         = STDIO
SAS Config name       = tdi
SAS Config file       = /opt/tom/github/saspy/saspy/sascfg_personal.py
WORK Path             = /usr/tmp/SAS_work3A36000033D4_tom64-7/
SAS Version           = 9.04.01M8P01182023
SASPy Version         = 5.4.4
Teach me SAS          = False
Batch                 = False
Results               = Pandas
SAS Session Encoding  = utf-8
Python Encoding value = utf_8
SAS process Pid value = 13268

>>>

Once we see what we see, then we can figure out the next part. Tom

weg-jr commented 7 months ago

Here is what that message prints:

LOG: b'16.79s - pydevd: Sending message related to process being replaced timed-out after 5 seconds\n'

I will be in meetings for the next several hours but it seems like it is hanging because the log is not getting sent to it from sas. I do have access to an interactive debugger if there is anything else you want to check

tomweber-sas commented 7 months ago

Wow, I've never seen that message before. No real need to debug saspy further, that is clearly the problem. But, that doesn't look like a SAS message, that looks like something from python. But I'm at a loss at the moment. I've not seen anything like this before. Well, maybe some debugging later. I'll let this spin in my head a little and see what we can come up with later. Have fun in meetings :) I'm on multiple other tasks too, but I'll let this spin in the background.

tomweber-sas commented 7 months ago

Just to see what I got, I googled that message. Got a hit, but doesn't seem to make sense. You're just running this in line mode from a Python shell, right? Yeah, that's in the original post. Could there be somethin gin the Python config that's causing this, and not really the SAS config? Just thinking ... Spin ...

tomweber-sas commented 7 months ago

Sorry, this was the google hit: image

weg-jr commented 7 months ago

Whoops, I had switched to my interactive debugger in vs code to try to see where it was failing and then I ran from vs code, and then falsely attributed that print statement to my changes. Which doesn't make sense since there is no "LOG:" prefix to the text string. Thats what I get for rushing...

I dont get any output when I run with those print statements, either before or after I terminate the sas session

I realize I don't actually get any output from those print statements when running directly from the terminal. when I use the debugger that while true block never gets hit.

tomweber-sas commented 7 months ago

That's just strange. So, it sounds like you know how to step through the debugger. That seems the next step. I just do this from a terminal, using pdb. If you can do that, it'll eliminate other things from the equation. Since you never got to the first submit that's trying to run code and get the log back, I wonder if the SAS process ever really started or if there's some other Python thing happening that's problematic.

Here's what I walked through, and I set breakpoints at the places I want to see if you make it to, and what's happening. The last breakpoint is in the access methods submit(), which it seems you're not getting to, else you'd have gotten something from those print statements. Can you just step through this and see where you get to?

tom64-7> python3
Python 3.9.12 (main, Apr  5 2022, 06:56:58)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import saspy, pdb
>>> pdb.set_trace(); sas = saspy.SASsession(cfgname='tdi')
--Call--
> /opt/tom/github/saspy/saspy/sasbase.py(526)__init__()
-> def __init__(self, **kwargs):
(Pdb) l
521         """
522         # SAS Epoch: 1960-01-01
523         SAS_EPOCH = datetime.datetime(1960, 1, 1)
524
525         # def __init__(self, cfgname: str ='', kernel: 'SAS_kernel' =None, saspath :str ='', options: list =[]) -> 'SASsession':
526  ->     def __init__(self, **kwargs):
527             self._loaded_macros    = False
528             self._obj_cnt          = 0

Then just set the following breakpoints and 'c' to each of them. Be sure not to step
though lines between the following 2 BP's as that can tangle up things. From bp 2, 
do 'c' to get to bp 3. You can step through other stuff, but that's where the subprocess
is being created and stepping line by line there and cause a problem. 

2   breakpoint   keep yes   at /opt/tom/github/saspy/saspy/sasiostdio.py:346
3   breakpoint   keep yes   at /opt/tom/github/saspy/saspy/sasiostdio.py:474

(Pdb) b
Num Type         Disp Enb   Where
1   breakpoint   keep yes   at /opt/tom/github/saspy/saspy/sasbase.py:556
2   breakpoint   keep yes   at /opt/tom/github/saspy/saspy/sasiostdio.py:346
3   breakpoint   keep yes   at /opt/tom/github/saspy/saspy/sasiostdio.py:474
4   breakpoint   keep yes   at /opt/tom/github/saspy/saspy/sasiostdio.py:484
5   breakpoint   keep yes   at /opt/tom/github/saspy/saspy/sasiostdio.py:786
(Pdb) c
(Pdb) w
  <stdin>(1)<module>()
  /opt/tom/github/saspy/saspy/sasbase.py(558)__init__()
-> self._io = SASsessionSTDIO(sascfgname=self.sascfg.name, sb=self, **kwargs)
  /opt/tom/github/saspy/saspy/sasiostdio.py(263)__init__()
-> self._startsas()
  /opt/tom/github/saspy/saspy/sasiostdio.py(484)_startsas()
-> ll = self.submit("options svgtitle='svgtitle'; options validvarname=any validmemname=extend; ods graphics on;", "text")
> /opt/tom/github/saspy/saspy/sasiostdio.py(786)submit()
-> def submit(self, code: str, results: str ="html", prompt: dict = None, **kwargs) -> dict:
(Pdb)
weg-jr commented 7 months ago

I see I've made another mistake. Apparently there is a vs code override that disables breakpoints for files you are not the author of, even if you put a breakpoint in that file. After disabling it my breakpoints now get get triggered for both while loops waiting on the log/lst files that you suggested to print here:

                while True:
                   try:
                      lst = self.stdout.get_nowait()
+                     print("LST:", lst)
                   except Empty:
                      lst = b''

Nothing gets printed because it always hits the exception block before the print statement can be called. both lst and log are b''

so the packages is not getting anything piped to it, which points back to sas being misconfigured. This happens even if I type "proc print data=sashelp.class(obs=3);run;" and then step through the breakpoints after several times. I would think it should be getting at least the lst output here though? since when I make the sas call like in my prev comment and pipe that input I do get the lst output even if I dont get the log. But maybe that can be explained by the newlines in python not making there way to sas, so sas would just see "end\n\n\n\n" instead of a newline or something?

tomweber-sas commented 7 months ago

Ok, well that's why I wanted to take vscode outa the picture and run python command line, so it's just python. But, You've verified that we are getting to subimt() and it's not getting anything back from SAS. So that does sound like the issue, like I was expecting. So next is to dig into the SAS side and see why that is. Can you just run this from the comand line and hopefully we can see something: /apps/SAS/v9.4/SASHome/SASFoundation/9.4/bin/sas_en -nodms

and then submit proc options option=config value; run; proc options; run;

Then we'll be able to see what options are set, esp regarding the log, and what config files are used ...

weg-jr commented 7 months ago

incoming wall of text

 NOACCESSIBLECHECK Do not detect and log ODS output that is not accessible.
 NOACCESSIBLEGRAPH Do not create accessible ODS graphics by default.
 NOACCESSIBLEPDF   Do not create accessible PDF files by default.
 NOACCESSIBLETABLE Do not create accessible tables for enabled procedures, by 
                   default.
 ANIMATION=STOP    Specifies whether to start or stop animation.
 ANIMDURATION=MIN  Specifies the number of seconds that each animation frame 
                   displays.
 ANIMLOOP=YES      Specifies the number of iterations that animated images 
                   repeat.
 ANIMOVERLAY       Specifies that animation frames are overlaid in order to 
                   view all frames.
 APPEND=           Specifies an option=value pair to insert the value at the 
                   end of the existing option value.
 APPENDVARLENWARN  Emit PROC APPEND variable length warning.
 APPLETLOC=/apps/SAS/v9.4/SASHome/SASGraphJavaApplets/9.4
                   Specifies the location of Java applets, which is typically 
                   a URL.
 ARMAGENT=         Specifies an ARM agent (which is an executable module or 
                   keyword, such as LOG4SAS) that contains a specific 
                   implementation of the ARM API.
 ARMLOC=ARMLOG.LOG Specifies the location of the ARM log.
 ARMSUBSYS=(ARM_NONE)
                   Specifies the SAS ARM subsystems to enable or disable.
 AUTOCORRECT       Automatically corrects misspelled procedure names and 
                   keywords, and global statement names.
 AUTOEXEC=         Specifies the location of the SAS AUTOEXEC files.
 AUTOSAVELOC=      Specifies the location of the Program Editor auto-saved 
                   file.
 NOAUTOSIGNON      Disables a SAS/CONNECT client from automatically 
                   submitting the SIGNON command remotely with the RSUBMIT 
                   command.
 AZUREAUTHCACHELOC=
                   Path to Azure Login info
 AZURETENANTID=    Azure Tenant ID
 BINDING=DEFAULT   Specifies the binding edge type of duplexed printed output.
 BOMFILE           Writes the byte order mark (BOM) prefix when a 
                   Unicode-encoded file is written to an external file.
 BOTTOMMARGIN=0.000 IN
                   Specifies the size of the margin at the bottom of a 
                   printed page.
 BUFNO=1           Specifies the number of buffers for processing SAS data 
                   sets.
 BUFSIZE=0         Specifies the size of a buffer page for output SAS data 
                   sets.
 BYERR             SAS issues an error message and stops processing if the 
                   SORT procedure attempts to sort a _NULL_ data set.
 BYLINE            Prints the BY line above each BY group.
 BYSORTED          Requires observations in one or more data sets to be 
                   sorted in alphabetic or numeric order.
 NOCAPS            Does not convert certain types of input, and all data 
                   lines, into uppercase characters.
 NOCARDIMAGE       Does not process SAS source code and data lines as 80-byte 
                   records.
 CASAUTHINFO=      Specifies an authinfo or netrc file that includes 
                   authentication information.
 CASDATALIMIT=100M Specifies the maximum number of bytes that can be read 
                   from a file.
 CASHOST=          The CAS server name associated with a CAS session.
 CASLIB=           Specify the default CASLIB name.
 CASNCHARMULTIPLIER=1
                   Specifies a multiplication factor to increase the number 
                   of bytes when transcoding fixed CHAR data.
 CASNWORKERS=ALL   Specify the number of workers to use with a CAS session.
 CASPORT=0         The port associated with a CAS session.
 CASSESSOPTS=      Identify CAS server session options.
 CASTIMEOUT=60     The CAS session timeout in seconds.
 CASUSER=          The userid associated with a CAS session.
 CATCACHE=0        Specifies the number of SAS catalogs to keep open in cache 
                   memory.
 CBUFNO=0          Specifies the number of extra page buffers to allocate for 
                   each open SAS catalog.
 CENTER            Center SAS procedure output.
 CGOPTIMIZE=3      Specifies the level of optimization to perform during code 
                   compilation.
 NOCHARCODE        Does not substitute specific keyboard combinations for 
                   special characters that are not on the keyboard.
 NOCHKPTCLEAN      Does not erase files in the Work library after a batch 
                   program successfully executes in checkpoint mode or 
                   restart mode.
 CLEANUP           Performs automatic continuous cleanup of non-essential 
                   resources in out-of-resource conditions.
 NOCMDMAC          Does not check window environment commands for 
                   command-style macros.
 CMPLIB=           Specifies one or more SAS data sets that contain compiler 
                   subroutines to include during compilation.
 CMPMODEL=BOTH     Specifies the output model type for the MODEL procedure.
 CMPOPT=(NOEXTRAMATH NOMISSCHECK NOPRECISE NOGUARDCHECK NOGENSYMNAMES 
NOFUNCDIFFERENCING SHORTCIRCUIT NOPROFILE NODEBUGHOST NODEBUGPORT)
                   Specifies the type of code-generation optimizations to use 
                   in the SAS language compiler.
 NOCOLLATE         Does not collate multiple copies of printed output.
 COLOPHON=         Specifies the comment text that is included in graphic 
                   stream files.
 COLORPRINTING     Prints in color if color printing is supported.
 COMAMID=TCP       Specifies the communication access method for connecting 
                   client and server sessions across a network.
 COMPRESS=NO       Specifies the type of compression to use for observations 
                   in output SAS data sets.
 NOCONNECTEVENTS   Clients do not receive SAS events propagated from a 
                   SAS/CONNECT server.
 CONNECTMETACONNECTION
                   At sign-on, connects the SAS/CONNECT server to the SAS 
                   Metadata server.
 CONNECTOUTPUT=BUFFERED
                   Specifies whether to send the SAS/CONNECT server log and 
                   list output immediately, or to buffer the output.
 CONNECTPERSIST    Continues a client/server connection after an RSUBMIT 
                   statement has completed.
 CONNECTREMOTE=    Specifies the ID of a specific server session that a 
                   client connects to.
 CONNECTSTATUS     Displays the Transfer Status window during file transfers.
 CONNECTWAIT       Executes RSUBMIT statements synchronously.
 COPIES=1          Specifies the number of copies to print.
 CPUCOUNT=4        Specifies the number of processors that thread-enabled 
                   applications should assume are available for concurrent 
                   processing.
 CPUID             Prints the CPU identification number at the beginning of 
                   the SAS log.
 CSTGLOBALLIB=     Specifies the location of the SAS Clinical Standards 
                   Toolkit global library.
 CSTSAMPLELIB=     Specifies the location of the SAS Clinical Standards 
                   Toolkit sample library.
 DATAPAGESIZE=CURRENT
                   Specifies whether the page size for a data set or utility 
                   file is compatible with SAS 9.3 processing, or is 
                   determined by the current version of SAS.
 DATASTMTCHK=COREKEYWORDS
                   Specifies which SAS statement keywords are prohibited from 
                   being specified as a one-level DATA step name to protect 
                   against overwriting an input data set.
 DATE              Prints the date and time that a SAS program started.
 DATESTYLE=MDY     Specifies the sequence of month, day, and year when 
                   ANYDTDTE, ANYDTDTM, or ANYDTTME informat data is ambiguous.
 NODBFMTIGNORE     Uses the numeric data type in tables.
 DBIDIRECTEXEC     The SQL pass-through facility optimizes the handling of 
                   SQL statements.
 DBSLICEPARM=(THREADED_APPS, 2)
                   Specifies whether SAS procedures, applications, and the 
                   DATA step can read DBMS tables in parallel, and the number 
                   of threads to use to read the DBMS tables.
 DBSRVTP=NONE      Specifies whether SAS/ACCESS engines hold or block the 
                   originating client while making performance-critical calls 
                   to the database.
 DCSHOST=LOCALHOST Specifies the host name of the SAS Document Conversion 
                   Server.
 DCSPORT=7111      Specifies the port number of the SAS Document Conversion 
                   Server.
 DECIMALCONV=COMPATIBLE
                   Specifies the binary to decimal conversion and formatting 
                   methodology.
 DEFLATION=6       Specifies the level of compression for device drivers that 
                   support the Deflate compression algorithm.
 NODETAILS         Does not display additional information when files are 
                   listed in a SAS library.
 DEVICE=           Specifies the device driver to which SAS/GRAPH sends 
                   procedure output.
 DFLANG=ENGLISH    Specifies the language for international date informats 
                   and formats.
 DKRICOND=ERROR    Specifies the error level to report when a variable is 
                   missing from an input data set during the processing of a 
                   DROP=, KEEP=, or RENAME= data set option.
 DKROCOND=WARN     Specifies the error level to report when a variable is 
                   missing from an output data set during the processing of a 
                   DROP=, KEEP=, or RENAME= data set option.
 NODLCREATEDIR     Does not create a directory for the SAS library that is 
                   named in a LIBNAME statement when the directory does not 
                   already exist.
 DLDMGACTION=REPAIR
                   Specifies the type of action to take when a SAS data set 
                   or a SAS catalog is detected as damaged.
 NODMR             Does not invoke a server session for use with a 
                   SAS/CONNECT client.
 NODMS             Starts SAS using an interactive line-mode session.
 NODMSEXP          Starts SAS using an interactive line-mode session.
 DMSLOGSIZE=99999  Specifies the maximum number of rows that the SAS Log 
                   window can display.
 DMSOUTSIZE=2147483647
                   Specifies the maximum number of rows that the SAS Output 
                   window can display.
 DMSPGMLINESIZE=136
                   Specifies the maximum number of characters in a Program 
                   Editor line.
 NODMSSYNCHK       Disables syntax check mode for DATA step and PROC step 
                   processing in the windowing environment.
 DQLOCALE=         Specifies the Data Quality Server ordered list of locales 
                   for data cleansing.
 DQOPTIONS=        Specifies the SAS session parameters for data quality 
                   programs.
 DQSETUPLOC=!SASROOT/misc/dquality/dqsetup.txt
                   Specifies the location of the Quality Knowledge Base root 
                   directory.
 DS2ACCEL=NONE     Provides support for DS2 code pass-through acceleration.
 DS2SCOND=WARN     Specifies the type of message that PROC DS2 generates.
 DSACCEL=NONE      Provides support for code pass-through acceleration.
 DSCAS             Runs the DATA step on the CAS server.
 DSNFERR           Issues an error message and stops processing when a SAS 
                   data set cannot be found.
 NODTRESET         SAS does not update the date and time in the titles of the 
                   SAS log and procedure output file.
 NODUPLEX          Does not print output using duplex (two-sided) printing.
 NOECHOAUTO        Does not write statements that are in the AUTOEXEC file to 
                   the SAS log as they are executed.
 EMAILACKWAIT=30   Specifies the number of seconds to wait for the SMTP 
                   server acknowledgement.
 EMAILAUTHPROTOCOL=NONE
                   Specifies the SMTP e-mail authentication protocol.
 NOEMAILFROM       Does not require the FROM e-mail option when sending 
                   e-mail by using the FILE or FILENAME statements.
 EMAILHOST=LOCALHOST
                   Specifies one or more domain names for SMTP e-mail servers.
 EMAILID=          Specifies the SAS user's logon ID, profile or e-mail 
                   address.
 EMAILPORT=25      Specifies the port number for the SMTP e-mail server that 
                   is specified in the EMAILHOST option.
 EMAILPW=XXXXXXXX  Specifies the password for the e-mail address specified by 
                   the EMAILID option.
 EMAILUTCOFFSET=   For SMTP e-mail sent using the FILENAME statement, 
                   specifies a UTC offset that is used in the Date header 
                   field of the e-mail message.
 NOENCRYPTFIPS     Does not limit SAS/SECURE and SSL security services to use 
                   FIPS 140-2 algorithms.
 ENGINE=V9         Specifies the default access method for SAS libraries.
 NOERRORABEND      Does not end SAS for most errors, issues an error message, 
                   sets OBS=0, and goes into syntax check mode.
 NOERRORBYABEND    Does not end a SAS program when an error occurs in 
                   BY-group processing, issues an error, and continues 
                   processing.
 ERRORCHECK=NORMAL Specifies whether SAS enters syntax-check mode when errors 
                   are found in the LIBNAME, FILENAME,  %INCLUDE, and LOCK 
                   statements.
 ERRORS=20         Specifies the maximum number of observations for which SAS 
                   issues complete error messages.
 EVENTDS=(DEFAULTS)
                   Specifies one or more data sets that define custom holiday 
                   events.
 NOEXPLORER        Does not invoke Explorer and the Program Editor when SAS 
                   starts.
 EXTENDEDDATATYPES=NO
                   Specifies whether SAS processes all supported data types 
                   or converts nontraditional SAS data types to CHAR and 
                   DOUBLE.
 EXTENDOBSCOUNTER=YES
                   Specifies whether to extend the maximum number of 
                   observations in a new SAS data file.
 FILESYNC=HOST     Specifies when operating system buffers that contain 
                   contents of permanent SAS files are written to disk.
 FIRSTOBS=1        Specifies the observation number or external file record 
                   that SAS processes first.
 FMTERR            Issues an error message when a variable format cannot be 
                   found.
 FMTSEARCH=(WORK LIBRARY)
                   Specifies the order in which format catalogs are searched.
 FONTEMBEDDING     Enables font embedding for Universal Printing and 
                   SAS/GRAPH printing.
 FONTRENDERING=FREETYPE_POINTS
                   Specifies whether some SAS/GRAPH devices render fonts by 
                   using the operating system or by using the Free Type 
                   engine.
 FONTSLOC=/apps/SAS/v9.4/SASHome/ReportFontsforClients/9.4
                   Specifies the location of the fonts that are supplied by 
                   SAS. Names the default font file location for registering 
                   fonts that use the FONTREG procedure.
 FORMCHAR=|----|+|---+=|-/\<>*
                   Specifies the default output formatting characters.
 FORMDLIM=         Specifies the character to delimit page breaks in SAS 
                   output for the LISTING destination.
 FORMS=DEFAULT     If forms are used for printing, specifies the default form 
                   to use.
 GRIDINSTALLLOC=   Identifies the location on the machine cluster where the 
                   SAS High-Performance Analytics environment is installed.
 GRIDRSHCOMMAND=   Specifies the path to the executable to use to launch the 
                   SAS High-Performance Analytics environment.
 GSTYLE            Uses ODS styles to generate graphs that are stored as 
                   GRSEG catalog entries.
 GWINDOW           Displays SAS/GRAPH output in the GRAPH window.
 HADOOPPLATFORM=MAPRED
                   Specifies the execution platform for the SAS In-Database 
                   Code Accelerator for Hadoop.
 HELPADDR=         Specifies the address of the remote Help system.
 HELPBROWSER=SAS   Specifies the browser to use for SAS Help and ODS output.
 HELPENCMD         Uses the English version of the keyword list for the 
                   command-line Help.
 HELPHOST=         Specifies the name of the computer where the remote 
                   browser is to send Help and ODS output.
 HELPINDEX=(/help/common.hlp/index.txt /help/common.hlp/keywords.htm 
common.hhk)
                   Specifies one or more index files for SAS Help and 
                   Documentation.
 HELPPORT=0        Specifies the port number for the remote browser client.
 HELPTOC=(/help/helpnav.hlp/navigation.xml /help/common.hlp/toc.htm 
common.hhc)
                   Specifies the table of contents files for the online SAS 
                   Help and Documentation.
 HOSTINFOLONG      Print operating environment information in the SAS log 
                   when SAS starts.
 HPRUSERFILES      Allow HPRISK users to copy files to TKGrid cluster
 HTTPSERVERPORTMAX=0
                   Specifies the highest port number that can be used by the 
                   SAS HTTP server for remote browsing.
 HTTPSERVERPORTMIN=0
                   Specifies the lowest port number that can be used by the 
                   SAS HTTP server for remote browsing.
 IBUFNO=0          Specifies the number of extra buffers to be allocated for 
                   navigating an index file.
 IBUFSIZE=0        Specifies the buffer page size for an index file.
 IMLPACKAGEPRIVATE=~/sas/iml/packages
                   Specifies the location for SAS/IML packages in the private 
                   collection.
 IMLPACKAGEPUBLIC=/opt/sas/iml/packages
                   Specifies the location for SAS/IML packages in the public 
                   collection.
 IMLPACKAGESYSTEM=!SASROOT/misc/iml/packages
                   Specifies the location for SAS/IML packages in the system 
                   collection.
 NOIMPLMAC         Does not check for statement-style macros.
 INITCMD=          Specifies commands to open applications, or windows and 
                   text editor commands, after SAS executes the AUTOEXEC= 
                   file and the INITSTMT= value.
 INITSTMT=         Specifies SAS statements to execute after any statements 
                   in the AUTOEXEC= file and before any statements from the 
                   SYSIN= file.
 INSERT=           Specifies an option=value pair to insert the value at the 
                   beginning of the existing option value.
 INTERVALDS=       Specifies interval=library pairs.  Library is a SAS data 
                   set that contains a custom interval data set; interval can 
                   be used in the INTNX and INTCK functions.
 INVALIDDATA=.     Specifies the value that SAS assigns to a variable when 
                   invalid numeric data is encountered.
 NOIPADDRESS       Disables the IP address to appear in SAS/CONNECT messages 
                   when using TCP/IP.
 JPEGQUALITY=75    Specifies the JPEG quality factor that determines the 
                   ratio of image quality to the level of compression for 
                   JPEG files produced by the JPEG device driver.
 LABEL             Enables procedures to use labels with variables.
 NOLABELCHKPT      For batch programs, disables the recording of 
                   checkpoint-restart data for labeled code sections.
 LABELCHKPTLIB=WORK
                   Specifies the libref of the library where the 
                   checkpoint-restart data is saved for labeled code sections.
 NOLABELRESTART    Disables restart mode, which executes batch programs using 
                   checkpoint-restart data collected at labeled code sections.
 LEFTMARGIN=0.000 IN
                   Specifies the print margin for the left side of the page.
 LINESIZE=78       Specifies the line size for the SAS log and for SAS 
                   procedure output for the LISTING destination.
 LOCALEDATA=SASLOCALE
                   Specifies the location of the locale database.
 NOLOCKDOWN        Specifies that access to files and certain SAS features 
                   will not be restricted. This feature is only applicable 
                   for a SAS session executing in a batch or server 
                   processing mode.
 LOGAPPLNAME=base_audit
                   Specifies a SAS session name for SAS logging.
 LOGCONFIGLOC=/apps/SAS/v9.4/sc1_config/sasenv/logconfig.xml
                   Specifies the name of the XML configuration file or a 
                   basic logging configuration that is used to initialize the 
                   SAS logging facility.
 NOLOGLANGCHG      Disables changing the language of the SAS log when the 
                   LOCALE= option is changed.
 NOLOGLANGENG      Write SAS log messages based on the values of the 
                   LOGLANGCHG, LSWLANG=, and LOCALE= options when SAS started.
 LOGPARM=WRITE=BUFFERED ROLLOVER=NONE OPEN=REPLACE
                   Specifies when SAS log files are opened, closed, and 
                   according to the LOG= system option, how they are named.
 LRECL=32767       Specifies the default logical record length to use for 
                   reading and writing external files.
 LSWLANG=LOCALE    Specifies the language for SAS log and ODS messages when 
                   the LOCALE= option is set after SAS starts.
 MACRO             Enables the macro facility.
 MAPEBCDICTOASCII= Specifies the transcoding table that is used to convert 
                   characters from ASCII to EBCDIC and EBCDIC to ASCII.
 MAPS=!SASROOT/maps
                   Specifies the location of SAS/GRAPH map data sets.
 MAPSGFK=!SASROOT/mapsgfk
                   Specifies the location of GfK maps.
 MAPSSAS=!SASROOT/maps
                   Specifies the location of SAS map data sets.
 NOMAUTOCOMPLOC    Does not display the autocall macro source location in the 
                   SAS log when the autocall macro is compiled.
 NOMAUTOLOCDISPLAY Disables the macro facility from displaying the autocall 
                   macro source location in the log.
 NOMAUTOLOCINDES   Does not prepend the full pathname of the autocall macro 
                   source file to the autocall macro catalog entry 
                   description field in the WORK.SASMACR catalog.
 MAUTOSOURCE       Enables the macro autocall feature.
 MAXSEGRATIO=75    Specifies the upper limit for the percentage of index 
                   segments that the SPD Engine identifies as containing the 
                   value referenced in the WHERE expression.
 MCOMPILE          Allows new macro definitions.
 MCOMPILENOTE=NONE Specifies what to write to the SAS log when a macro 
                   compiles successfully.
 NOMCOVERAGE       Disables the generation of coverage analysis data for SAS 
                   macros.
 MCOVERAGELOC=     Specifies the location of the macro coverage analysis data 
                   file.
 MERGENOBY=NOWARN  Specifies the type of message that is issued when MERGE 
                   processing occurs without an associated BY statement.
 MERROR            Issues a warning message for an unresolved macro reference.
 METAAUTORESOURCES=
                   Specifies the metadata resources that are assigned when 
                   SAS starts.
 METACONNECT=      Specifies the profile from the metadata user connection 
                   profiles that is used to connect to the SAS Metadata 
                   Server.
 METAENCRYPTALG=SASPROPRIETARY
                   Specifies the type of encryption to use to communicate 
                   with the SAS Metadata Server.
 METAENCRYPTLEVEL=CREDENTIALS
                   Specifies the level of encryption that is used to 
                   communicate with the SAS Metadata Server.
 METAID=           Specifies the ID of the SAS Metadata Server.
 METAPROFILE=      Specifies the XML document that contains SAS Metadata 
                   Server user connection profiles.
 METAPROTOCOL=BRIDGE
                   Specifies the network profile to use to connect to the SAS 
                   Metadata Server.
 METAREPOSITORY=Foundation
                   Specifies the name of the SAS Metadata Server Repository.
 METASERVER=       Specifies the host name or address of the SAS Metadata 
                   Server.
 METASPN=          Specifies the service principal name (SPN) for the SAS 
                   Metadata Server.
 METAUSER=         Specifies the user ID that is used to connect to the SAS 
                   Metadata Server.
 NOMEXECNOTE       Does not display the macro execution information in the 
                   SAS log when the macro is invoked.
 MEXECSIZE=65536   Specifies the maximum macro size that can be executed in 
                   memory.
 NOMFILE           Does not write MPRINT output to an external file.
 MINDELIMITER=     Specifies the character delimiter for the macro IN 
                   operator.
 NOMINOPERATOR     Disables IN logical operators in expressions.
 MINPARTSIZE=16777216
                   Specifies the minimum size of the data component 
                   partitions for SPD Engine data sets.
 MISSING=.         Specifies the character to print for missing numeric 
                   values.
 NOMLOGIC          Does not trace macro execution or write the results to the 
                   SAS log.
 NOMLOGICNEST      Does not display the macro nesting information in the SAS 
                   log for MLOGIC output.
 NOMPRINT          Does not display the SAS statements that are generated by 
                   macro execution.
 NOMPRINTNEST      Does not display the macro nesting information from the 
                   MPRINT output in the SAS log.
 NOMRECALL         Searches the autocall libraries only once for a requested 
                   macro.
 MREPLACE          Enables updates to macro definitions in the Work library.
 MSGLEVEL=N        Specifies the level of detail in SAS log messages.
 NOMSTORED         Does not search for stored compiled macros.
 MSYMTABMAX=4194304
                   Specifies the maximum amount of memory available to the 
                   macro variable symbol table or tables.
 NOMULTENVAPPL     List only operating environment fonts in the font selector 
                   window of a SAS application.
 MVARSIZE=65534    Specifies the maximum size for a macro variable that is 
                   stored in memory.
 NETENCRYPT        Requires encryption for client/server data transfers.
 NETENCRYPTALGORITHM=AES
                   Specifies one or more algorithms to use for encrypted 
                   client/server data transfers.
 NETENCRYPTKEYLEN=0
                   Specifies the key length that is used by the encryption 
                   algorithm for encrypted client/server data transfers.
 NEWS=!SASROOT/misc/base/news
                   Specifies the location of the news file that is to be 
                   written to the SAS log immediately after the header.
 NONLDECSEPARATOR  Disables formatting of numeric output using the decimal 
                   separator for the locale.
 NOTES             SAS writes notes to the SAS log.
 NUMBER            Prints the page number on the first title line of each 
                   page of SAS output.
 NOOBJECTSERVER    Disables SAS from running an Integrated Object Model (IOM) 
                   server.
 ODSDEST=AUTO      Specifies the default ODS destination.
 ODSGRAPHICS=AUTO  Specifies the setting for ODS graphics.
 NOODSLANGCHG      Disables changing the language of the SAS message text in 
                   ODS output when the LOCALE option is set after start up.
 ODSSTYLE=AUTO     Specifies the ODS HTML default style.
 OLAPCONFIG=       Specifies the name of the XML configuration file that is 
                   used to initialize an OLAP server.
 ORIENTATION=PORTRAIT
                   Specifies the paper orientation to use when printing to a 
                   printer.
 NOOVP             Disables overprinting of error messages to make them bold.
 NOPAGEBREAKINITIAL
                   Does not begin SAS log and procedure output for the 
                   LISTING destination on a new page.
 PAGENO=1          Resets the SAS output page number.
 PAGESIZE=21       Specifies the number of lines that compose a page of the 
                   SAS log and SAS output.
 PAPERDEST=        Specifies the name of the output bin to receive printed 
                   output.
 PAPERSIZE=LETTER  Specifies the paper size to use for printing.
 PAPERSOURCE=      Specifies the name of the paper bin to use for printing.
 PAPERTYPE=PLAIN   Specifies the type of paper to use for printing.
 PARM=             Specifies a parameter string that is passed to an external 
                   program.
 PARMCARDS=FT15F001
                   Specifies the file reference to open when SAS encounters 
                   the PARMCARDS statement in a procedure.
 PDFACCESS         Enables screen readers to read PDF text and graphics.
 NOPDFASSEMBLY     Disables assembly of PDF documents.
 NOPDFCOMMENT      Disables comments in PDF documents from being modified.
 NOPDFCONTENT      Disables modification of PDF document content.
 PDFCOPY           Enables PDF document text and graphics to be copied.
 PDFFILLIN         Enables PDF forms to be filled in.
 PDFPAGELAYOUT=DEFAULT
                   Specifies the page layout for PDF documents.
 PDFPAGEVIEW=DEFAULT
                   Specifies the page viewing mode for PDF documents.
 PDFPRINT=HRES     Specifies the resolution to print PDF documents.
 PDFSECURITY=NONE  Specifies the level of encryption to use for PDF documents.
 NOPRESENV         Specifies that collecting data for the preservation of the 
                   SAS environment is disabled.
 PRIMARYPROVIDERDOMAIN=
                   Specifies the domain name of the primary authentication 
                   provider.
 PRINTERPATH=      Specifies the name of a registered printer to use for 
                   Universal Printing.
 NOPRINTINIT       Preserves the procedure output file for the LISTING 
                   destination if no new output is created.
 PRINTMSGLIST      Specifies to print the entire list of messages to the SAS 
                   log.
 PROTOLIBS=NONE    Specifies the paths that PROC PROTO can use to find and 
                   register load modules.
 QUOTELENMAX       Writes a warning message to the SAS log if a quoted string 
                   exceeds the maximum length allowed.
 REPLACE           Enables replacement of permanent SAS data sets.
 REUSE=NO          Specifies whether SAS reuses space when observations are 
                   added to a compressed SAS data set.
 RIGHTMARGIN=0.000 IN
                   Specifies the print margin for the right side of the page.
 NORLANG           Disables SAS from executing R language statements.
 RSASIOTRANSERROR  Displays a transcoding error when illegal values are read 
                   from a remote application.
 NORSASUSER        Opens the Sasuser library in Write mode.
 S=0               Specifies the length of statements on each line of a 
                   source statement, and the length of data on lines that 
                   follow a DATALINES statement.
 S2=0              Specifies the length of statements of each line of a 
                   source statement from an %INCLUDE statement, an AUTOEXEC= 
                   file, or an autocall macro file.
 S2V=0             Specifies the column to begin reading a file with variable 
                   length records that is specified in an %INCLUDE statement, 
                   an autoexec file, or an autocall macro.
 SASAUTOS=(          '!SASROOT/forecastbat/sasmacro'         
'!SASROOT/sasautos'         )
                   Specifies the location of one or more autocall libraries.
 SASCMD=           Specifies the command that starts a server session on a 
                   symmetric multiprocessing (SMP) computer.

 SASHELP=( '!SASROOT/nls/en/sascfg' '!SASROOT/nls/en/sascfg'         
'!SASROOT/sashelp' )
                   Specifies the location of the Sashelp library.
 SASMSTORE=        Specifies the libref of a SAS catalog for stored compiled 
                   SAS macros.
 SASSCRIPT=(          '!SASROOT/misc/connect'         
'!SASROOT/samples/connect'         )
                   Specifies one or more locations of SAS/CONNECT server 
                   sign-on script files.
 SASUSER=~/sasuser.v94
                   Specifies a libref or a path that identifies a library for 
                   the user's profile catalog.
 SECPACKAGE=Negotiate
                   Specifies the security package that the IOM server uses to 
                   authenticate incoming client connections.
 SECPACKAGELIST=Kerberos,NTLM
                   Specifies the security authentication packages that are 
                   used by the server.
 SEQ=8             Specifies the length of the numeric portion of the 
                   sequence field in input source lines or data lines.
 SERROR            Issues a warning message when a macro variable reference 
                   does not match a macro variable.
 SERVICESBASEURL=  Specifies the URL for services.
 SESSREF=CASAUTO   Identify the name to associate with a generated CAS 
                   session.
 NOSETINIT         Disables PROC SETINIT to prevent updating site licensing 
                   information.
 SHARESESSIONCNTL=SERVER
                   Specifies whether the SAS/SHARE server has one or multiple 
                   connections to clients.
 SIGNONWAIT        Executes the SIGNON statement synchronously, signing on 
                   clients to the server one at a time.
 SKIP=0            Specifies the number of lines to skip at the top of each 
                   page of SAS output for the LISTING destination.
 SOLUTIONS         Displays the Solutions menu in SAS windows.
 SORTDUP=PHYSICAL  Specifies whether PROC SORT removes duplicate variables 
                   based on the DROP and KEEP options or on all data set 
                   variables.
 SORTEQUALS        PROC SORT maintains the relative position in the output 
                   data set for observations with identical BY-variable 
                   values.
 SORTSEQ=          Specifies a language-specific collating sequence for the 
                   SORT and SQL procedures.
 SORTSIZE=1073741824
                   Specifies the amount of memory that is available to the 
                   SORT procedure.
 NOSORTVALIDATE    SORT does not verify whether a data set is sorted 
                   according to the variables in the BY statement.
 NOSOURCE          Does not write program source statements to the SAS log.
 NOSOURCE2         Does not write secondary source statements from included 
                   files to the SAS log.
 NOSPDEFILECACHE   Disables caching of opened SPD Engine files.
 SPDEINDEXSORTSIZE=33554432
                   Specifies the memory size for sorting index values.
 SPDEMAXTHREADS=0  Specifies the maximum number of threads that the SPD 
                   Engine can spawn for I/O processing.
 SPDEPARALLELREAD=NO
                   Enables or disables SPD Engine parallel reads when no 
                   WHERE clause is in effect.
 SPDESORTSIZE=33554432
                   Specifies the memory size that is used for sorting by the 
                   SPD Engine.
 SPDEUTILLOC=      Specifies one or more locations where the SPD Engine can 
                   temporarily store utility files.
 SPDEWHEVAL=COST   Specifies the WHERE statement evaluation process for the 
                   SPD Engine.
 NOSPOOL           Does not write SAS statements to a utility data set in the 
                   Work library.
 SQLCONSTDATETIME  PROC SQL replaces references to the DATE, TIME, DATETIME, 
                   and TODAY functions with their equivalent constant values 
                   before a query executes.
 SQLGENERATION=(NONE DBMS='TERADATA DB2 ORACLE NETEZZA ASTER GREENPLM HADOOP 
SAPHANA IMPALA HAWQ POSTGRES REDSHIFT SQLSVR VERTICA BIGQUERY SNOW YBRICK 
SPARK MYSQL')
                   Specifies whether and when SAS procedures generate SQL for 
                   in-database processing of source data.
 NOSQLIPONEATTEMPT Allows an SQL query to continue processing when an 
                   implicit pass-through request fails.
 SQLLDRPATH=       Path for Oracle bulkload.
 SQLMAPPUTTO=SAS_PUT
                   Specifies the PUT function mapping to SQL.
 SQLREDUCEPUT=DBMS For PROC SQL, specifies the engine type to use to optimize 
                   a PUT function in a query.
 SQLREDUCEPUTOBS=0 For PROC SQL, specifies the minimum number of observations 
                   that must be in a table for PROC SQL to optimize the PUT 
                   function in a query.
 SQLREDUCEPUTVALUES=0
                   For PROC SQL, specifies the maximum number of SAS format 
                   values that can exist in a PUT function expression to 
                   optimize the PUT function in a query.
 SQLREMERGE        PROC SQL processes queries that use remerged data.
 SQLUNDOPOLICY=REQUIRED
                   Specifies how PROC SQL handles updated data if errors 
                   occur while you are updating data.
 NOSSLCLIENTAUTH   Does not require the server to perform client 
                   authentication for a server connection.
 NOSSLCRLCHECK     Does not check the Certificate Revocation List (CRL) when 
                   a digital certificate is validated.
 SSLMINPROTOCOL=   Specifies the minimum TLS or SSL protocol that can be 
                   negotiated when using OpenSSL.
 SSLMODE=          Specifies the TLS version and the cipher suites for SSL.
 NOSSPI            Does not use Security Support Provider Interface for 
                   single sign-on connections to IOM servers.
 NOSTARTLIB        Does not assign user-defined permanent librefs when SAS 
                   starts.
 NOSTEPCHKPT       Disables recording of checkpoint-restart data for DATA and 
                   PROC steps for batch programs.
 STEPCHKPTLIB=WORK Specifies the libref of the library where 
                   checkpoint-restart data for DATA and PROC steps is saved.
 NOSTEPRESTART     Disables restart mode which executes batch programs using 
                   checkpoint-restart data collected for DATA and PROC steps 
                   in a prior execution.
 STRIPESIZE=       Specifies path and size pairs to identify I/O device 
                   stripe size.  Stripe size indicates page size when 
                   creating a data set or utility file.
 SUMSIZE=0         Specifies a limit on the amount of memory that is 
                   available for data summarization procedures when class 
                   variables are active.
 SVGAUTOPLAY       Starts animation when the page is loaded in the browser.
 NOSVGCONTROLBUTTONS
                   Does not display the paging control buttons and an index 
                   in a multipage SVG document.
 SVGFADEIN=0       Specifies the number of seconds for the fade-in effect for 
                   a graph.
 SVGFADEMODE=OVERLAP
                   Specifies whether to use sequential frames or to overlap 
                   frames for the fade-in effect of a graph.
 SVGFADEOUT=0      Specifies the number of seconds for a graph to fade out of 
                   view.
 SVGHEIGHT=        Specifies the height of the viewport. Specifies the value 
                   of the height attribute of the outermost SVG element.
 NOSVGMAGNIFYBUTTON
                   Disables the SVG magnifier tool.
 SVGPRESERVEASPECTRATIO=
                   Specifies whether to force uniform scaling of SVG output. 
                   Specifies the preserveAspectRatio attribute on the 
                   outermost SVG element.
 SVGTITLE=         Specifies the text in the title bar of the SVG output. 
                   Specifies the value of the TITLE element in the SVG file.
 SVGVIEWBOX=       Specifies the coordinates, width, and height that are used 
                   to set the viewBox attribute on the outermost SVG element.
 SVGWIDTH=         Specifies the width of the viewport. Specifies the value 
                   of the width attribute of the outermost SVG element.
 SVGX=             Specifies the x-axis coordinate of one corner of the 
                   rectangular region for an embedded SVG element. Specifies 
                   the x attribute in the outermost SVG element.
 SVGY=             Specifies the y-axis coordinate of one corner of the 
                   rectangular region for an embedded SVG element. Specifies 
                   the y attribute in the outermost SVG element.
 NOSYMBOLGEN       Does not display the results of resolving macro variable 
                   references in the SAS log.
 SYNCHIO           Requires that data set I/O must be completed before other 
                   logical SAS tasks can be executed.
 SYNTAXCHECK       Enables syntax check mode for multiple steps in 
                   non-interactive or batch SAS sessions.
 SYSPARM=          Specifies a character string that can be passed to SAS 
                   programs.
 SYSPRINTFONT=     Specifies the default font to use for printing.
 NOSYSRPUTSYNC     Sets the %SYSRPUT macro variables in the client session 
                   when a synchronization point is encountered.
 TBUFSIZE=0        Specifies the size of the buffer that is used by SAS 
                   applications to transfer client/server data across a 
                   network.
 TCPLISTENTIME=300 Specifies the amount of time that a SAS/CONNECT server 
                   listens for a client to connect before terminating the 
                   CONNECT server session.
 TCPPORTFIRST=0    Specifies the first value in a range of TCP/IP ports for a 
                   client to use to connect to a server.
 TCPPORTLAST=0     Specifies the last value in a range of TCP/IP ports for a 
                   client to use to connect to a server.
 TENANTID=         Specifies a name that identifies a tenant in a 
                   multi-tenant environment.
 TERMINAL          Associates a terminal with a SAS session.
 TERMSTMT=         Specifies the SAS statement to execute when SAS terminates.
 TEXTURELOC=!SASROOT/misc/textures
                   Specifies the location of textures and images that are 
                   used by ODS styles.
 THREADS           Uses threaded processing for SAS applications that support 
                   it.
 TIMEZONE=         Specifies a time zone.
 TOOLSMENU         Displays the Tools menu in SAS windows.
 TOPMARGIN=0.000 IN
                   Specifies the print margin at the top of the page.
 TRAINLOC=         Specifies the URL for SAS online training courses.
 TRANTAB=(lat1wlt1,wlt1lat1,lat1_ucs,lat1_lcs,lat1_ccl,,,)
                   Specifies the translation table catalog entries.
 TSID=             Specifies a logical server metadata object that identifies 
                   a table service definition.
 UBUFNO=0          Specifies the number of utility file buffers.
 UBUFSIZE=0        Specifies the size of utility file buffers.
 UNIVERSALPRINT    Enables the Universal Printing windows to display and sets 
                   up default values for printing.
 UPRINTCOMPRESSION Enables compression of files that are created by some 
                   Universal Printers and SAS/GRAPH devices.
 URLENCODING=SESSION
                   Specifies whether the argument to the URLENCODE function 
                   and to the URLDECODE function is interpreted using the SAS 
                   session encoding or UTF-8 encoding.
 USER=             Specifies the default permanent library to use for 
                   one-level SAS data set names.
 UTILLOC=WORK      Specifies one or more file system locations in which 
                   threaded applications can store utility files.
 UUIDCOUNT=100     Specifies the number of UUIDs to acquire from the UUID 
                   Generator Daemon.
 UUIDGENDHOST=     Specifies the host and port, or the LDAP URL that the UUID 
                   Generator Daemon runs on.
 V6CREATEUPDATE=NOTE
                   Specifies the type of message to write to the SAS log when 
                   Version 6 data sets are created or updated.
 VALIDFMTNAME=LONG Specifies the maximum size that user-created formats and 
                   informat names can be before an error or warning is issued.
 VALIDMEMNAME=COMPAT
                   Specifies the rules for naming SAS data sets, SAS data 
                   views, and item stores.
 VALIDVARNAME=V7   Specifies the rules for valid SAS variable names that can 
                   be created and processed during a SAS session.
 VARINITCHK=NOTE   Specifies the type of message to write to the SAS log when 
                   a variable is not initialized.
 VARLENCHK=WARN    Specifies the type of message to write to the SAS log when 
                   the length of the variable that is being read is longer 
                   than the length that is defined for the variable.
 VBUFSIZE=65536    Specifies the buffer size for a view.
 VIEWMENU          Displays the View menu in SAS windows.
 VNFERR            SAS issues an error message when a BY variable exists in 
                   one data set but not another when the other data set is 
                   _NULL_.

 WORKINIT          At SAS invocation, erases files that exist from a previous 
                   SAS session in an existing Work library.
 WORKTERM          Erases the Work files when SAS terminates.
 YEARCUTOFF=1940   Specifies the first year of a 100-year span that is used 
                   by date informats and functions to read a two-digit year.
 _LAST_=_NULL_     Specifies the most recently created data set.

Host Options:

 ALIGNSASIOFILES   Aligns SAS files on a page boundary for improved 
                   performance.
 ALTLOG=           Specifies the location for a copy of the SAS log when SAS 
                   is running in batch mode.
 ALTPRINT=         Specifies the location for a copy of the SAS procedure 
                   output when SAS is running in batch mode.
 AUTHPROVIDERDOMAIN=
                   Specifies the authentication provider that is associated 
                   with a domain.
 BLKSIZE=256       Specifies the number of bytes that are read or written in 
                   one I/O operation.
 COMAUX1=          Specifies the first alternate communication access method.
 COMAUX2=          Specifies the second alternate communication access method.
 CONFIG=( /apps/SAS/v9.4/SASHome/SASFoundation/9.4/sasv9.cfg 
/apps/SAS/v9.4/SASHome/SASFoundation/9.4/nls/en/sasv9.cfg 
/apps/SAS/v9.4/SASHome/SASFoundation/9.4/sasv9_local.cfg )
                   Specifies the configuration file that is used when 
                   initializing or overriding the values of SAS system 
                   options.
 NODBCS            Disables double-byte character sets.
 DBCSLANG=NONE     Specifies a double-byte character set language.
 DBCSTYPE=NONE     Specifies the encoding method that is used for a 
                   double-byte character set.
 ECHO=             Specifies a message that is echoed to the SAS log while 
                   initializing SAS.
 EDITCMD=          Specifies the host editor that is used with the HOSTEDIT 
                   command.
 EMAILSYS=smtp     Specifies the email protocol that is used for sending 
                   electronic mail.
 ENCODING=LATIN1   Specifies the default character-set encoding for the SAS 
                   session.
 FILELOCKS=( '/' FAIL )
                   Specifies whether file locking is turned on or off and 
                   what action should be taken if a file cannot be locked.
 FILELOCKWAIT=0    Specifies the number of seconds that SAS will wait for a 
                   locked file.
 FILELOCKWAITMAX=600
                   Specifies the maximum number of seconds that SAS waits for 
                   a locked file to become available.
 FSDBTYPE=DEFAULT  Specifies a full-screen double-byte character set (DBCS) 
                   encoding method.
 FSIMM=            Specifies input method modules (IMMs) for full-screen 
                   double-byte character sets (DBCS).
 FSIMMOPT=         Specifies options for input method modules (IMMs) that are 
                   used with a full-screen double-byte character set (DBCS).
 NOFULLSTIMER      Does not write performance statistics to the SAS log.
 HELPLOC=( '!SASROOT/X11/native_help' '!SASROOT/X11/native_help'         
'!SASUSER/classdoc' )
                   Specifies the location of the text and index files for the 
                   facility that is used to view the online SAS Help and 
                   Documentation.
 INGOPTS=          Specifies the options to pass to the INGRES database.
 JREOPTIONS=(           
-DPFS_TEMPLATE=/apps/SAS/v9.4/SASHome/SASFoundation/9.4/misc/tkjava/qrpfstpt.x
ml         
-Djava.class.path=/apps/SAS/v9.4/SASHome/SASVersionedJarRepository/eclipse/plu
gins/sas.launcher.jar         
-Djava.security.auth.login.config=/apps/SAS/v9.4/SASHome/SASFoundation/9.4/mis
c/tkjava/sas.login.config         
-Djava.security.policy=/apps/SAS/v9.4/SASHome/SASFoundation/9.4/misc/tkjava/sa
s.policy         -Djava.system.class.loader=com.sas.app.AppClassLoader        

-Dlog4j.configuration=file:/apps/SAS/v9.4/SASHome/SASFoundation/9.4/misc/tkjav
a/sas.log4j.properties         
-Dsas.app.class.path=/apps/SAS/v9.4/SASHome/SASVersionedJarRepository/eclipse/
plugins/tkjava.jar         
-Dsas.ext.config=/apps/SAS/v9.4/SASHome/SASFoundation/9.4/misc/tkjava/sas.java
.ext.config         
-Dtkj.app.launch.config=/apps/SAS/v9.4/SASHome/SASVersionedJarRepository/pickl
ist          )
                   Specifies the Java Runtime Environment options for SAS.
 LOADMEMSIZE=0     Specifies a suggested amount of memory that is needed for 
                   executable programs loaded by SAS.
 LOCALE=EN_US      Specifies a set of attributes in a SAS session that 
                   reflect the language, local conventions, and culture for a 
                   geographical region.
 LOG=              Specifies a location for the SAS log when SAS is running 
                   in batch mode.
 LPTYPE=           Specifies the UNIX command that is used to print files.
 MAXMEMQUERY=268435456
                   For certain procedures, specifies the maximum amount of 
                   memory that can be allocated per request.
 MEMSIZE=2147483648
                   Specifies the limit on the amount of virtual memory that 
                   can be used during a SAS session.
 MSG=!SASROOT/sasmsg
                   Specifies the path to the library that contains SAS 
                   messages.
 NOMSGCASE         Specifies that SAS writes notes, warning, and error 
                   messages in mixed casing.
 NONLSCOMPATMODE   Encodes data using the SAS session encoding.
 NOOPLIST          Does not write SAS system option settings to the SAS log.
 PATH=!SASROOT/sasexe
                   Specifies one or more search paths for SAS executable 
                   files.
 PRINT=            Specifies a location for SAS output when running in batch 
                   mode.
 PRINTCMD=         Specifies the print command that SAS is to use.
 REALMEMSIZE=0     Specifies the amount of real memory SAS can expect to 
                   allocate.
 RTRACE=NONE       Specifies whether to produce a list of resources that are 
                   read or loaded during a SAS session.
 RTRACELOC=        Specifies the pathname for the file that records the list 
                   of resources that are read or loaded during a SAS session.
 SEQENGINE=TAPE    Specifies the sequential engine to be used for accessing 
                   tapes and pipes.
 SET=[SASROOT = /apps/SAS/v9.4/SASHome/SASFoundation/9.4] [SASAUTOS = (       
           '!SASROOT/sasautos'         )] [SAMPSRC = (                  
'!SASROOT/samples/accesssample'         '!SASROOT/samples/base'         
'!SASROOT/samples/connect'         '!SASROOT/samples/dbi'         
'!SASROOT/samples/dquality'         '!SASROOT/samples/eis'         
'!SASROOT/samples/ets'         '!SASROOT/samples/gis'         
'!SASROOT/samples/graph'         '!SASROOT/samples/hpf'         
'!SASROOT/samples/hps'         '!SASROOT/samples/hpstat'         
'!SASROOT/samples/iml'         '!SASROOT/samples/intrnet'         
'!SASROOT/samples/inttech'         '!SASROOT/samples/or'         
'!SASROOT/samples/qc'         '!SASROOT/samples/share'         
'!SASROOT/samples/stat'         )] [SAMPSIO = (                  
'!SASROOT/samples/accesssample'         '!SASROOT/samples/base'         
'!SASROOT/samples/connect'         '!SASROOT/samples/dbi'         
'!SASROOT/samples/dquality'         '!SASROOT/samples/eis'         
'!SASROOT/samples/ets'         '!SASROOT/samples/gis'         
'!SASROOT/samples/graph'         '!SASROOT/samples/hpf'         
'!SASROOT/samples/hps'         '!SASROOT/samples/hpstat'         
'!SASROOT/samples/iml'         '!SASROOT/samples/intrnet'         
'!SASROOT/samples/inttech'         '!SASROOT/samples/or'         
'!SASROOT/samples/qc'         '!SASROOT/samples/share'         
'!SASROOT/samples/stat'         )] [SASHOME = /apps/SAS/v9.4/SASHome]
                   Defines an environment variable.
 SORTANOM=         Specifies options for the host sort utility.
 SORTCUT=0         Specifies the data size in number of observations above 
                   which SAS uses the host sort instead of the internal SAS 
                   sort.
 SORTCUTP=0        Specifies the data size in bytes above which SAS uses the 
                   host sort instead of the internal SAS sort.
 SORTDEV=          Specifies the pathname to temporary files that are created 
                   by the host sort utility.
 SORTNAME=         Specifies the name of the host sort utility.
 SORTPARM=         Specifies the parameters for the host sort utility.
 SORTPGM=BEST      Specifies whether to use the SAS sort utility or the host 
                   sort utility or to let SAS choose the sort utility.
 SSLCACERTDATA=    Specifies the CA certificate text string that is usually 
                   found in a CA certificate PEM file.
 SSLCACERTDIR=     Specifies the directory for PEM format CA certificates 
                   used by OpenSSL. Each file has only one certificate and 
                   the file name must be the certificate hash value.

 NOSTDIO           SAS does not use the standard streams stdin, stdout, and 
                   stderr.
 STIMEFMT=(NLDATM2. HMS TIMEAMPM KB MEMFULL TSFULL NC)
                   Specifies the format that is used to display the 
                   FULLSTIMER and STIMER output for timestamp, memory, CPU 
                   and elapsed time statistics.
 STIMER            Writes real and CPU time to the SAS log.
 SYSIN=            Specifies the SAS program to execute in batch.
 SYSPRINT=         Specifies the destination for printed output.
 TAPECLOSE=REREAD  CLOSE disposition for a library on tape.
 USERCONFIG        Process .sasv9.cfg and sasv9.cfg configuration files in 
                   user's home directory.
 NOVERBOSE         Does not write start-up system options to the SAS log.
 WORKPERMS=777     Sets the permissions of the SAS Work library when it is 
                   initially created.
 XCMD              Enables the X command in SAS.
weg-jr commented 7 months ago

and as an extensions, here is the log relevant contents of /apps/SAS/v9.4/SASHome/SASFoundation/9.4/sasv9.cfg which I guess is the main config. There doesnt seem to be any other logging related settings in that file that are not in the proc.

/* Options for SAS Audit Logging */
-LOGAPPLNAME base_audit
-LOGCONFIGLOC /apps/SAS/v9.4/sc1_config/sasenv/logconfig.xml

contents of /apps/SAS/v9.4/sc1_config/sasenv/logconfig.xml

<?xml version="1.0" encoding="UTF-8"?>
<logging:configuration xmlns:logging="http://www.sas.com/xml/logging/1.0/">

   <!-- syslog enabled so any SAS logging is not enabled      -->
   <!-- Audit of Data set appender definition used in logging -->

   <!-- Audit of Data set activity to syslog -->
   <appender class="UNXFacilityAppender" name="AuditLibraryFile">
      <layout>
         <param name="ConversionPattern"
            value="%d{yyyy-MM-dd hh:mm:ss},%S{App.Name},%u,%p,%S{hostname},%S{os_family},%S{sup_ver_long2},%E{Audit.Dataset.Status},%S{startup_cmd},%E{Audit.Dataset.Action},%E{Audit.Dataset.Libref},%E{Audit.Dataset.Openmode},%E{Audit.Dataset.Member},%E{Audit.Dataset.Path}"/>
      </layout>
   </appender>
   <appender class="FileAppender" name="BatchFileAppender">
      <param name="Append" value="false"/>
      <param name="Unique" value="false"/>
      <param name="FileNamePattern" value="%S{App.Log|/dev/null}"/>
      <layout>
         <param name="ConversionPattern" value="%m"/>
      </layout>
   </appender>
   <!-- Audit.Data.Dataset.Open logger definition -->

   <logger name="Audit.Data.Dataset.Open" additivity="false">
      <appender-ref ref="AuditLibraryFile"/>
      <level value="Trace"/>
   </logger>

   <!-- Audit.Data.Dataset.Delete logger definition -->

   <logger name="Audit.Data.Dataset.Delete" additivity="false">
      <appender-ref ref="AuditLibraryFile"/>
      <level value="Trace"/>
   </logger>

   <!-- Audit.Data.Dataset.Rename logger definition -->

   <logger name="Audit.Data.Dataset.Rename" additivity="false">
      <appender-ref ref="AuditLibraryFile"/>
      <level value="Trace"/>
   </logger>

   <logger name="Admin.Session" additivity="false">
      <level value="Trace"/>
      <appender-ref ref="AuditLibraryFile"/>
   </logger>

   <!--  root logger events not enabled  -->
   <root>
      <level value="Info"/>
      <appender-ref ref="BatchFileAppender"/>
   </root>

</logging:configuration>
tomweber-sas commented 7 months ago

Ok, thanks for that, and yes, I was expecting the wall :)

So, I'm not great at knowing what that logger xml will do, but it sure is the first thing I'm thinking of to see if it's the reason saspy isn't getting any output from SAS. If it's routing the log somewhere else, that would be it. But I'm not really sure that's what's happening. Let's try something and see. Can edit that sasv9.cfg, just comment out the -LOGCONFIGLOC and see what happens?

You can see in the proc options output that those were applied, as they would be. But I can't tell if that' really the problem or not without seeing if not having that fixes it.

weg-jr commented 7 months ago

I will need to submit a ticket for the administrator to do that which will take a while. I think I have enough reason to justify it with what you provided.

I'm pretty convinced that is what is causing the issue, and that the saspy package is working as intended.

Do you think I should close this out now or leave it open till I can get get an update?

tomweber-sas commented 7 months ago

Let's leave it open till it's solved! Or we just can't stand it anymore :)

tomweber-sas commented 6 months ago

Just checking on the status of this. Did your SAS admins provide any help on this?

weg-jr commented 6 months ago

I'm still waiting on approval to go up the chain to have one assigned unfortunately. TLDR is Im still very interested in an answer but right now getting this package to work is low priority in my org. It will probably still be several weeks before I even get an admin assigned to look at our config and probably a month before I have results.

I did play around a bit more. but ultimately didn't find anything that worked. I cant remove the -LOGCONFIGLOC since one must always be configured, and I cant modified the global one, but I can point to a custom logconfig xml in my home drive.

First I tried modifying the file appender to point to /dev/stdout. obviously this breaks batch sas file logging but I wanted to see if I could force log data through stdout. This did work but it only appends the log after all data has come through, so I am still typing blind.

I looked through the example configs in a sas administrator guide and found a consolappender which looked promising, but it my output duplicates the notes and I couldn't figure out how to remove them since this doesn't happen for any other log output mode. This is where I paused looking at the issue since it seemed I cant make progress without modifying the base logconfig.

Probably doesnt matter but for reference here is how my logconfig modifies the one I posted earlier.

<?xml version="1.0" encoding="UTF-8"?>
<logging:configuration xmlns:logging="http://www.sas.com/xml/logging/1.0/" modify="True">

 <appender class="ConsoleAppender" name="SAS_CONSOLE">
             <layout>
                      <param name="ConversionPattern" value="%d [%t] %-5p %c - %m%n"/>
             </layout>
</appender>
</logging:configuration>

output of echo 'proc print data=sashelp.class(obs=3);run;' | sas -fullstimer -nodms -stdio -terminal -nosyntaxcheck -pagesize MAX

                                                           The SAS System                     08:28 Wednesday, December 20, 2023   1

                                          Obs     Name      Sex    Age    Height    Weight

                                            1    Alfred      M      14     69.0      112.5
                                            2    Alice       F      13     56.5       84.0
                                            3    Barbara     F      13     65.3       98.0
NOTE: 1                                                          The SAS System                         08:28 Wednesday, December 20, 2023
NOTE: 
NOTE: NOTE: Copyright (c) 2023 by SAS Institute Inc., Cary, NC, USA. 
NOTE: NOTE: SAS (r) Proprietary Software 9.4 (TS1M8) 
NOTE:       Licensed to *****.
NOTE: NOTE: This session is executing on the Linux ***** (LIN X64) platform.
NOTE: 
NOTE: 
NOTE: 
NOTE: NOTE: Analytical products:
NOTE:       
NOTE:       SAS/STAT 15.3
NOTE:       SAS/ETS 15.3
NOTE:       SAS/OR 15.3
NOTE:       SAS/IML 15.3
NOTE:       SAS/QC 15.3
NOTE: 
NOTE: NOTE: Additional host information:
NOTE: 
NOTE:  Linux LIN X64 ***** 
NOTE:       **** 
NOTE: 
NOTE: You are running SAS 9. Some SAS 8 files will be automatically converted 
NOTE: by the V9 engine; others are incompatible.  Please see 
NOTE: http://support.sas.com/rnd/migration/planning/platform/64bit.html
NOTE: 
NOTE: PROC MIGRATE will preserve current SAS file attributes and is 
NOTE: recommended for converting all your SAS libraries from any 
NOTE: SAS 8 release to SAS 9.  For details and examples, please see
NOTE: http://support.sas.com/rnd/migration/index.html
NOTE: 
NOTE: 
NOTE: This message is contained in the SAS news file, and is presented upon
NOTE: initialization.  Edit the file "news" in the "misc/base" directory to
NOTE: display site-specific news and information in the program log.
NOTE: The command line option "-nonews" will prevent this display.
NOTE: 
NOTE: 
NOTE: 
NOTE: 
NOTE: SAH231999I Batch, State, started, Startup command-line is: /apps/SAS/v9.4/SASHome/SASFoundation/9.4/sasexe/sas -fullstimer -nodms -stdio -terminal -nosyntaxcheck -pagesize MAX
NOTE: NOTE: SAS initialization used:
NOTE:       real time           0.05 seconds
NOTE:       user cpu time       0.01 seconds
NOTE:       system cpu time     0.01 seconds
NOTE:       memory              5945.21k
NOTE:       OS Memory           6564.00k
NOTE:       Timestamp           12/20/2023 08:28:58 AM
NOTE:       Step Count                        0  Switch Count  6
NOTE:       Page Faults                       3
NOTE:       Page Reclaims                     6163
NOTE:       Page Swaps                        0
NOTE:       Voluntary Context Switches        142
NOTE:       Involuntary Context Switches      7
NOTE:       Block Input Operations            3704
NOTE:       Block Output Operations           72
NOTE:       
NOTE: NOTE: A byte-order mark in the file "*****" (for fileref "#LN00005") indicates that the data is encoded in 
NOTE:       "utf-8".  This encoding will be used to process the file.
NOTE: 
NOTE: NOTE: AUTOEXEC processing beginning; file is *****/autoexec.sas.
NOTE: 
NOTE: NOT SAS STUDIO - autoexec ignored
NOTE: test_audit
NOTE: *****
NOTE: 
NOTE: NOTE: AUTOEXEC processing completed.
NOTE: 
NOTE: 1    proc print data=sashelp.class(obs=3);run;
NOTE: NOTE: There were 3 observations read from the data set SASHELP.CLASS.
NOTE: NOTE: The PROCEDURE PRINT printed page 1.
NOTE: NOTE: PROCEDURE PRINT used (Total process time):
NOTE:       real time           0.06 seconds
NOTE:       user cpu time       0.02 seconds
NOTE:       system cpu time     0.00 seconds
NOTE:       memory              2770.09k
NOTE:       OS Memory           9896.00k
NOTE:       Timestamp           12/20/2023 08:28:58 AM
NOTE:       Step Count                        1  Switch Count  1
NOTE:       Page Faults                       6
NOTE:       Page Reclaims                     1565
NOTE:       Page Swaps                        0
NOTE:       Voluntary Context Switches        63
NOTE:       Involuntary Context Switches      4
NOTE:       Block Input Operations            18016
NOTE:       Block Output Operations           0
NOTE:       
NOTE: SAH239999I Batch, State, stopped
NOTE: NOTE: SAS Institute Inc., SAS Campus Drive, Cary, NC USA 27513-2414
NOTE: NOTE: The SAS System used:
NOTE:       real time           0.13 seconds
NOTE:       user cpu time       0.03 seconds
NOTE:       system cpu time     0.03 seconds
NOTE:       memory              9094.12k
NOTE:       OS Memory           9896.00k
NOTE:       Timestamp           12/20/2023 08:28:58 AM
NOTE:       Step Count                        1  Switch Count  12
NOTE:       Page Faults                       10
NOTE:       Page Reclaims                     8314
NOTE:       Page Swaps                        0
NOTE:       Voluntary Context Switches        226
NOTE:       Involuntary Context Switches      13
NOTE:       Block Input Operations            23104
NOTE:       Block Output Operations           88
NOTE:         
tomweber-sas commented 6 months ago

Yes, it seems clear that the admin for this deployment configured it specifically for their purpose; batch submission and audit logging for those? Is there no other SAS deployment you can access? I workspace server like you could connect to using EG or something? One thing I noticed, looking back through this is (from your proc options output): LOGPARM=WRITE=BUFFERED ROLLOVER=NONE OPEN=REPLACE which goes along with the LOGCONFIGLOC= for the xml file. The one thing I'm curious about this is if it's the WRITE=BUFFERED, as opposed to ImmediateFlush that could help. I can't tell if the logging that's configured ALSO write to a file (as opposed to ONLY write to a file), but would go back to the client via STDIO or not, but if it would, and it's just not being written immediately (buffered for both output locations), that would produce the same symptom. You can see in this doc (https://go.documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/logug/p1agg698ws7m8wn10s93w2s7ohbz.htm) lines like <param name="ImmediateFlush" value="true"/> where it specifies to write the output immediately, not buffer it. It's a long shot, but if you can get that changed, maybe just copy their xml file to yours and add the immediatefluch line to the appender (if that takes president over how they have it configured) and it actually writes back to saspy, maybe it could work. Dunno, but something you might be able to try out.

Either way, have a great holiday! Tom

tomweber-sas commented 5 months ago

Hey @weg-jr , I'm going to close this, only because it's been open for months now and it's not something I can fix; it's the SAS configuration of the deployment you're trying to use.

But, feel free to open it back up if you get further on this or need more once your admin folks get involved. Always happy to help, just can't do anything about this so far.

Thanks! Tom