Ability to reconnect to the same connection from several python interpreters

[mtoma]

Is your feature request related to a problem? Please describe. I have several saspy submits that need to be done from separate python files and thus distinct calls to a python interpreter.

Describe the solution you'd like I'd like to be able to "save" (like with pickle) and "load" a connection and execute more code in the same session from a different python script.

Describe alternatives you've considered The only alternative I found so far is to code a simple HTTP or TCP python server and make the calls by asking the server to execute it in the same python session.

Additional context I'm working on a small project that is able to "extract" the code from the SAS Enterprise Guide .egp files (https://github.com/mtoma/PyEGP) and in one of my use cases each task needs to be executed from successive python interpreter calls. I considered converting the code into a jupyter notebook that would give me the python server/kernel functionality but I didn't find any option to execute one specific cell from a jupyer notebook. I'm using the IOM access method and it already starts the session to the server with a JAVA session. Wondering if a detach and attach to a IOM session would be something feasible in SASPy

[tomweber-sas]

Hey Michal, I'm out on vacation till the end of the week. I can catch up w/ you when I get back. In general, you can't really do this. But, I'd like to get more details on what you're trying to do and see what we can come up with. Can we touch base maybe Friday? Thanks! Tom

[mtoma]

Hi Tom, No problem, this is not an urgent issue. I was just wondering if this might be feasible as I noticed that if I dont call explicit sas.disconnect() several connections are "auto-closed" with a list of IDs printed out in stdout. This gave me the impression that the connections might still be around if not explicitly closed and thus there might be feasible to reconnect to them. Enjoy your holidays, we'll discuss this when you'll be back.

[tomweber-sas]

Thanks Michal, I did have a good vacation. I also did research this today, after getting back, so I can now have a better conversation with you about what you really need. As it turns out, we have Monday off; for the first time ever - new holiday for us. Is there a good time Tuesday? I have potential other calls till about noon here, eastern time. I think you're a bit east of me, if I remember right, so would early afternoon work? If that's still late for you, first thing Wed would work, then it wouldn't be too late in the day for you. Have a good weekend! Tom

[tomweber-sas]

Hey Michal, is this still a concern? I'd been waiting to hear back from you, and had been pulled of on other things. Though, I did investigate this and still want to understand other options for what you're trying to accomplish. Let me know if you have time to touch base on this. Thanks, Tom

[mtoma]

Hi Tom, The use case is simple. In essence the point would be to be able to somehow pass the active connection from one python script to another: script1.py: opens the connection and for example sets a libref. script2.py: reuses the librefs set by the previous scripts on the same SAS session.

[tomweber-sas]

So, when I looked into this when you first opened this, there's only one possible case where this could work, and it's pretty restrictive, and not real user friendly. So, not being something I can support generally, kinda begs the question of whether you can do what you are trying, from python itself. So unless you are using the one and only path, you couldn't get at this in the first place. Librefs can be assigned from different sessions and accessed by all. Not WORK, obviously, but any other directory you have available. One python session can read in different SAS code and submit it, against the single session, as opposed to needing different python sessions to submit their code individually. So, I was wondering if there was another way for you to accomplish what you're trying to do.

That being said, the one case where something like this could work is in the IOM access method, and only if your workspace server supports reconnecting: https://sassoftware.github.io/saspy/advanced-topics.html#disconnecting-from-an-iom-session-and-reconnecting-back-to-it

What I prototyped was to get back the token this uses and return it to your python call. In this case, you can start a new process, or just a new SASsession from the same process (doesn't know what process you're running from), by supplying this token on the next SASsession(), to reconnect to that. This is the same as how it works now, except I pass you back the token so you can then reconnect from a different session. The caveat with this is that each disconnect creates a different token, and you can't use that token but from one session, and you can't reconnect with a previous session after disconnecting if you use this token. So, it's only one at a time, sequentially with you having to manage this yourself. But, this does sound like what you're asking for, so even though it's an advanced feature, and can only be used in one connection scenario, if this is what you think you want, I can push this out to a branch for you o try out.

First, are you using IOM and when you try to do sas.disconnect(), does it work for you? If not, then this is all moot.

Here's an example of the prototype; some things may change, but this is good enough to try out:

tom64-5> python3.5
Python 3.5.6 (default, Nov 16 2018, 15:50:39)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-23)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import saspy
>>> sas = saspy.SASsession(cfgname='oda'); sas
SAS Connection established. Subprocess id is 21938

No encoding value provided. Will try to determine the correct encoding.
Setting encoding to utf_8 based upon the SAS session encoding value of utf-8.

Access Method         = IOM
SAS Config name       = oda
SAS Config file       = /opt/tom/github/saspy/saspy/sascfg_personal.py
WORK Path             = /saswork/SAS_work336200012EE0_odaws03-usw2.oda.sas.com/SAS_work7ACF00012EE0_odaws03-usw2.oda.sas.com/
SAS Version           = 9.04.01M6P11072018
SASPy Version         = 3.6.4
Teach me SAS          = False
Batch                 = False
Results               = Pandas
SAS Session Encoding  = utf-8
Python Encoding value = utf_8
SAS process Pid value = 77536

>>> sas.submitLOG('data a; x=1; run;')
5                                                          The SAS System                         Monday, March  1, 2021 03:18:00 PM

24         ods listing close;ods html5 (id=saspy_internal) file=_tomods1 options(bitmap_mode='inline') device=svg style=HTMLBlue;
24       ! ods graphics on / outputfmt=png;
25
26         data a; x=1; run;
27
28
29         ods html5 (id=saspy_internal) close;ods listing;
30
6                                                          The SAS System                         Monday, March  1, 2021 03:18:00 PM

31
>>> dataa = sas.sasdata('a')
>>> dataa
Libref  = WORK
Table   = a
Dsopts  = {}
Results = Pandas

>>> sas.disconnect()
'Succesfully disconnected. Be sure to have a valid network connection before submitting anything else.'
>>> # this is normal reconnect, same session same process
...
>>> dataa.head()
   x
0  1
>>> # now, lets disconnect and reconnect with the token from another process
...
>>> sas.disconnect()
'Succesfully disconnected. Be sure to have a valid network connection before submitting anything else.'
>>> sas.reconuri
'iom://odaws03-usw2.oda.sas.com:8591;;servername=B972C441-5B7D-074C-890D-1C0E73EFC46C,domain=DefaultAuth,CLSID=440196D4-90F0-11D0-9F41-00A024BB830C,encr=AES,encrLvl=everything/'
>>>
SAS Connection terminated. Subprocess id was 21938
tom64-5>
tom64-5> python3.5
Python 3.5.6 (default, Nov 16 2018, 15:50:39)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-23)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import saspy
>>> uri = 'iom://odaws03-usw2.oda.sas.com:8591;;servername=B972C441-5B7D-074C-890D-1C0E73EFC46C,domain=DefaultAuth,CLSID=440196D4-90F0-11D0-9F41-00A024BB830C,encr=AES,encrLvl=everything/'
>>> sas = saspy.SASsession(cfgname='oda', reconuri=uri); sas                                                                                                                                                                            SAS Connection established. Subprocess id is 21984

No encoding value provided. Will try to determine the correct encoding.
Setting encoding to utf_8 based upon the SAS session encoding value of utf-8.

Access Method         = IOM
SAS Config name       = oda
SAS Config file       = /opt/tom/github/saspy/saspy/sascfg_personal.py
WORK Path             = /saswork/SAS_work336200012EE0_odaws03-usw2.oda.sas.com/SAS_work7ACF00012EE0_odaws03-usw2.oda.sas.com/
SAS Version           = 9.04.01M6P11072018
SASPy Version         = 3.6.4
Teach me SAS          = False
Batch                 = False
Results               = Pandas
SAS Session Encoding  = utf-8
Python Encoding value = utf_8
SAS process Pid value = 77536

>>> dataa = sas.sasdata('a')
>>> dataa
Libref  = WORK
Table   = a
Dsopts  = {}
Results = Pandas

>>> dataa.head()
   x
0  1
>>>

So, what do you think, is this what you're looking for? Tom

[tomweber-sas]

Michal, I went ahead and pushed this to a new branch, 'procrecon' (PROCess RECONect) for you to try out, if this is what could work for you. Give it a go and see what you think. You can see the usage in the above example, which is just showing it reconnected to the same session by virtue of the table 'a' still being in the work library for the new session - so it's the same session. Again, each time you disconnect, you need to get the new sas.reconuri, and use that in the next session via reconuri=. Tom

[tomweber-sas]

Hey Michal, were you able to try this out, and did it work as expected?

Thanks! Tom

[tomweber-sas]

Hey Michal, I have added this to the production codebase, and it's now in V3.6.7. I hope you're doing well, and that this is working for your use case. I'll close this out, as it's now production. Let me know if you need anything else.

Thanks, Tom