HTML Formatting of Log

[aaronsmith1234]

Is your feature request related to a problem? Please describe. I would like SASPy to have the option of returning the SASLog as an HTML object so that the color coding of errors\notes\etc would be taken care of for me. I am using SasPy to automate some SAS jobs that used to be EG projects, and the color coded log is helpful for EG users.

Describe the solution you'd like A variable that allows the log to be returned as HTML instead of just text

Describe alternatives you've considered I've thought about coding this myself to look for the things I care about (e.g. warnings\errors)

[tomweber-sas]

Hey Arron, let me investigate this and see what I can come up with ...

[aaronsmith1234]

Thank you! Happy to provide some examples if its helpful.

[tomweber-sas]

Thanks! I actually have the ability to do this in saspy, though it requires you have the pygments module installed. I'm playing around with it to see how I want to surface it. Just as a prototype, I created a colorLOG() method to try it out (you can pass the LOG from submit to it). Then I added an option to the other submit methods to ask for it colored. That works and I can add it to submit too, to return the colored HTML as the LOG instead of text, then you still have to sas.HTML(res['LOG']) as opposed to print(res['LOG']), so I'm looking at that as well as a global option that can be set, so if you want it all the time then you won't have to add the option to every submit*() you ever type in. Here's a couple screen shots of that so far. What do you think about that and is it what you're looking for?

[aaronsmith1234]

Yep, this is fantastic! Exactly what I was looking for.

Yeah, ideally it'd be something you set at the session level, and then the choice is remembered throughout the execution instead of having to retrieve each time. However, for my use case, that's not a big deal. The implementation above would work perfectly for my use case! If you push the branch, I can play around with it tomorrow and confirm it works for what I need. Thank you!

[tomweber-sas]

I agree that it should simply be a session attribute and not need to be set on individual submits. I was able to just quickly code that up to play with it interactively to see how the underlying functionality behaved. It all looks good though. I need to rework it to be a session attribute and test that all out. I was out the past 2 weeks and am multitasking to catch up and work on all the other stuff I'm doing, but I hope to have this also done and ready to provide you to try out, by the end of the day. Thanks! Tom

[tomweber-sas]

Ok Arron, I've pushed this change to a branch named colorLOG. Give it a try and see what you think. The new Configuration Definition key is colorLOG and it's a boolean. You can also override on the SASsession() method of course, like any of the other keys:

default  = {'saspath': '/opt/sasinside/SASHome/SASFoundation/9.4/bin/sas_u8',
            'colorLOG' : True,
            'options' : ["-fullstimer"]
            }

or 

import saspy
sas = saspy.SASsession(colorLOG=True)
sas

Again, it requires the pygments module for this to work, and it's html now instead of a just text, so from res = sas.submit() you need to use sas.HTML(res['LOG']) not print(res['LOG']), just like for the LST from submit(). The submitLOG() and submitLST(), of course, render it for you.

Let me know what you find/think, Tom

[aaronsmith1234]

Hi, This looks great, works for what I need! As a weird side note, I'm doing development in VSCode, popped open a notebook, to test, and as soon as I called conn.HTML(conn.submit("data temp; set sashelp.cars;run;")["LOG"]) (where conn is a SASsession object) the whole notebook rendered funny (everything turned white except code cells). I haven't had a chance to do any more digging, and for what its worth, I am in dark mode. Screenshots below:

before: After:

As you can see, its not just the output of the cell immediately below, its all the non-code cell areas in the notebook (including markdown cells). If someone isn't in dark mode, they may not have noticed this, and it may not be worth handling, but just thought I'd call it out.

[tomweber-sas]

Hmm, yeah, I've seen issues with Jupyter Notebooks (older versions, now Lab is much better) over the years with problems limiting HTML rendering to the local cell/window, whatever, and it affecting the whole page or outer context from just the output it's trying to render. This sounds like that.

Can you try running w/out the new changes and try a simple test with the existing code. Two cases, one with SAS html output and the other with the same pygments html output. With a SASsession of sas, try the following

cars = sas.sasdata('cars','sashelp')
cars
stat = sas.sasstat()
stat_results = stat.reg(model='horsepower = Cylinders EngineSize',data=cars)

then, try this to see about SAS html:

stat_results.DIAGNOSTICSPANEL

and this for the same pygments html of the log (this is in the existing code base and is what I'm using in the colorLOG branch)

stat_results.LOG

Do either/both of those cause the same issue in your UI? I would expect, at least, the stat_results.LOG would as it's the same code. If the DIAG... also does, then one thing to try would be to change the SAS ODS style to be a 'dark' scheme too, and see if that keep this from polluting the whole UI. You can change the in the config or at runtime with an attribute on the session object. FWIW, I'm not seeing this in either Notebook or LAB.

HTML_Style - This is the Style for ODS output, set from SAS_output_options {‘style’ : ‘’} value in your config file.
You can change this value on the fly by setting the value for this attribute.

sas.HTML_Style='Raven'

[aaronsmith1234]

The first one appears to note break the UI

but as soon as I run the second, it breaks the UI.

[aaronsmith1234]

I also just gave the raven theme a shot; no change, UI still breaks. This isn't a breaking change or anything for me, I can get what I need fine, so not a blocker :)

[tomweber-sas]

Ok, thanks for trying those out. So, the first one is the SAS HTML and it's not causing an issue. But the Pygments generated HTML for coloring the SASLOG is the culprit here. That's also why changing the ODS Style didn't matter, since that's not SAS HTML output. I don't know enough about html rendering to know what to do about this; I could try hacking at the Pygments HTML to see if there's something that could be change to keep that from happening. But I don't think that's a real answer.

So, are you saying that this is ok and you'd still rather have this functionality?

[aaronsmith1234]

Yes; for my actual use case I won't be rendering the html in a notebook in production. What would think about merging and closing this issue, and then raising a separate issue for the html rendering issue in notebooks? I.e. put it on the backlog, and if enough people use it and it bothers them, then tackle it then?

[tomweber-sas]

Ok, yes, I can merge this. I need to do a little more testing and such, but it seems pretty solid. I did try this in vscode (I don't use that but installed it before to test something out). I do see the issue when rendering the Pygments HTML. If you clear that cell, the whole thing goes back to the way it was (just FWIW). I don't see this problem with Jupyter Lab, which has a dark theme. The version of Jupyter Notebooks I have doesn't have a dark theme, so I can't be sure, but I don't see anything change it that either when rendering the Pygments HTML. I was able to change to a light theme in VSC and then I also don't see any noticeable change when rendering it either. So, maybe there's still an issue in Notebook like before where they can't segregate HTML w/in a cell from the overall page rendering. They fixed that with Lab, as it's not a problem, but at least the notebook in VSCode still seems to exhibit it. I don't know that I can do anything about that though.

[tomweber-sas]

Well, I found how to change the theme in Noitebook, I'm using The version of the notebook server is: 6.5.2 and in that, with a dark theme, the Pygments HTML for the LOG doesn't affect the notebook either; like Lab. So, I'm only seeing this in VSCode in it's Python Notebook. Just FYI.

[tomweber-sas]

I've merged, pushed doc and built a new release with this; V5.6.0. It's on github, on PyPI, and will be on conda-forge when their bot runs. If you're good with this we can close this issue.

Thanks! Tom

[aaronsmith1234]

Thanks, works great for what I need; much appreciated!!