png/jpg and html tables

[awb99]

iPython/Jupyter supports sending back images and html markup.

One of the most useful things in using Python in Jupyter is making inline charts and nicely printing array of data classes.

This can be quite easily achieved in icsharp also. I already have written a generator that generates html and text markup for a List and either displays all columns that the given type supports, or just selective columns by an specified parameter.

Sending this data (images or html markup)back is quite easy. images are encoded as base64 and html does not need more encoding.

What we should do is this: make a new project icsharp that contains the follwing classes:

1. MultiMediaEncoding interface. if an object is returned to icsharp that implements this interface, then icsharp sends the data (text/image/html/javascript) directly back to ipython jupyter. If users want to create their own graphical output routines, then they only need to reference this one library, and they are ready to o.
2. Encoding helper classes; some simple Encoding helpers; say one Encoding Helper that encodes bitmaps. The encoding helpers can be registered, so they would encode certain .net objects automatically without further calls needed
3. Grid Encoder (I have it, and will contribute this)

Should be all very simple to do, and I think very helpful/powerful.

[zabirauf]

@awb99 Thats a great suggestion. Do you have the Grid encoder in an open repo so that I can take a look at it to get a better idea of what you are proposing design wise.

[awb99]

@zabirauf Not yet. I have it already, but need to factor this part out to a dedicated project. I promise to do so!

[awb99]

@zabirauf https://github.com/awb99/ObjectPrinter.git
I factored out some basic functionality into this github project. I added a feature to print a list of objects of a specific type to a html table. For individual objects, I would rather export as a format more similar to JSON

I am not sure if we should put the dataType (jpg/html/text/...) into the ObjectPrinter; or if this should be done icsharp.

[zedr0n]

Just had a go at supporting PNG images and it seems to be working fine. See Commit

And then the code

    #r System.Drawing

    using System.Drawing;
    using System.Drawing.Imaging;

    public static partial class Test
    {
            public static string GetPng()
            {
                    using (Bitmap b = new Bitmap(50, 50)) 
                    {
                            using (Graphics g = Graphics.FromImage(b))
                            {
                                    g.Clear(Color.Green);
                            }
                            var str = new MemoryStream();
                            b.Save(str,ImageFormat.Png);
                            return System.Convert.ToBase64String(str.ToArray());
                    }        
            }
    }

can be used to output a green square in a separate cell with

 Test.GetPng()

But yes, ideally, the results of the execution should be parsed for type, mime type deduced and then encoded to base64. Doesn't sound that complicated

[zedr0n]

Html seems to be possible as well automatically but need a proper function to check for presence of tags. See commit

var html = @"<b>Bold</b>" + "<br>" + "<i>Italic</i>";

This would output Test Italic

[zedr0n]

Re the design concerns. While it might be nice to be able to specify explicitly in user code the return type it's a bit cumbersome dependencies wise. What would need to happen is creating a separate interfaces project which will be a common dependency of icsharp and your user code. And there's no nuget support in Ipython so instead of just distributing the kernel as an .exe file an additional dev package will need to be included plus installed to GAC as ScriptCS can only reference either those or absolute paths.

What I would rather suggest is a combination of automatic detection plus some conventions. E.g. if a function returns a Dictionary< string,object > then the first component is expected to represent the mime type. Then the conversion functions to base64 can be pulled into isharpcs directly.

Although now that I think about it one other option is maybe preinjecting the interfaces into each ScriptCS istance explicitly in ISharpCS code. Not sure if that's completely straightforward though. That could also be combined with attributes e.g.

[OutputType("text/html")]
public static string WriteHtml()
{
    return @"<b>Bold</b>" + "<br>" + "<i>Italic</i>";
}

[zedr0n]

Here's also an example of directly returning HTML with embedded images

    using System.Math;

    public static partial class Test
    {
            public static string Html()
            {
                    var html = @"<img src='data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==' alt='Red dot'></img>";
                    return html;
            }
    }

will result in a red dot displayed.

Just put all examples in a single notebook

https://github.com/zedr0n/icsharp/blob/dev/iPythonNotebook/MimeTest.ipynb