V0 Dummy for issue #3

[MatheusCod]

Initial version of the Energy Profiler Plugin for TensorBoard #3

Objectives

1. Build a dummy plugin that, given random data, shows a graph visualization of that data.
   
2. Try integrate the dummy plugin with the TensorBoard Profiler plugin (trace_viewer).
   

[MatheusCod]

Update

I was able to get data from the backend (plugin.py) to the frontend (/static/index.js) by doing the following modifications:

index.js:

const newJson = await fetch('./printdata').then((response) => response.json());
  var str = JSON.stringify(newJson, null, 2);
  const newHead = document.createElement('h1');
  const newText = document.createTextNode(str);
  newHead.appendChild(newText);
  document.body.appendChild(newHead);

plugin.py:

A new route ("/printdata": self._serve_printdata) was added to get_plugin_apps function.

def get_plugin_apps(self):
        return {
            "/index.js": self._serve_js,
            "/printdata": self._serve_printdata,
            "/plotgraph": self._serve_plotgraph,
            "/tags": self._serve_tags,
            "/greetings": self._serve_greetings
        }

The new route function was implemented as following:

 @wrappers.Request.application
    def _serve_printdata(self, request):
        del request
        new_dict = {'Second': 'Print'}
        contents = json.dumps(new_dict)
        return werkzeug.Response(contents, content_type="application/json")

Preview

The header which contains the text "{ "Second": "Print" }" was generated by the modifications in the code described above.

[MatheusCod]

Update

I was able to create a graph visualization from the data passed from plugin.py to index.js.

Library used:

https://www.chartjs.org/ Files used: Chart.min.js and Chart.min.css

index.js:

Import Chart.min.css:

const stylesheet = document.createElement('link');
  stylesheet.rel = 'stylesheet';
  stylesheet.type = 'text/css'
  stylesheet.href = './static/Chart.min.css';

Import Chart.min.js:

const newChart = document.createElement("canvas");
  newChart.id = "myChart";
  newChart.width = "600";
  newChart.height = "400";
  newChart.style.margin = "auto";
  document.body.appendChild(newChart);

  const graphData = await fetch('./plotgraph').then((response) => response.json());

  var script = document.createElement('script');
  script.onload = function () {
    //const graphData = await fetch('./plotgraph').then((response) => response.json());
    var ctx = document.getElementById('myChart').getContext('2d');
    // Disable automatic style injection
    Chart.platform.disableCSSInjection = true;
    var myLineChart = new Chart(ctx, {
      type: 'line',
      data: {
          labels: graphData['x'],
          datasets: [{
              label: 'Energy Consumption Through Time',
              data: graphData['y'],
              borderColor: 'rgba(132, 132, 255, 1)',
              borderWidth: 1
          }]
      },
      options: {
                responsive: false,
                scales: {
                    xAxes: [{
                        ticks: {
                            beginAtZero: true
                        },
                     scaleLabel: {
                        display: true,
                        labelString: 'Time (s)'
                        }
                    }],
                    yAxes: [{
                        ticks: {
                            beginAtZero: true
                        },
                     scaleLabel: {
                        display: true,
                        labelString: 'Energy Consumption (W)'
                        }
                    }]
                }
            }
      });
  };
  script.src = "./static/Chart.min.js"//"https://cdn.jsdelivr.net/npm/chart.js@2.9.4/dist/Chart.min.js";
  document.head.appendChild(script);

plugin.py:

New imports:

import mimetypes
from tensorboard.backend import http_util

New variable:

_PLUGIN_DIRECTORY_PATH_PART = "/data/plugin/energy_profiling/"

New routes ("/plotgraph": self._serve_plotgraph) and ("/static/*": self._serve_static_file) were added to get_plugin_apps function.

def get_plugin_apps(self):
        return {
            "/index.js": self._serve_js,
            "/printdata": self._serve_printdata,
            "/plotgraph": self._serve_plotgraph,
            "/static/*": self._serve_static_file,
            "/tags": self._serve_tags,
            "/greetings": self._serve_greetings
        }

The new routes functions was implemented as following: Plot graph:

@wrappers.Request.application
    def _serve_plotgraph(self, request):
        del request
        size = random.randint(10, 15)
        new_dict = {
            "x": [i for i in range(size)],
            "y": [random.randint(0, 20) for i in range(size)]
        }
        contents = json.dumps(new_dict)
        return werkzeug.Response(contents, content_type="application/json")

Make the index.js read files from the ./static folder:

@wrappers.Request.application
    def _serve_static_file(self, request):
        """Returns a resource file from the static asset directory.
        Requests from the frontend have a path in this form:
        /data/plugin/example_raw_scalars/static/foo
        This serves the appropriate asset: ./static/foo.
        Checks the normpath to guard against path traversal attacks.
        """
        static_path_part = request.path[len(_PLUGIN_DIRECTORY_PATH_PART) :]
        resource_name = os.path.normpath(
            os.path.join(*static_path_part.split("/"))
        )
        if not resource_name.startswith("static" + os.path.sep):
            return http_util.Respond(
                request, "Not found", "text/plain", code=404
            )

        resource_path = os.path.join(os.path.dirname(__file__), resource_name)
        with open(resource_path, "rb") as read_file:
            mimetype = mimetypes.guess_type(resource_path)[0]
            return http_util.Respond(
                request, read_file.read(), content_type=mimetype
            )

Preview

[MatheusCod]

IMPORTANT UPDATE

Because of the following warning:

In 3.0, this warning will become an error:
X-Content-Type-Options is required to be "nosniff"

from

tensorboard/tensorboard/backend/security_validator.py:51

I had to comment/delete all

del request

from plugin.py.

And change all werkzeug.Response for http_util.Respond as the following: Change this:

return werkzeug.Response(contents, content_type="application/json")

To this:

return http_util.Respond(
            request, contents, content_type='application/json'
        )

[juliokiyoshi]

Update

I was able to create a graph visualization from the data passed in demo.py (here is an idea to create the graph from a database)

The architecture behind

First we need to define how the file will read the tensorboard output file, for this we will need to define the summary. in the project the summary created is summary_v2.py.

Now the demo.log creates the fake data using summary_v2. The data that is obtained from the demo.py we can get this data from multiplier_event in plugin.py . Now we need to create the channel between the back end and the front end, in this part it was building by _serve_greetings.

in can acess the data doing this in index.js:

 const data = await Promise.all(
    Object.entries(runToTags).flatMap(([run, tagToDescription]) =>
      Object.keys(tagToDescription).map((tag) =>
        fetch('./greetings?' + new URLSearchParams({run, tag}))
          .then((response) => response.json())
          .then((greetings) => ({
            run,
            tag,
            greetings,
          }))
      )
    )
  );
  console.log(data) // print the data in log of your browser.

Follow the photo below showing how to see in the log the data that was passed in the demo.py

Some problems

• The way i found to pass the data ( tensors obtained by the multiplexer function), you can only pass 10 tensor's by file create
• we need to find alternatives ways to pass the data

[MatheusCod]

Update

I was able to get another plugin visualization inside ours by using the iframe tag.
This could be a way to get the Profiler plugin graph and misc it with ours graph (like the mockup at the beginning of the this issue). In order to do that, it will be necessary to modify the iframe content to get only what is necessary (eg. removing the superior orange toolbar).

index.js:

var iframe = document.createElement("iframe");
iframe.id = "myframe";
iframe.width = screen.width;
iframe.height = screen.height / 2;
iframe.src = "/#scalars"
document.body.appendChild(iframe);

Preview

More information:

I also tried to get another plugin visualization inside ours by downloading the other plugin page with python.
I tried that using the following libraries: requests, BeatifulSoup, and Selenium library. And even downloading the page manually with the Ctrl+S command, but the downloaded page was always blank (although with what appeared to be a basic template for the tensorboard frontend).

[juliokiyoshi]

Update

I was able to get another plugin visualization inside ours by using the iframe tag. This could be a way to get the Profiler plugin graph and misc it with ours graph (like the mockup at the beginning of the this issue). In order to do that, it will be necessary to modify the iframe content to get only what is necessary (eg. removing the superior orange toolbar).

index.js:

var iframe = document.createElement("iframe");
iframe.id = "myframe";
iframe.width = screen.width;
iframe.height = screen.height / 2;
iframe.src = "/#scalars"
document.body.appendChild(iframe);

Preview

More information:

I also tried to get another plugin visualization inside ours by downloading the other plugin page with python. I tried that using the following libraries: requests, BeatifulSoup, and Selenium library. And even downloading the page manually with the Ctrl+S command, but the downloaded page was always blank (although with what appeared to be a basic template for the tensorboard frontend).

Add this line to remove Border

iframe.frameBorder =0;

Result:

To Do:

• The next thing to use the Iframe method is to remove the duplicated dashboard on Iframe

[MatheusCod]

Update

The toolbar was removed with the following code, which shows how to get access to the iframe content. It's important to notice that the iframe method has some limitations. For instance, if the iframe content is in a Shadow Root, only the .getElementById() method is apparently available. Futher investigation on that could be needed.

var iframe = document.getElementById("myframe");

iframe.onload = function() {
    var docframe = (iframe.contentWindow || iframe.contentDocument);
    if (docframe.document)docframe = docframe.document;
    docframe.body.appendChild(h1var);
    docframe.getElementById("printID").style.color = "blue";
    var itoolbar = document.getElementById("myframe").contentWindow.document.getElementsByTagName("tf-tensorboard")[0].root.getElementById("toolbar");
    itoolbar.style.display = "none";
    //document.getElementById("myframe").contentWindow.document.getElementsByTagName("tf-tensorboard")[0].root.getElementById("toolbar").style.display = "none";
  };

In the code above, the docframe variable could have been used to get the itoolbar variable, but I decided to show the complete command for understanding purpose.

The h1var constant is a simple h1 created with:

const h1var = document.createElement('h1');
h1var.id = 'printID';
const textvar = document.createTextNode("First print");
h1var.appendChild(textvar);

Preview

[juliokiyoshi]

UPDATE

Add a download button to download the data from a CSV file. First we need send the data from back-end to front-end, for this we need create a server to send the data.

Back-End:

first we need to create a directory called data, this directory is responsible to store the data in a csv, for this example i named the csv file as "data.csv". The folder structure should be like this:

IPMI_plugin_tensorboard
│
├── <-   data  directory created to store a csv file.
│     │
│     └── data.csv
│
├── demo.py   
│
├── __init__.py
│
├──  metada.py
│
├── summary_v2.py
│
├── plugin.py
│
└──    <- static directory
      │
      └── index.js
      │
      └── Chart.min.js
      │
      └── Chart.min.css

Now in plugin.py, we need to add the server:

  @wrappers.Request.application
    def _serve_data(self,request):
        df= pd.read_csv("./data/data.csv")
        power=df["Power"].tolist()
        time =df["Time"].tolist()
        dict={'x': power, 'y': time}
        contents = json.dumps(dict)
        return werkzeug.Response(contents, content_type="application/json") 

Notice i send the data to the front-end as a json-object. Now we need add this server on get_plugin_server:

 def get_plugin_apps(self):
        return {
            "/index.js": self._serve_js,
            "/tags": self._serve_tags,
        "/static/*": self._serve_static_file,
            "/index.html": self._serve_html,   
            "/scalars": self.scalars_route,
            "/greetings": self._serve_greetings,
            "/data": self._serve_data, # here is the channel add
        }

In Front-End:

Now we are able to get the data from the back-end. So, to get the data on front end we need to get by a request. So, in index.js we need to add this:

// to get the data:
  const data2= await fetch('./data').then((response)  => response.json());

• Add a download button(notice the console log will help you to see what is the output, on log in the browser):

  
  var csv=JSON2CSV(data2)
  var encodedUri = encodeURI(csv);
  var link = document.createElement("a");
  link.id = "link";
  link.href='data:text/csv;charset=utf-8,'+escape(csv)
  link.setAttribute("download", "data.csv");
  document.body.appendChild(link);
  var button = document.createElement('button');
  button.textContent='download';
  document.getElementById("link").appendChild(button);

function JSON2CSV(object) { var array1 = object['x']; var array2=object['y']; var str = ''; var line = '';

// get all distinct keys
let titles = ["Power(w)", "Time(s)"]; console.log(titles)

let htext = '"' + titles.join('","') + '"'; console.log('header:', htext); // add to str str += htext + '\r\n'; // // lines for (var i = 0; i < array1.length; i++) { var line = ''; line += ',"' + array1[i] + '"'+',"' + array2[i] + '"'; str += line.slice(1) + '\r\n'; } return str; }

### Preview

![image](https://user-images.githubusercontent.com/38574345/109015521-f5245080-7693-11eb-90fb-be0ff7961618.png)

[MatheusCod]

Our issue on the TensorBoard repository was answered.

Quote: "Hi @MatheusCod! We definitely don’t have any stability guarantees around the DOM structure of the TensorBoard shell or any particular plugins. Things like getElementsByTagName("tf-tensorboard") can change and in fact have changed in latest TensorBoard: tf-tensorboard has been deleted entirely. So I’d not advise relying on that structure.

I’d probably recommend either integrating the functionality that you need into the profile plugin or creating your own, separate plugin. The profile plugin lives at https://github.com/tensorflow/profiler. Do you think that either of those approaches might work for you?"

[MatheusCod]

Update

Graph element can be set inside the trace_viewer iframe with some limitations like no interactivty. The way of doing that is using the setInterval() function in order to see if some particular element is already loaded.

Preview

Code:

  const stylesheet = document.createElement('link');
  stylesheet.rel = 'stylesheet';
  stylesheet.type = 'text/css'
  stylesheet.href = './static/Chart.min.css';
  //document.head.appendChild(stylesheet);

  var iframe = document.createElement("iframe");
  iframe.id = "myframe";
  //iframe.width = screen.width;
  //iframe.height = screen.height / 2;
  //iframe.frameBorder = 0;
  iframe.src = "/#profile";
  iframe.style = "width: 100vw; height: 100vh;";
  document.body.appendChild(iframe);

  document.body.style = "margin: 0;";

  const newChart = document.createElement("canvas");
  newChart.id = "myChart";
  newChart.width = "600";
  newChart.height = "400";
  newChart.style.margin = "auto";
  document.body.appendChild(newChart);

  const graphData = await fetch('./plotgraph').then((response) => response.json());

  var script = document.createElement('script');
  //script.onload = function () {};
  script.src = "./static/Chart.min.js"//"https://cdn.jsdelivr.net/npm/chart.js@2.9.4/dist/Chart.min.js";
  document.head.appendChild(script);

  var iframe = document.getElementById("myframe");
  iframe.onload = function() {
    var docframe = (iframe.contentWindow || iframe.contentDocument);
    if (docframe.document)docframe = docframe.document;
    var iroot = (docframe.getElementsByTagName("tf-tensorboard")[0].root || docframe.getElementsByTagName("tf-tensorboard")[0].shadowRoot);
    // Delete toolbar
    var itoolbar = iroot.getElementById("toolbar");
    itoolbar.style.display = "none";

    iroot.appendChild(h1var);

    var iframe2;
    var docframe2;
    var iframe3;
    var docframe3;
    var load1 = setInterval(function() {getIframe2(iroot);}, 5000);
    var load2 = setInterval(function() {getIframe3(docframe2);}, 5000);

    function getIframe2(iroot) {
      iframe2 = iroot.getElementById("dashboard");
      if (iframe2) {
        clearInterval(load1);
        docframe2 = (iframe2.contentWindow || iframe2.contentDocument);
        if (docframe2.document)docframe2 = docframe2.document;
      }
    }

    function getIframe3(docframe2) {
      iframe3 = docframe2.getElementsByTagName("iframe")[0];
      if (iframe3) {
        clearInterval(load2);
        docframe3 = (iframe3.contentWindow || iframe3.contentDocument);
        if (docframe3.document)docframe3 = docframe3.document;
        const h1var2 = docframe3.createElement('h1');
        h1var2.id = 'printID2';
        const textvar2 = docframe3.createTextNode("First print");
        h1var2.appendChild(textvar2);
        console.log(docframe2.getElementById("printID"));

        const newChart2 = docframe3.createElement("canvas");
        newChart2.id = "myChart";
        newChart2.width = "600";
        newChart2.height = "400";
        newChart2.style.margin = "auto";

        docframe3.getElementById("event-details").appendChild(newChart2);

        var script2 = docframe3.createElement('script');
        script2.onload = function () {
          //const graphData = await fetch('./plotgraph').then((response) => response.json());
          var ctx = docframe3.getElementById('myChart').getContext('2d');
          // Disable automatic style injection
          Chart.platform.disableCSSInjection = true;
          var myLineChart = new Chart(ctx, {
            type: 'line',
            data: {
                labels: graphData['x'],
                datasets: [{
                    label: 'Power Consumption Through Time',
                    data: graphData['y'],
                    borderColor: 'rgba(132, 132, 255, 1)',
                    borderWidth: 1
                }]
            },
            options: {
                      responsive: false,
                      scales: {
                          xAxes: [{
                              ticks: {
                                  beginAtZero: true
                              },
                           scaleLabel: {
                              display: true,
                              labelString: 'Time (s)'
                              }
                          }],
                          yAxes: [{
                              ticks: {
                                  beginAtZero: true
                              },
                           scaleLabel: {
                              display: true,
                              labelString: 'Power Consumption (W)'
                              }
                          }]
                      }
                  }
            });
        };
        script2.src = script.src //"./static/Chart.min.js"//"https://cdn.jsdelivr.net/npm/chart.js@2.9.4/dist/Chart.min.js";
        docframe3.head.appendChild(script2);
      }
    }
  };

[MatheusCod]

For now, we'll maintain this version current state and move into the next version.