XY-plots

[KOLANICH]

XY-plots are needed. The difference is that in xy-plots there are no bins. A user passes either a tensor nx2 or nx3 or 3 tensors: x, y, color (color is optional). These curves are needed for each step, so ...

They should look like histograms, but 1 every point has 0 width 2 custom color provided as a tensor (optiknally) 3 custom marker for each point 4 it should be selectable if points should be connected by a line or not 5 polar coordinates should be supported too.

[wchargin]

Hi @KOLANICH—thanks for the suggestion. Can you give us a bit more background about the kinds of curves that you’re interested in plotting? For instance, we have an existing dashboard for plotting PR curves over time: https://github.com/tensorflow/tensorboard/blob/1.14/tensorboard/plugins/pr_curve/README.md

[KOLANICH]

I am predicting parameters of a distribution. So to make sure that the th9ngs go right I may want to plot reference PDF and the PDF I get from the paramaters predicted by network on the current step for some samples.

[wchargin]

That’s helpful; thanks. This doesn’t sound like it fits into any of the existing dashboards, so I’ve tagged it as a new plugin request.

We’re unlikely to add first-class support for this any time soon, but we are working on improving the plugin mechanism to make it easier for contributors like yourself to add such plugins. Preliminary versions are available now in the example plugin directory of this repository; I’ll post an update here when we have a full-featured solution.

[ucesfpa]

Hi, want to use the example given here but it is not working.

tensorboard.version '2.2.1'

tensorflow.version '2.2.0-rc3'

`

import os
import tensorflow as tf
from itertools import cycle
from tensorflow import keras
import matplotlib.pyplot as plt
from sklearn.metrics import average_precision_score
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input
import tf_losses as tfl
import tf_utils
import data
import argparse
import numpy as np
import pandas as pd
import io
from tensorflow.keras.metrics import Mean, MeanIoU, PrecisionAtRecall, Precision, Recall, Accuracy, AUC
from tensorboard import summary as summary_lib

class CustomCallback(keras.callbacks.Callback):
                def __init__(self, metrics, writer, n_thresholds=10):
                                super().__init__()

                                self.metrics = metrics
                                self.thresholds = n_thresholds
                                self.writer = writer
                def on_epoch_end(self, epoch, logs={}):

                        _prec = np.empty([self.thresholds])
                        _rec = np.empty([self.thresholds])
                        for t in range(self.thresholds):
                                _prec[t] = self.metrics[5 + t].result()
                                _rec[t] = 1 - t / self.thresholds

                        summary_lib.pr_curve_raw_data_op(name='pr_curve', # data = [_prec,_rec], display$
                                                        true_positive_counts=-np.ones(self.thresholds),
                                                        false_positive_counts=-np.ones(self.thresholds),
                                                        true_negative_counts=-np.ones(self.thresholds),
                                                        false_negative_counts=-np.ones(self.thresholds),
                                                        precision = _prec,
                                                        recall = _rec,
                                                        num_thresholds = self.thresholds,
                                                        display_name = 'PR_Curve',
                                                        description = 'Precision at several Recall thresholds.')

but I get this error:

ceback (most recent call last):
  File "train_2.py", line 276, in <module>
    train(args, params)
  File "train_2.py", line 229, in train
    use_multiprocessing=False,
  File "/home/ucesfpa/tf/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 66, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/home/ucesfpa/tf/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 879, in fit
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/home/ucesfpa/tf/lib/python3.6/site-packages/tensorflow/python/keras/callbacks.py", line 365, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "train_2.py", line 55, in on_epoch_end
    summary_lib.pr_curve_raw_data_op(name='pr_curve', # data = [_prec,_rec], display_name = 'PR_Curve', description = 'Precision at several Recall thresholds.')
AttributeError: module 'tensorboard.summary' has no attribute 'pr_curve_raw_data_op'
2020-07-09 14:44:01.805220: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1430] function cupti_interface_->EnableCallback( 0 , subscriber_, CUPTI_CB_DOMAIN_DRIVER_API, cbid)failed with error CUPTI could not be loaded or symbol could not be found.

Instead if I do: import tensorboard.summary as summary_lib

I get this error:

ceback (most recent call last):
  File "train_2.py", line 276, in <module>
    train(args, params)
  File "train_2.py", line 229, in train
    use_multiprocessing=False,
  File "/home/ucesfpa/tf/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 66, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/home/ucesfpa/tf/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 879, in fit
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/home/ucesfpa/tf/lib/python3.6/site-packages/tensorflow/python/keras/callbacks.py", line 365, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "train_2.py", line 55, in on_epoch_end
    summary_lib.pr_curve_raw_data_op(name='pr_curve', # data = [_prec,_rec], display_name = 'PR_Curve', description = 'Precision at several Recall thresholds.')
AttributeError: module 'tensorboard.summary' has no attribute 'pr_curve_raw_data_op'
2020-07-09 15:05:22.183583: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1430] function cupti_interface_->EnableCallback( 0 , subscriber_, CUPTI_CB_DOMAIN_DRIVER_API, cbid)failed with error CUPTI could not be loaded or symbol could not be found.

[ucesfpa]

tensorboard.version '2.2.1'

tensorflow.version '2.2.0-rc3'

I solved the problem reported in the previous comment removing the callback and creating the pr_summary directly in the training step as follows:

class CustomModel(Model):

        def train_step(self, data):
                # Unpack the data. Its structure depends on your model and
                # on what you pass to `fit()`.
                thresholds = 10
                x, y = data
                with tf.GradientTape() as tape:
                        y_pred = self(x, training=True)  # Forward pass
                        # Compute the loss value
                        # (the loss function is configured in `compile()`)
                        loss = self.compiled_loss(y, y_pred, regularization_losses=self.losses)

                # Compute gradients
                trainable_vars = self.trainable_variables
                gradients = tape.gradient(loss, trainable_vars)
                # Update weights
                self.optimizer.apply_gradients(zip(gradients, trainable_vars))
                # Update metrics (includes the metric that tracks the loss)
                self.compiled_metrics.update_state(tf.squeeze(y,axis=-1), tf.argmax(tf.nn.softmax(y_pred$
                summary_lib.op(name='pr',
                                labels = tf.squeeze(tf.cast(y, bool), axis=-1),
                                predictions = tf.cast(tf.argmax(tf.nn.softmax(y_pred,axis=-1),axis=-1),t$
                                num_thresholds = 20,
                                display_name = 'PR_Curve') 

                merged_summary=tf.compat.v1.summary.merge_all()
                file_writer.add_summary(merged_summary,step = self.optimizer.iterations)
                return {m.name: m.result() for m in self.metrics}

with

file_writer = tf.summary.create_file_writer(args['log_dir'])
model = CustomModel(inputs, outputs)
model.compile(
                optimizer=optimizer,
                loss = tfl.DiceLoss(from_logits=True),
                metrics=[train_metrics]
        )
model.fit(
        x=train_dataset,
        batch_size=params['batch_size'],
        epochs=params['Epochs'],
        verbose=1,
        callbacks=callbacks,
        validation_data=val_dataset,
        shuffle=True,
        class_weight=None,
        sample_weight=None,
        initial_epoch=0,
        steps_per_epoch=train_steps_per_epoch,
        validation_steps=val_steps_per_epoch,
        validation_freq=1,
        max_queue_size=10,
        workers=1,
        use_multiprocessing=False,
    )

I receive the following error:

train_step  *
        file_writer.add_summary(merged_summary,step = self.optimizer.iterations)

    AttributeError: 'ResourceSummaryWriter' object has no attribute 'add_summary'

Any recommendations about how to solve?

Thanks