Problem with Dashboard

[lordfiftyfive]

I am running sherpa with tensorflow 2.1 on google colab. The first problem I ran into was that the previous link explaining how to run sherpa with tensorflow 2.1 in google didn't work fortunately I was able to resolve this and pull up the dashboard using

from tensorboard import notebook notebook.display(port=8880, height=1000)

Unfortunately it is not printing anything to the dashboard. I believe that it may be related to this error which I am receiving in my final line of code after it finishes running all the trials

---> 34 study.finalize(trial=trial)

TypeError: reduction operation 'argmin' not allowed for this dtype

[LarsHH]

Hi @lordfiftyfive ,

Thanks for the feedback! Mind if I ask which link you mean by the previous link?

The error you mentioned would definitely result in nothing being written to the dashboard. Can you share code for your study.add_observation ? It seems like objective in add_observation is getting a dtype that it argmin cannot be applied to.

Best, Lars

[lordfiftyfive]

I think I actually saw a link which explained how to run sherpa on google colab on a runtime which was not tensorflow 2.0 so after doing some digging I figured out that this was the correct way to pull up the sherpa dashboard with the latest tensorflow version.

from tensorboard import notebook notebook.display(port=8880, height=1000)

This is the code for my study. ' import sherpa.algorithms.bayesian_optimization as bayesian_optimization parameters = [sherpa.Continuous('lrinit', [0.01, 0.011], 'log')]

sherpa.Continuous('lrdecay', [1e-2, 1e-7], 'log')]

alg = bayesian_optimization.GPyOpt(max_num_trials=50)#sherpa.algorithms.GPyOpt('GP', num_initial_data_points='infer',initial_data_points=[0.1,0.11,0.12], acquisition_type='MPI',verbosity=True) study = sherpa.Study(parameters=parameters, algorithm=alg, lower_is_better=True,port=8884)

batch_size =19 loss = lambda y, rv_y: rv_y.variational_loss( y, kl_weight=np.array(batch_size, x.dtype) / x.shape[0]) num_iterations = 4 epochs = 18 '

' for trial in study:

lr = trial.parameters['lrinit'] model = tf.keras.Sequential([ tf.keras.Input(shape=(1,14), dtype=x.dtype), tf.keras.layers.LSTM(25,activation = 'relu',kernel_initializer='ones', dtype = x.dtype, use_bias=False),

tf.keras.layers.InputLayer(input_shape=(10),dtype=x.dtype),#put a 1 before the 9 later

  tf.keras.layers.Dense(50,kernel_initializer='ones', use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(75,kernel_initializer='ones', use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(100,kernel_initializer='ones', use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(125,kernel_initializer='ones', use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(150,kernel_initializer='ones',use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(175,kernel_initializer='ones',use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(200,kernel_initializer='ones',use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(225,kernel_initializer='ones',use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(250,kernel_initializer='ones',use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(225,kernel_initializer='ones',use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(200,kernel_initializer='ones',use_bias=False),
  #goal is to eventually replace the first dense layer with an LSTM layer
  #tf.keras.layers.LSTM
  #tf.keras.layers.TimeDistributed(Dense(vocabulary)))
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(150,kernel_initializer='ones',use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(125,kernel_initializer='ones', use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(100,kernel_initializer='ones',use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(75,kernel_initializer='ones', use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(50,kernel_initializer='ones',use_bias=False),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(25, kernel_initializer='ones',use_bias=False,),

  tfp.layers.VariationalGaussianProcess(
      num_inducing_points=num_inducing_points,
      kernel_provider=RBFKernelFn(dtype=x.dtype),
      inducing_index_points_initializer=tf.compat.v1.constant_initializer(
          np.linspace(0,x_range, num=1125,#num_inducing_points,
                      dtype=x.dtype)[..., np.newaxis]),
      unconstrained_observation_noise_variance_initializer=(tf.initializers.constant(100.0)
          ),
      event_shape=[num_distributions_over_Functions],jitter=1e-06

  )
  #in unconstrained thing replace astype with 

])

optimizer = tf.keras.optimizers.Adam(learning_rate=lr)# model.compile(optimizer=optimizer, loss=loss) for i in range(epochs):

model.fit(x, y,epochs=epochs, verbose=True,validation_split=0.2)
loss= model.evaluate(x[189::],y[189::])
loss = np.abs(loss)
study.add_observation(trial=trial,iteration=i,objective=loss,context={'loss':loss})

study.finalize(trial=trial) # '

[LarsHH]

Hmmm....does model.evaluate(x[189::],y[189::]) return a float? It must be related to the type of loss. Sherpa expects a float for the objective value.

Regarding the dashboard link. I hadn't used

from tensorboard import notebook
notebook.display(port=8880, height=1000)

before. Thanks for sharing. I normally just put localhost:<port> in the browser to get to the dashboard when running locally (which should correspond to the link given by Sherpa.

[lordfiftyfive]

Thanks! That did the trick.