search

Ryan-Rhys / Heteroscedastic-BO

Heteroscedastic Bayesian Optimisation in Numpy
21 stars 2 forks source link

Question about Tanimoto kernel, ARD #25

Open mohammad-saber opened 3 years ago

mohammad-saber commented 3 years ago

Thank you for sharing your great code. I am using the implementation of Tanimoto kernel. I was wondering if I want to use it as ARD kernel, how to calculate the final result in the following formula:

K = sigma_f**2 * outer_product/denominator

In case of non-ARD, sigma_f is scaler and there is no problem with calculation. In case of ARD, sigma_f is a matrix with size [1, No. of features]. And, there will an error about incompatibility of matrix size in the above formula.

Ryan-Rhys commented 3 years ago

Hi Mohammad,

Thanks for your comment!

The FlowMO library:

https://github.com/Ryan-Rhys/FlowMO/blob/master/GP/kernels.py

is probably a better example for the Tanimoto kernel. sigma_f in this instance is the output scale for the kernel (i.e. what GPyTorch would apply via its ScaleKernel class). In terms of ARD, I have typically seen it applied through kernel lengthscales. In the FlowMO code above we chose not to include lengthscales as hyperparameters for the Tanimoto kernel as it is intended to operate on molecular bit vectors!

Best, Ryan

mohammad-saber commented 2 years ago

Dear Ryan,

Thank you for your reply. It was very helpful.

I would like to confirm one more thing. The shape of input (X) is 2D. If the input has only one feature, I think I need to reshape it to a 2D numpy array like [No. of Observations, 1].

I think if I use 1D numpy array like [No. of Observations, ], it will be wrong. Is my understanding correct?

Ryan-Rhys commented 2 years ago

Hi Mohammad,

Are you referring to the numpy implementation here or to the Tanimoto kernel in FlowMO?

Best, Ryan

mohammad-saber commented 2 years ago

I am referring to numpy implementation here.

Ryan-Rhys commented 2 years ago

Hi Mohammad,

Judging from the test for the Tanimoto kernel in test_mlhgp.py module it seems that the input is assumed to be [No. of Observations, 1]. If you would prefer to avoid using TensorFlow or PyTorch, you might be interested in the following implementation of the Tanimoto kernel in scikit-learn written by Dries Van Rompaey:

https://github.com/Ryan-Rhys/The-Photoswitch-Dataset/blob/master/property_prediction/sklearn_kernels.py

Best, Ryan

mohammad-saber commented 2 years ago

Thank you very much for your help. Best wishes, Mohammad