In addition, if we fix the above line, the following errors occur in the call of curve.hyperparameters[:3].numpy().tolist().
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[9], line 2
1 for curve in context:
----> 2 plt.plot(curve.t, curve.y, color=curve.hyperparameters[:3].numpy().tolist() + [0.5])
3 for curve in query:
4 plt.scatter(curve.t, curve.y, color=curve.hyperparameters[:3].numpy().tolist() + [0.5], s=7, marker="*")
TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
This is because of the Tensor object in context and query are supposed to be created on the CPU device, but it actually created on GPU if it's available. We should fix the instance creation of those tensors, or moving them onto CPU when calling .numpy() methods.
In addition, if we fix the above line as curve.hyperparameters[:3].cpu().numpy().tolist(), the following errors occur in the call of predictions = model.predict(context=context, query=query).
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Cell In[12], line 8
5 for hp in curve.hyperparameters:
6 assert hp.get_device() == 0
----> 8 predictions = model.predict(context=context, query=query)
9 predictions[0].ucb().shape, predictions[0].quantile(0.5).shape
File [/usr/local/lib/python3.11/site-packages/torch/utils/_contextlib.py:115](https://user-mamu--ll-workload-v1-zjnmw--12322.mnjg2.ingress.landlord.cluster-services.nf10.net/usr/local/lib/python3.11/site-packages/torch/utils/_contextlib.py#line=114), in context_decorator.<locals>.decorate_context(*args, **kwargs)
112 @functools.wraps(func)
113 def decorate_context(*args, **kwargs):
114 with ctx_factory():
--> 115 return func(*args, **kwargs)
File [/mnt/nfs-mnj-home-43/mamu/ifBO/ifbo/surrogate.py:91](https://user-mamu--ll-workload-v1-zjnmw--12322.mnjg2.ingress.landlord.cluster-services.nf10.net/lab/tree/ifBO/examples/ifBO/ifbo/surrogate.py#line=90), in FTPFN.predict(self, context, query)
82 @torch.no_grad()
83 def predict(
84 self, context: List[Curve], query: List[Curve]
85 ) -> List[PredictionResult]:
86 """Obtain the logits for the given context and query curves.
87
88 Function to perform Bayesian inference using FT-PFN that uses the logits obtained to
89 compute various measures like likelihood, UCB, EI, PI, and quantile.
90 """
---> 91 x_train, y_train, x_test = tokenize(context, query)
92 logits = self(x_train=x_train, y_train=y_train, x_test=x_test)
93 results = torch.split(logits, [len(curve.t) for curve in query], dim=0)
File [/mnt/nfs-mnj-home-43/mamu/ifBO/ifbo/utils.py:463](https://user-mamu--ll-workload-v1-zjnmw--12322.mnjg2.ingress.landlord.cluster-services.nf10.net/lab/tree/ifBO/examples/ifBO/ifbo/utils.py#line=462), in tokenize(context, query)
460 num_points = curve.t.size(0)
461 for i in range(num_points):
462 context_tokens.append(
--> 463 torch.cat(
464 (torch.tensor([curve_id, curve.t[i].item()]), curve.hyperparameters)
465 )
466 )
467 context_y_values.append(curve.y[i])
469 for curve in query:
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument tensors in method wrapper_CUDA_cat)
This is because that several tensors are created on CPU but others are created on GPU. This behavior should be fixed.
Steps to reproduce
Run the script on the environment where a GPU is available.
Expected behavior
The example notebook here should be executable with the GPU available environment. I would like to send PRs to fix these issues.
Environment
pip list.Error messages, stack traces, or logs
There are several reasons why we cannot execute the notebook. I will breakdown the reasons as follows.
The call of
context, query = ifbo.utils.detokenize(batch, context_size=single_eval_pos)fails as follows.This is because of the unintended module structures. We should fix this line not disclose priors.utils directly. https://github.com/automl/ifBO/blob/main/ifbo/__init__.py#L9
In addition, if we fix the above line, the following errors occur in the call of
curve.hyperparameters[:3].numpy().tolist().This is because of the Tensor object in context and query are supposed to be created on the CPU device, but it actually created on GPU if it's available. We should fix the instance creation of those tensors, or moving them onto CPU when calling
.numpy()methods.In addition, if we fix the above line as
curve.hyperparameters[:3].cpu().numpy().tolist(), the following errors occur in the call ofpredictions = model.predict(context=context, query=query).This is because that several tensors are created on CPU but others are created on GPU. This behavior should be fixed.
Steps to reproduce
Run the script on the environment where a GPU is available.