This is not currently the purpose of EagerPy. EagerPy, right now, gives you an API to write code that works the same way with PyTorch tensors and TensorFlow tensors, but not mixing them simultaneously. In principle, we might be able to add that if it is needed (keep in mind though that TensorFlow and PyTorch on the same GPU don't behave that well).
This is not currently the purpose of EagerPy. EagerPy, right now, gives you an API to write code that works the same way with PyTorch tensors and TensorFlow tensors, but not mixing them simultaneously. In principle, we might be able to add that if it is needed (keep in mind though that TensorFlow and PyTorch on the same GPU don't behave that well).