Open DubiousCactus opened 2 years ago
I'm not sure batching tasks will be very helpful because we're already running OOM when looping through individual tasks with MAML. I'll leave this open for now and we can see if there's more interest in the future.
With PyTorch's latest update, the introduction of the functorch library allows, via composable function transforms, to "efficiently batching together tasks in the inner-loop of MAML". There seems to be a new way to vectorise gradient computation on a batch of tasks during the adaptation phase. I will try and look into this when I have some time, and maybe submit a PR. Unless someone knows how to do this? :)