We noticed a massive slowdown (~20x) after upgrading packages. Turned out to be a regression in pytorch-geometric in the handling of sparse tensors (https://github.com/pyg-team/pytorch_geometric/pull/5299). This has been fixed in August but hasn't made it to a release yet, so our best bet was to downgrade pytorch-geometric (to 2.0.4). This also meant we had to use older versions of pytorch (1.11) and cudatoolkit (11.3).
We noticed a massive slowdown (~20x) after upgrading packages. Turned out to be a regression in
pytorch-geometricin the handling of sparse tensors (https://github.com/pyg-team/pytorch_geometric/pull/5299). This has been fixed in August but hasn't made it to a release yet, so our best bet was to downgradepytorch-geometric(to2.0.4). This also meant we had to use older versions ofpytorch(1.11) andcudatoolkit(11.3).