search

SKrisanski / FSCT

GNU General Public License v3.0
143 stars 60 forks source link

Pytorch 2.2.1 and Multipule GPU's #39

Open Rotoslider opened 5 months ago

Rotoslider commented 5 months ago

I forked your wonderful program and updated the requirements.txt and a few other files to now work with python 3.10 and Pytorch 2.2.1. I also fixed some of the warnings for DataLoader and the Pandas float issue from upgrading. I have it running smoothly on a old server with 4 Tesla P40 24gb gpus. I was wondering if you have given any thought to what needs to be done to get this working with multiple gpu's? Could you tell me what scripts use the gpu and what use the cpu?

PyTorch, including torch_geometric, supports multi-GPU setups using Data Parallelism and Distributed Data Parallelism (DDP). Multi-GPU Inference Basics

Data Parallelism: This involves splitting the input data across different GPUs, where each GPU computes a part of the data and then aggregates the results. PyTorch’s DataParallel module can be used for this purpose, but it's more suited for single-node setups.

Distributed Data Parallelism (DDP): DDP is more scalable and efficient, especially for multi-node setups or when maximizing GPU utilization is crucial. It involves distributing both the model and the data across multiple GPUs, with each GPU (or process) handling a portion of the input data and the model.
BoonlueKac commented 5 months ago

Thank you, let me try.

tariqislam commented 1 week ago

Hi @Rotoslider, great work updating the requirements! Were you able to run this on a Mac? I am assuming no, because I don't see any MacOSX libraries in https://pytorch-geometric.com/whl/torch-2.2.1+cu121.html, but just verifying. Also, do you run this on Linux?

Thanks!

Rotoslider commented 1 week ago

Yes this is running on Ubuntu 22.04 I do not own a Mac so I am unsure if the changes will run on one.

tariqislam commented 1 week ago

Awesome, thanks @Rotoslider !