search

Mathux / TEMOS

Official PyTorch implementation of the paper "TEMOS: Generating diverse human motions from textual descriptions", ECCV 2022 (Oral)
https://mathis.petrovich.fr/temos/
MIT License
366 stars 25 forks source link

Training on multi-GPU #9

Closed Xuehao-Gao closed 9 months ago

Xuehao-Gao commented 2 years ago

Hi dear author, I would like to train the temos on multi-GPU. As shown in Fig.1, It runs well when using one GPU as default. But, when I try to run it on multi-GPU and set the gpus: 4 in the trainer config file, I meet a Type Error shown in Fig.2. So, I would like to know how to train temos on multi-GPU well.

One GPU Training: 微信截图_20220610094505

Four GPUs Training: 微信截图_20220610094621

Mathux commented 2 years ago

Hello @Lucky-Maximize,

I just solved this "pickle problem" with this commit: https://github.com/Mathux/TEMOS/commit/3e0efdd8acc253e7144a819c98e755b9e5c4dc1e

Actually, the current code is not designed for GPU-training and it is not working yet. When I will have more time, I will probably investigate and make it work. I need to redesign a bit the data loading, and the metric computation.