[Enhancement][Bugfix] CPU inferencing does not work

[dogarrowtype]

Doing animatediff generate --device cpu does not work. Generating with cuda does, and the program works fine otherwise, but cpu support does not appear to be functional.

╭──────────────────────────── Traceback (most recent call last) ─────────────────────────────╮
│/animatediff-cli/src/animatediff/cli.py:154 in     │
│ generate                                                                                   │
│                                                                                            │
│   151 │   set_diffusers_verbosity_error()                                                  │
│   152 │                                                                                    │
│   153 │   device = torch.device(device)                                                    │
│ ❱ 154 │   device_info = torch.cuda.get_device_properties(device)                           │
│   155 │                                                                                    │
│   156 │   logger.info(device_info_str(device_info))                                        │
│   157 │   has_bf16 = torch.cuda.is_bf16_supported()                                        │
│                                                                                            │
│ /animatediff-cli/venv/lib/python3.10/site-packages │
│ /torch/cuda/__init__.py:396 in get_device_properties                                       │
│                                                                                            │
│    393 │   │   _CudaDeviceProperties: the properties of the device                         │
│    394 │   """                                                                             │
│    395 │   _lazy_init()  # will define _get_device_properties                              │
│ ❱  396 │   device = _get_device_index(device, optional=True)                               │
│    397 │   if device < 0 or device >= device_count():                                      │
│    398 │   │   raise AssertionError("Invalid device id")                                   │
│    399 │   return _get_device_properties(device)  # type: ignore[name-defined]             │
│                                                                                            │
│ /animatediff-cli/venv/lib/python3.10/site-packages │
│ /torch/cuda/_utils.py:32 in _get_device_index                                              │
│                                                                                            │
│   29 │   │   │   if device.type not in ['cuda', 'cpu']:                                    │
│   30 │   │   │   │   raise ValueError('Expected a cuda or cpu device, but got: {}'.format( │
│   31 │   │   elif device.type != 'cuda':                                                   │
│ ❱ 32 │   │   │   raise ValueError('Expected a cuda device, but got: {}'.format(device))    │
│   33 │   if not torch.jit.is_scripting():                                                  │
│   34 │   │   if isinstance(device, torch.cuda.device):                                     │
│   35 │   │   │   return device.idx                                                         │
╰────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: Expected a cuda device, but got: cpu

If you follow the error further down the line, it turns into an issue with fp16 not being possible on cpu. It might be extremely slow, but proper cpu support may be worth adding.

[neggles]

Ah, yeah, right now this does sorta assume a CUDA device and XFormers support. This is mostly because of the attention code I carried over from the original repo; it needs a refactor to use Diffusers' new-ish AttentionProcessor, which comes with a bunch of other fun improvements for both GPU and CPU use.

This should actually be fairly straightforward and is pretty much next on my todo list. In the short term I can make a few small tweaks to make CPU inference work - no promises about performance, though.

Will try to get to that in the next day or two; if you want to sorta hack around it yourself, change the dtype variables in src/animatediff/cli.py's generate() function to torch.float32, comment out the torch.cuda.device_info line that threw the error above, error and comment out the part of create_pipeline() where it enables xformers; that should cover it.

[neggles]

CPU support should work now 😄 feel free to reopen this if not!

[dogarrowtype]

So, it does get all the way to starting to generate (thank you), but there's some kind of insane memory leak and it tries to use more than 68gb of ram when generating. So it doesn't actually work.