Make fp8 work on older GPUs

[yorickvP]

Tested on RTX A5000.

• Use float32 matmul on older GPUs
• Offload based on total GPU memory instead of 'A40' in name
• Tell the FP8 code about offloading and compiling

[daanelson]

@yorickvP this is great, making 8-bit inference work regardless of hardware is useful. That said, when I run cog predict -i prompt=<whatever> on an A40, it takes ~10 minutes to compile and then there's no output. can you take a look? wary of pushing a broken path here