Open eigenein opened 2 months ago
mps doesn't work at all. this model requires fp64
@bghira There is a workaround. See #30
the easiest way would be to convert it to a Apple Core ML model, similar to the other diffusion models, which will make possible to run it locally
Here are steps to troubleshoot and potentially resolve the issue:
Step-by-Step Resolution Ensure Correct --device Argument Passing: Ensure that the --device argument is being passed correctly to the Pipeline object within your script. You might need to modify the script to ensure it accepts and uses the --device argument appropriately.
Modify Your Script: If the script isn't handling the --device argument, you can modify it to include device handling. For example, in a PyTorch-based script, you would typically have something like: import torch def get_device(device_name): if device_name == 'mps': return torch.device('mps') if torch.backends.mps.is_available() else torch.device('cpu') elif device_name == 'cuda': return torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu') else: return torch.device('cpu')
Example usage: import argparse parser = argparse.ArgumentParser() parser.add_argument('--device', type=str, default='cpu', help='Device to run the model on') args = parser.parse_args() device = get_device(args.device) model.to(device) Check Compatibility: MPS currently supports only float32 tensors, and your model might be using float64 tensors. If this is the case, you can convert the model to float32. model = model.to(torch.float32) Convert to Core ML Model: If MPS support remains problematic, converting your model to an Apple Core ML model might be an effective workaround. This approach is similar to what is done with other diffusion models.
Use an Alternative Backend: If MPS doesn't support your model requirements (such as float64), you might need to use an alternative backend like CPU or CUDA (if you have a compatible NVIDIA GPU).
Example Script Modification: Here’s a snippet incorporating the --device argument correctly:
import argparse import torch
Function to get the appropriate device def get_device(device_name): if device_name == 'mps': return torch.device('mps') if torch.backends.mps.is_available() else torch.device('cpu') elif device_name == 'cuda': return torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu') else: return torch.device('cpu')
Argument parsing
parser = argparse.ArgumentParser()
parser.add_argument('--device', type=str, default='cpu', help='Device to run the model on')
args = parser.parse_args()
Select the device
device = get_device(args.device)
Assuming model
is your model object
model.to(device)
print(f'Model is running on {device}')
Workaround or Alternative Approaches
Consult Issue #30:
Refer to the workaround mentioned in issue #30 as suggested by @Xuzzo. This might provide additional insights or alternative solutions specific to your use case.
Apple Core ML Conversion: As @mgierschdev suggested, converting the model to Apple Core ML might be beneficial for running it on Apple Silicon efficiently.
nope. fp32 doesn't work
torch 2.4, m3 max
Script Modification Here’s a snippet incorporating the --device argument
import argparse import torch
def get_device(device_name): if device_name == 'mps': return torch.device('mps') if torch.backends.mps.is_available() else torch.device('cpu') elif device_name == 'cuda': return torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu') else: return torch.device('cpu')
parser = argparse.ArgumentParser() parser.add_argument('--device', type=str, default='cpu', help='Device to run the model on') args = parser.parse_args()
device = get_device(args.device) model = ... # Your model initialization model.to(device) print(f'Model is running on {device}')
It seems as if the
--device
were not passed down: