search

pytorch-labs / segment-anything-fast

A batched offline inference oriented version of segment-anything
Apache License 2.0
1.19k stars 70 forks source link

Running on CPU. #94

Open shersoni610 opened 10 months ago

shersoni610 commented 10 months ago

Hello,

Is there a way to run this code on "CPU" instead of cuda. I get the following error which I changed device='cpu" in the example code.

torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised: LoweringException: AttributeError: 'PermuteView' object has no attribute 'freeze_layout' target: aten.convolution.default args[0]: TensorBox( PermuteView(data=StorageBox( ComputedBuffer(name='buf943', layout=FlexibleLayout('cpu', torch.bfloat16, size=[1, 64, 64, 1280], stride=[5242880, 81920, 1280, 1]), data=Pointwise( 'cpu', torch.bfloat16, def innerfn(index): , i1, i2, i3 = index tmp0 = ops.load(buf935, i3 + 1280 i2 + 81920 i1) tmp1 = ops.load(buf942, i3 + 1280 i2 + 81920 i1) tmp2 = tmp0 + tmp1 return tmp2 , ranges=[1, 64, 64, 1280], origin_node=add_352, origins={add_352} )) ), dims=[0, 3, 1, 2]) ) args[1]: TensorBox(StorageBox( InputBuffer(name='arg455_1', layout=FixedLayout('cpu', torch.bfloat16, size=[256, 1280, 1, 1], stride=[1280, 1, 1, 1])) )) args[2]: None args[3]: [1, 1] args[4]: [0, 0] args[5]: [1, 1] args[6]: False args[7]: [0, 0] args[8]: 1

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information

You can suppress this exception and fall back to eager by setting: import torch._dynamo torch._dynamo.config.suppress_errors = True

cpuhrsch commented 9 months ago

You can run this on CPU, but I don't know if it'll be much faster. We didn't really spend any time on that. The error you're getting seems unrelated to the device you're running this on though. Are you using a latest PyTorch nightly?