search

szymonmaszke / torchlambda

Lightweight tool to deploy PyTorch models to AWS Lambda
MIT License
125 stars 3 forks source link

Loading a custom model fails on AWS #15

Closed ellie-ei closed 3 years ago

ellie-ei commented 3 years ago

Thank you for the great work you've done!

I've been to trying to build a lambda function for a custom model that has an STFT front-end. Following all the steps, during inference I encountered the following error:

terminate called after throwing an instance of 'std::runtime_error'
  what():  The following operation failed in the TorchScript interpreter.
.
.
.
RuntimeError: fft: ATen not compiled with MKL support

END RequestId: 33c3a92e-d21d-428a-836f-979e29710e93

I moved on and implemented the STFT as a convolutional layer to fix the previous error. However, loading the new model fails by throwing the this error:

START RequestId: 87c2ded9-5be7-44ce-a709-f20a60fdba96 Version: $LATEST
terminate called after throwing an instance of 'c10::Error'
  what():  open file failed, file path: /opt/model.ptc
Exception raised from FileAdapter at ../caffe2/serialize/file_adapter.cc:11 (most recent call first):
frame #0: /var/task/bin/torchlambda() [0x5118f3]
frame #1: /var/task/bin/torchlambda() [0x1619d82]
frame #2: /var/task/bin/torchlambda() [0x1a2baf0]
frame #3: /var/task/bin/torchlambda() [0x41b46a]
frame #4: __libc_start_main + 0xea (0x7fcd29b1902a in /var/task/lib/libc.so.6)
frame #5: /var/task/bin/torchlambda() [0x459b7a]

END RequestId: 87c2ded9-5be7-44ce-a709-f20a60fdba96

Do you have any insight on how can I fix this issue?

szymonmaszke commented 3 years ago

Sorry for a very late reply. :( You could try to compile the torchlambda as a custom docker image with mkl support, see here, the whole command would be (assuming source is a folder containing your generated/written .cpp code):

torchlambda build source --pytorch USE_MKLDNN=ON

Please re-open if the error persists.