Open ibegleris opened 2 years ago
When deploying a question answering model with later versions of python, transformers and pytorch the logs do not seem to be seen in cloudwatch. If I deploy the same model but with older python and module versions the logs can be seen in cloudwatch.
You say with an older version you see logs but not with a newer one? The Cases you attached are vice versa. What is correct?
Yeah sorry, fixed it. I can see the logs with the old versions but not with the new ones.
@ibegleris i got in touch with the SageMaker Team. I ll come back when i know something
Looks like the logs are present in HF DLC with PT 1.8 as well.
I can confirm that with transformers_version = '4.17'
and pytorch_version = '1.10'
the logs appearing again.
Update: The issue is with the HF PT 1.9 containers not having latest version of sagemaker-inference
. We are working internally to resolve this.
When deploying a question answering model with later versions of python, transformers and pytorch the logs do not seem to be seen in cloudwatch. If I deploy the same model but with older python and module versions the logs can be seen in cloudwatch.
Recreate the issue
Case 1, No logs seen
Code
logs:
Case 2, logs visible
Code
logs: