Toolkit for allowing inference and serving with PyTorch on SageMaker. Dockerfiles used for building SageMaker Pytorch Containers are at https://github.com/aws/deep-learning-containers.
Describe the bug
I am using a package called Farm to perform the training and the inference. The training works fine: I pass the requirements.txt and all the packages are correctly installed. When I try to deploy it, there is also no problem. But at the moment of inference, when it is required to import farm, I get an error showing that the package doesn't exist.
To reproduce
estimator = PyTorch(
base_job_name="imdb",
max_run= 60*30,
entry_point="train_aws.py",
source_dir="source",
framework_version="1.5",
py_version = 'py3',
instance_count=1,
role=role,
hyperparameters=hyperparameters,
instance_type= "ml.p3.2xlarge",
output_path="s3://<bucket>/opt/ml/model", # the S3 path where model is outputted
)
model = estimator.create_model(role=role, entry_point='inference.py')
mdm = multidatamodel.MultiDataModel(name="Models",
model_data_prefix="s3://<bucket>/opt/ml/model/",
model = model,
sagemaker_session=sess
)
predictor = mdm.deploy(initial_instance_count=1,instance_type='ml.t2.medium', endpoint_name="inference")
Note: when I create the model.tar.gz, I include 'requirements.txt' and 'inference.py' on code folder.
Expected behavior
Inference should be working, but the packages are not being correctly installed. I try to do it using brute force, including these lines on inference.py:
Describe the bug I am using a package called Farm to perform the training and the inference. The training works fine: I pass the requirements.txt and all the packages are correctly installed. When I try to deploy it, there is also no problem. But at the moment of inference, when it is required to import farm, I get an error showing that the package doesn't exist.
To reproduce
Note: when I create the model.tar.gz, I include 'requirements.txt' and 'inference.py' on code folder.
Expected behavior
Inference should be working, but the packages are not being correctly installed. I try to do it using brute force, including these lines on inference.py:
But then it says that another package is missing: ImportError: cannot import name 'rng_integers'
Is there a better way to install the packages I want inside the inference container?