pharmapsychotic
commented
1 year ago Did you figure out how to run it locally?
Open ManuelVs opened 2 years ago
pharmapsychotic
commented
1 year ago Did you figure out how to run it locally?
Lunatik00
commented
7 months ago I was able to run endpoints locally, it is..... not simple.
Well, let's assume you know a little about how sagemaker works, mainly what goes into the /opt/ml/model is not the .tar.gz file but what is inside that file, I also will be using boto3.
First, the easy way, just run the container directly, docker run --rm -p 8080:8080 --gpus all -v $PATH_TO_FOLDER:/opt/ml/model $CONTAINER_NAME serve and you use request to send whatever you want to http://localhost:8080/invocations
This works, with the bonus that you don't need to use python, but it will be outside the sagemaker environment, so, to get a little closer to sagemaker you have to use the local mode, install sagemaker[local] and boto3, now the long part.
first import and initialize the local sagemaker
import boto3
import sagemaker
from sagemaker.local import LocalSession
boto_session = boto3.Session(region_name='us-west-2')
session = LocalSession(boto_session=boto_session)
session.config = {'local': {'local_code': True}}Explaination, botosession must be set with one region or you will get an error, the last line ensures that all will be done in local mode, or at least that is what it says in the documentation, now you have a sagemaker session, you can do everything and it will be done in the local version, but there are small differences, I use custom containers, so, I will be using the single container custom inference in sagemaker., first, we create the model
model = session.create_model(
name='local',
role='arn:aws:iam::123456789012:role/service-role/AmazonSageMaker', #dummy execution role
primary_container={
"Image": "$YOUR_IMAGE",
"ModelDataUrl": f"file://$FOLDER_WITH_TAR_GZ_CONTENTS",
"Environment": {}, # Variables de entorno
},
)The file is from your current working directory, and you must set environments even when they are empty
you create the config
config = session.sagemaker_client.create_endpoint_config(
EndpointConfigName='local-endpoint-config',
ProductionVariants=[
{
'VariantName': 'local-variant',
'ModelName': 'local',
'InstanceType': 'local',
'InitialInstanceCount': 1,
}
]
)As you can see, here InstanceType is different from the instances in aws, because it is local mode.
to finish you create the local endpoint
ep = session.sagemaker_client.create_endpoint(
EndpointName='local-endpoint',
EndpointConfigName='local-endpoint-config',
)All this is done in memory, meaning that it should be done in a .ipynb file to test and when that file is closed it will not be available.
Now you can invoke the endpoint as you would do in your final script, but using the local endpoint name
predictor = sagemaker.predictor.Predictor(endpoint_name='local-endpoint', sagemaker_session=session)
response = predictor.predict(payload)or
session.sagemaker_runtime_client.invoke_endpoint(
EndpointName='local-endpoint',
ContentType='application/json',
Body=payload,
)['Body']I couldn't find this info in any current guide and some of the details I have here were not explained in the little info I could gather, but this works as of today, and I used it 2 years ago, but with provided containers, I know it will probably not help you much, but I hope it would, to you or others that, like me, end up here when trying to make this work. AWS is seriously lacking on material to learn how to use their services.
What did you find confusing? Please describe.
I tried to extend the image adding my code and run it locally. However, the server does not start and it doesn't publish any logs from our scripts.
Dockerfile:
model.tar.gz:
Commands executed:
Output:
Describe how documentation can be improved
It would be nice to add a section in the README file (or similar) with an example on how to run the image / container in a local docker installation.
Additional context