search

aws / sagemaker-python-sdk

A library for training and deploying machine learning models on Amazon SageMaker
https://sagemaker.readthedocs.io/
Apache License 2.0
2.11k stars 1.14k forks source link

processing job infra spin up takes 16x longer than the job itself #4358

Open jlonge4 opened 11 months ago

jlonge4 commented 11 months ago

Describe the feature you'd like The feature I am proposing would be to implement some sort of warm pool (similar to estimator) or scheduled run to keep infra up to allow the processing job to take only as long as the script runtime. Requesting this due to my 30s script runtime taking 8min 17s total bc of infra allocation.

How would this feature be used? Please describe. This feature would be used to cut down on processing time and reduce latency

Describe alternatives you've considered Alternatives I have considered are sagemaker notebook jobs, and lambda container.

Additional context The goal of my processing job would read in a file, process and index it (create vector embeddings and add to docstore) using the library of my choice (langchain, haystack, etc)

lorenzwalthert commented 7 months ago

Did you consider Local Mode? We use this feature for local prototyping and it works well except some limitations in the context of Pipeline and Experiment integration, e.g. #4114.