aws sagemaker-huggingface-inference-toolkit issues

aws / sagemaker-huggingface-inference-toolkit

Apache License 2.0

235 stars 60 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Support load_lora_weights in inference API deploy

#131 haktan-suren opened 1 month ago
0
Remove Conversation pipeline utils

#130 ed-berry closed 1 month ago
4
SageMaker fails because Conversation object is not found

#129 adrihercer opened 3 months ago
6
Can’t create an inference service for models that depend on packages i.e. `espeak`

#128 ghubnerr opened 3 months ago
0
Update README.md

#127 dil-bhantos closed 3 months ago
0
Custom Inference Code - model_fn() takes more positional argument

#126 dil-bhantos opened 3 months ago
0
feat(initialize): default to first GPU when gpu_id not provided

#125 btruhand opened 4 months ago
0
fix(setup): install either tensorflow-macos or tensorflow depending on platform processor

#124 btruhand opened 4 months ago
0
Sagemaker HuggingfaceModel fails on phi3 model deployment

#123 manikawnth opened 4 months ago
2
Sagemaker endpoint inference Fails when following a tutorial

#122 IoannisDem opened 4 months ago
0
update version

#121 philschmid closed 4 months ago
0
Add Inferentai2 and Optimum Neuron Support

#120 philschmid closed 4 months ago
0
device kernel image is invalid

#119 geraldstanje1 opened 4 months ago
0
Add warning message to log this error.

#118 chen3933 closed 3 months ago
9
Sagemaker endpoint doesn't use GPU (instance ml.g4dn.xlarge)

#117 jypucca opened 5 months ago
1
Sagemaker inference not loading model weight from s3

#116 saichethan-a opened 5 months ago
2
Make DEFAULT_HF_HUB_MODEL_EXPORT_DIRECTORY configurable

#115 pcolazurdo opened 6 months ago
0
Log if handler service is using default or custom functions implementation

#114 aduverger closed 5 months ago
3
Add support for additional tasks via HuggingFace TaskManager

#113 samruds closed 6 months ago
0
ARCHITECTURES_2_TASK is limiting the tasks able to be deployed with HF DLC

#112 gwang111 closed 6 months ago
11
Endpoint creation completes before custom model_fn finishes loading resources

#111 Tripping-Hazard opened 7 months ago
0
HF_TASK Enviournment Variable error

#110 316usman opened 8 months ago
1
Support multiple return sequences

#109 Elanmarkowitz opened 9 months ago
0
Sagemaker endpoint inferencing error with HF model loading from s3bucket with new transformer update

#108 miteshkotak opened 10 months ago
5
🏷️ invalid

#107 pierrishm1987 opened 11 months ago
0
No support for multi-GPU

#106 parviste-fortum opened 11 months ago
3
get_pipeline function passes Path object rather than PretrainedTokenizer

#105 jpang32 opened 1 year ago
0
Add diffusers utils

#104 philschmid closed 10 months ago
5
Add context to handler functions

#103 sachanub closed 1 year ago
1
Using custom inference script and models from Hub

#102 Tarun02 opened 1 year ago
1
Dummy PR

#101 sachanub closed 1 year ago
0
Prioritize safetensors format and support sharded weights

#100 BaiqingL closed 5 months ago
1
Where is the logic for detecting custom inference.py?

#99 BaiqingL closed 1 year ago
6
How to enable Batch inference on AWS deployed Serverless model from Hub?

#98 jmparejaz opened 1 year ago
1
How can I delpoy a model with AWS S3 and without downloading model from hunggingface via TGI image on Sagemaker?

#97 weiZhenkun closed 1 year ago
2
Error on Sagemaker deployment for v1.0.1

#96 mer0mingian closed 1 year ago
1
Release 2 2 0

#95 philschmid closed 1 year ago
0
SageMaker deployment errors

#94 jonrossclaytor opened 1 year ago
2
Sharded checkpoint support

#93 JimAllanson closed 5 months ago
1
Add `optimum-neuron` pipeline support for zero code deployment.

#92 philschmid closed 1 year ago
0
Dead Link for Available HF_Tasks

#91 dcsena opened 1 year ago
1
[DOCS] List of available HF_TASK and default inference scripts

#90 austinmw closed 1 year ago
4
How to access CustomAttributes in async inferece request input_fn

#89 dcdieci opened 1 year ago
1
trust_remote_code=True in new Hugging Face LLM Inference Container for Amazon SageMaker

#88 krokoko closed 1 year ago
2
InternalServerException at runtime

#87 krokoko closed 7 months ago
3
Update version to 2.1.2

#86 maaquib closed 1 year ago
0
Support passing model_kwargs to pipeline

#85 lukealexmiller opened 1 year ago
1
Add support for JSON encoding `torch.tensor` to keep it consistent with sagemaker-inference-toolkit

#84 maaquib closed 1 year ago
0
Data format for inference

#83 alvaropp opened 1 year ago
1
Update mms_model_server.py

#81 ShubhamRaiGit closed 1 year ago
0