issues
search
aws-samples
/
foundation-model-benchmarking-tool
Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
MIT No Attribution
146
stars
21
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Lllama3 70B on ec2
#139
antara678
closed
1 day ago
0
adding phi config file
#138
madhurprash
closed
1 day ago
0
tokenizer file handling fix
#137
madhurprash
closed
1 day ago
0
handling endpoint metrics
#136
madhurprash
closed
4 days ago
0
Bump litellm from 1.34.0 to 1.40.29 in /src/fmbench
#135
dependabot[bot]
opened
4 days ago
0
config file for llama 2 70B
#134
antara678
opened
4 days ago
0
min/max iterations per combination handling
#133
madhurprash
closed
5 days ago
0
Update eks_cluster-creation_steps.md
#132
antara678
closed
6 days ago
0
Add Time to first token (TTFT), time per output token (TPOT) and time to last token (TTLT)
#131
madhurprash
opened
6 days ago
0
added the context for the respective sections for EKS.
#130
bainskb
closed
6 days ago
0
cloudformation template generalized for gov cloud
#129
madhurprash
closed
1 week ago
0
handling deployment return values from various scripts
#128
madhurprash
closed
1 week ago
0
Containerize FMBench
#127
dheerajoruganty
closed
1 week ago
0
EKS integration with FMBench
#126
madhurprash
closed
1 week ago
0
Ec2 deployment
#125
antara678
closed
1 week ago
0
update create combination with test count limit
#124
haozhx23
closed
5 days ago
0
add write multiple to s3
#123
haozhx23
closed
2 days ago
0
Bump urllib3 from 2.0.7 to 2.2.2
#122
dependabot[bot]
opened
2 weeks ago
0
added the changes to get get do-eks to work up untill the metrics not…
#121
bainskb
closed
2 weeks ago
0
Is it possible to run FMBench on an EC2 instance on endpoints currently not supported on Sagemaker.
#120
prasad-nair-amd
opened
2 weeks ago
1
Bump tornado from 6.4 to 6.4.1
#119
dependabot[bot]
opened
2 weeks ago
0
generate flatten test combinations
#118
haozhx23
closed
2 weeks ago
0
s3 upload concurrency
#117
haozhx23
closed
2 weeks ago
0
gemma support added to FMBench
#116
madhurprash
closed
2 weeks ago
0
adding gemma support
#115
madhurprash
closed
3 weeks ago
0
Reports only show information per instance instead of per endpoint
#114
jimburtoft
opened
3 weeks ago
0
adding EC2 djl rest prediction support for llama3 on fmbench
#113
madhurprash
closed
3 weeks ago
0
Bump litellm from 1.34.0 to 1.40.0 in /src/fmbench
#112
dependabot[bot]
opened
3 weeks ago
0
KeyError: "Column(s) ['GPUMemoryUtilization', 'GPUUtilization'] do not exist" for non-GPU instances
#111
jimburtoft
closed
5 days ago
5
Do eks
#110
bainskb
closed
3 weeks ago
1
Update config-bedrock-llama3.yml
#109
antara678
closed
4 weeks ago
0
Python Script to Plot Sagemaker Endpoint Utilization Metrics , Heatmap to visualize Cost for Various RPMs
#108
dheerajoruganty
closed
3 weeks ago
1
Add Python Script to Retrieve SageMaker Endpoint Invocation and Utilization Metrics
#107
dheerajoruganty
closed
1 month ago
0
config fixes byoe
#106
madhurprash
closed
1 month ago
0
Loosen dependencies for more flexible installation
#105
athewsey
opened
1 month ago
1
Support Python 3.10 for SageMaker Studio
#104
athewsey
closed
3 weeks ago
1
Error: need to escape, but no escapechar set
#103
athewsey
closed
3 weeks ago
1
No such file or directory: '/tmp/fmbench-read/tokenizer'
#102
athewsey
closed
3 weeks ago
2
updating configs additional metadata for llama3-8b
#101
madhurprash
closed
1 month ago
0
adding s3 support for local mode sm endpoints
#100
madhurprash
closed
1 month ago
0
readme update
#99
madhurprash
closed
1 month ago
0
Bump requests from 2.31.0 to 2.32.2
#98
dependabot[bot]
opened
1 month ago
0
Add render_fmbench_website.py and Update README
#97
dheerajoruganty
closed
1 month ago
0
Config files for Llama3 for different instance types
#96
aarora79
closed
1 month ago
0
config_filepath is incorrect
#95
jimburtoft
closed
3 weeks ago
1
Support Python 3.10 for SageMaker Studio notebooks
#94
athewsey
closed
3 weeks ago
1
Grammar nits
#93
jimburtoft
closed
1 month ago
0
fix broken symlinks
#92
aarora79
closed
1 month ago
0
config file reorg
#91
aarora79
closed
1 month ago
0
html report generation with quarto, llama3-8b on trn1
#90
aarora79
closed
1 month ago
0
Next