issues
search
google
/
jetstream-pytorch
PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"
Apache License 2.0
32
stars
14
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Update Jetstream, add optional sampler args.
#178
qihqi
opened
4 days ago
0
Update Jetstream, add optional sampler args.
#177
qihqi
closed
4 days ago
0
Add gemma support in better cli
#176
qihqi
closed
4 days ago
0
Use kwargs to simplify the call sites a bit
#175
yixinshi
closed
1 week ago
0
Add mixtral support to new CLI
#174
qihqi
closed
1 week ago
0
Issues with prefill & generate
#173
qihqi
opened
1 week ago
0
Fix the performance regression with ragged attention on for llama2 7b.
#172
wang2yn84
closed
2 weeks ago
2
Replace repeat kv with proper GQA handling.
#171
wang2yn84
closed
2 weeks ago
3
fix ray engine crashes on multihost
#170
sixiang-google
closed
2 weeks ago
0
Error Running `run_ray_serve_interleave` with Llama3 8B
#169
ryanaoleary
opened
3 weeks ago
0
Add a script to measure speed of basic ops
#168
qihqi
closed
4 weeks ago
0
Add page attention manager and kvcache manager
#167
FanhaiLu1
closed
4 weeks ago
0
Add page attention manager and kvcache manager
#166
FanhaiLu1
closed
4 weeks ago
0
Fix TPU head resource name for v4 and v5e
#165
richardsliu
closed
1 month ago
0
Fix Ray engine crash on multihost
#164
richardsliu
closed
1 month ago
0
Fixed exhausted bug between head and workers
#163
FanhaiLu1
closed
1 month ago
0
Handle v5e-8 in run_ray_serve_interleave
#162
richardsliu
closed
1 month ago
0
Update Ray version in Dockerfile and add v5 configs
#161
richardsliu
closed
1 month ago
0
Add newest llama-3 benchmarks
#160
qihqi
closed
1 month ago
0
V5e8 ray
#159
FanhaiLu1
closed
1 month ago
0
Return np instead of jax array for prefill result tokens
#158
FanhaiLu1
closed
1 month ago
0
Correct typo enbedding -> embedding
#157
tengomucho
closed
1 month ago
1
commit act quant for conditional ffn
#156
qihqi
opened
1 month ago
0
Stacked cache mixtral.
#155
wang2yn84
closed
1 month ago
0
Stacked cache for MLPerf
#154
wang2yn84
closed
1 month ago
0
Add mlperf benchmark for offline for mixtral
#153
qihqi
closed
1 month ago
2
Set accumulate type to bf16 in activation quant
#152
lsy323
closed
1 month ago
1
Optimize cache update.
#151
wang2yn84
closed
4 weeks ago
7
Ray engine crashes on multihost when fetching Jax.array from prefill_ray
#150
richardsliu
closed
1 month ago
1
Fix blockwise sharding
#149
lsy323
opened
1 month ago
0
Add mlperf benchmark scripts in-tree.
#148
qihqi
closed
1 month ago
0
Make Ray engine and worker process prefill returning first token
#147
richardsliu
closed
1 month ago
0
Jetstream + RayServe deployment for interleave mode
#146
richardsliu
closed
1 month ago
0
Set JAX_PLATFORMS to "tpu, cpu" for ray worker
#145
richardsliu
closed
1 month ago
0
Fix exception in ray_worker
#144
richardsliu
closed
1 month ago
0
Make prefilling return first token for loadgen integration
#143
sixiang-google
closed
1 month ago
1
Add server tests
#142
bvrockwell
closed
1 month ago
1
Update benchmark command in README.md
#141
bhavya01
closed
2 months ago
0
add enable jax profiler to run_server
#140
bvrockwell
closed
2 months ago
0
Update README.md to state the limitation of accessing GCS when conver…
#139
wang2yn84
closed
2 months ago
0
Minor fixes to README
#138
wang2yn84
closed
2 months ago
0
Empty response returned for prompt responses when using run_server_with_ray.py and batch_size > 1
#137
richardsliu
opened
2 months ago
2
Add layer id in scope for each TransformerBlock layer
#136
FanhaiLu1
closed
2 months ago
0
Checkpoint conversion script breaks for meta-llama/llama-2-7b on HF
#135
vivianrwu
opened
2 months ago
0
prototyping better UX
#134
qihqi
closed
1 month ago
2
Add left aligned cache support.
#133
wang2yn84
closed
2 months ago
0
fix mixtral quantization scaler axis when dimension > 2
#132
sixiang-google
closed
2 months ago
0
Add test for Mixtral model.
#131
wang2yn84
closed
2 months ago
0
make sure GPU works
#130
qihqi
closed
2 months ago
0
Update README.md
#129
bhavya01
closed
2 months ago
0
Next