-
This does not appear to cause any errors, and has been this way for a long time, but ...
A) in components/cam/cime_config/buildnml:
my $spmd = '-spmd';
if ($MPILIB eq 'mpi-serial') {$sp…
-
Here's an overview of the features we intend to work on in the near future.
## Core Keras
### Saving & export
- Implement saving support for sharded models (sharded weights files).
- Improve…
-
Ported this issue from https://github.com/google/jax/issues/21562
This code
```python
import jax
import numpy as np
import jax.numpy as jnp
from jax.sharding import PartitionSpec as PS, Name…
-
Hello!
As per https://github.com/google/jax/discussions/23427, I'm noticing that XLA on CPU isn't doing a **fused** reduction sum for a very simple function if the input tensor is > 32 elements:
…
-
Co-authored with @SolitaryThinker @Yard1 @rkooo567
We are landing multi-step scheduling (#7000) to amortize scheduling overhead for better ITL and throughput. Since the first version of multi-step…
-
**What the problem is:**
Both single-node and sharded `TensorParallelMultiheadAttention`(#477) modules diverge (the forward output becomes `-inf` after less than 10 iterations). Also they produce d…
-
```
2023-07-18 23:43:01.804619: F external/xla/xla/hlo/ir/hlo_sharding.cc:961] Check failed: !IsManual()
Thread 1 "python" received signal SIGABRT, Aborted.
__pthread_kill_implementation (no_tid…
-
### Issue type
Bug
### Have you reproduced the bug with TensorFlow Nightly?
Yes
### Source
source
### TensorFlow version
git HEAD
### Custom code
No
### OS platform and distribution
Ubuntu …
-
## ❓ Questions and Help
I'm running this official [script here](https://github.com/pytorch/xla/blob/master/test/test_train_mp_imagenet_fsdp.py), but I only see two xla devices being used, xla:0 and…
-
I found this while poking around easylm: https://github.com/young-geng/EasyLM/blob/main/examples/pretrain_llama_7b.sh#L12
`
export LIBTPU_INIT_ARGS='--xla_jf_spmd_threshold_for_windowed_einsum_mib…
dlwh updated
8 months ago