Verify Agama Rolling & LTS with PTDB 0.5.2

AshburnLee commented 4 months ago

I'll find machines then for Rolling & LTS do the following:

Run standard workflow to verify if there are FAILED cases in the current scope.
Run only skiplist to verify if there are PASSED cases.

AshburnLee commented 4 months ago

No FAILED cases on Rolling Agama 881(jupyterhub) with PTDB 0.5.2
110 FAILED in Language test on LTS Agama 803 (fox125) with PTDB 0.5.2 RuntimeError: Triton Error [ZE]: 0x70000004

FAILED Cases

FAILED language/test_core.py::test_dot[1-64-128-128-4-False-True-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-4-False-True-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-False-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-4-False-False-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-4-False-False-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-4-False-False-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-4-False-False-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-True-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-True-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-True-none-tf32-float16-float16-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-True-True-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-True-True-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-True-none-tf32-float16-float16-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-True-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-True-True-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-True-True-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-True-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-4-True-False-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-True-False-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-True-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-True-False-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-True-False-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-False-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-True-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-False-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-True-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-True-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot_max_num_imprecise_acc[32-float8e4b15-128-256-128-128-256-256]
FAILED language/test_core.py::test_dot_max_num_imprecise_acc[0-float8e5-128-256-128-128-256-256]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-True-none-tf32-float16-float16-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-True-none-tf32-float16-float16-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-False-none-tf32-float16-float16-1_0]
FAILED language/test_core.py::test_dot_max_num_imprecise_acc[64-float8e5-128-256-128-128-256-256]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-True-none-tf32-float16-float16-1_1]
FAILED language/test_core.py::test_dot_max_num_imprecise_acc[0-float8e4nv-128-256-128-128-256-256]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-True-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-True-none-tf32-float16-float16-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-True-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-False-none-tf32-float16-float16-1_1]
FAILED language/test_core.py::test_dot_max_num_imprecise_acc[64-float8e4nv-128-256-128-128-256-256]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-True-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot_max_num_imprecise_acc[0-float8e4b15-128-256-128-128-256-256]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-False-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-False-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-True-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-False-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot_max_num_imprecise_acc[64-float8e4b15-128-256-128-128-256-256]
FAILED language/test_core.py::test_dot_max_num_imprecise_acc[32-float8e5-128-256-128-128-256-256]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-False-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-False-none-tf32-float16-float16-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot_max_num_imprecise_acc[128-float8e5-128-256-128-128-256-256]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-False-none-tf32-float16-float16-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot_max_num_imprecise_acc[32-float8e4nv-128-256-128-128-256-256]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-False-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-False-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot_max_num_imprecise_acc[128-float8e4nv-128-256-128-128-256-256]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-False-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-True-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-False-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot_max_num_imprecise_acc[128-float8e4b15-128-256-128-128-256-256]
FAILED language/test_core.py::test_dot[1-64-128-128-4-True-True-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-True-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-4-True-True-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-4-True-False-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-False-none-tf32-float16-float16-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-4-True-False-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-True-none-tf32-float16-float16-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-False-none-tf32-float16-float16-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-4-True-False-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-4-True-False-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-False-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-False-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-True-none-tf32-float16-float16-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-True-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-True-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-True-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-True-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-False-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-True-none-tf32-float16-float16-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-False-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-True-none-tf32-float16-float16-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-True-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-False-none-tf32-float16-float16-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-True-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-False-none-tf32-float16-float16-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-False-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-False-none-tf32-int8-int8-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-False-none-tf32-float16-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-False-none-tf32-int8-int8-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-False-none-tf32-float16-float16-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-False-chain-dot-ieee-float8e5-float32-1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-False-none-tf32-float16-float16-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-False-none-tf32-float16-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-False-chain-dot-ieee-float8e4nv-float32-1]

pbchekin commented 4 months ago

No FAILED cases on Rolling Agama 881(jupyterhub) with PTDB 0.5.2

Please also validate with Agama 914.32 (in the newly added jupyterhub session).

pbchekin commented 4 months ago

110 FAILED in Language test on LTS Agama 803 (fox125) with PTDB 0.5.2 RuntimeError: Triton Error [ZE]: 0x70000004

Looks like at least some of the failed tests are in the lts skip list, for example:

language/test_core.py::test_dot[1-64-128-128-4-False-True-none-tf32-float16-float32-1_0]

https://github.com/intel/intel-xpu-backend-for-triton/blob/llvm-target/scripts/skiplist/lts/language.txt#L94

Please re-validate with the lts skip list, or check if all failed tests are in the lts skip list.

AshburnLee commented 4 months ago

110 FAILED in Language test on LTS Agama 803 (fox125) with PTDB 0.5.2

Sorry, when on LTS, skiplist should have been specified before execution. So: TRITON_TEST_SKIPLIST_DIR=/PATH/TO/skiplist/lts ./scripts/test-triton.sh --ignore-errors There are no FAILED cases.

AshburnLee commented 4 months ago

Please also validate with Agama 914.32 (in the newly added jupyterhub session).

106 FAILED cases

FAILED language/test_core.py::test_noinline[shared]
FAILED language/test_core.py::test_join_with_mma
FAILED language/test_core.py::test_dot[1-16-16-16-4-False-False-add-cols-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-16-16-16-4-False-False-softmax-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-64-64-64-4-False-False-add-matrix-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-16-16-16-4-False-False-chain-dot-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-64-64-64-4-False-False-add-rows-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-64-64-32-4-False-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-64-32-4-False-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-64-64-4-False-False-none-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-32-32-128-16-True-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-32-32-128-16-True-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-64-64-4-False-False-add-cols-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-128-256-32-8-True-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-256-32-8-True-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-64-64-4-False-False-trans-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-32-32-128-16-True-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-32-32-128-16-True-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-64-64-4-False-False-softmax-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-128-256-32-8-True-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-256-32-8-True-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot_without_load[float32]
FAILED language/test_core.py::test_dot[1-32-32-128-16-False-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-32-32-128-16-False-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-4-True-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-32-32-128-16-False-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-4-True-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-32-128-16-False-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-64-64-4-False-False-chain-dot-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-128-256-32-8-False-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-256-32-8-False-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-32-32-4-False-False-none-tf32-float32-float32-1]
FAILED language/test_core.py::test_masked_load_shared_memory[dtype2]
FAILED language/test_core.py::test_dot[1-32-32-32-4-False-False-trans-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-128-256-32-8-False-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-256-32-8-False-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-32-32-4-False-False-add-matrix-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-128-16-32-4-True-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-4-True-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-4-True-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-16-32-4-True-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-32-32-4-False-False-add-rows-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-128-16-32-4-True-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-16-32-4-True-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-32-32-4-False-False-add-cols-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-128-16-32-4-False-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-32-32-32-4-False-False-softmax-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-128-16-32-4-False-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-32-32-4-False-False-chain-dot-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-128-16-32-4-False-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-16-32-4-False-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-16-16-16-4-False-False-none-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-16-16-16-4-False-False-trans-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-64-128-128-4-False-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-4-False-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-128-64-4-True-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-32-128-64-4-True-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-16-16-16-4-False-False-add-matrix-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-16-16-16-4-False-False-add-rows-tf32-float32-float32-1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-128-64-4-True-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-32-128-64-4-True-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-128-64-4-False-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-32-128-64-4-False-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-128-64-4-False-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-32-128-64-4-False-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-4-False-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-4-False-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-128-64-2-True-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-32-128-64-2-True-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-4-True-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-True-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-128-64-2-True-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-32-128-64-2-True-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-4-True-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-True-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-128-64-2-False-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-32-128-64-2-False-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-True-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-32-128-64-2-False-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-32-128-64-2-False-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-64-32-4-True-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-64-32-4-True-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-64-32-4-True-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-64-32-4-True-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-64-32-4-False-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-64-32-4-False-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-4-False-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-128-128-64-2-False-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-True-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-True-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-True-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-False-none-tf32-float32-float32-1_0]
FAILED language/test_core.py::test_dot[1-64-128-128-2-False-False-none-tf32-float32-float32-1_1]
FAILED language/test_core.py::test_dot_mulbroadcasted[float32]

AshburnLee commented 4 months ago

Summary:

No FAILED cases on Rolling Agama 881(jupyterhub) with PTDB 0.5.2
106 FAILED cases on Rolling Agama 914.32(jupyterhub) with PTDB 0.5.2
No FAILED cases on LTS Agama 803 (fox125) with PTDB 0.5.2

Two question:

I found in a Jupyterhub session, when I switch to root user to install (Option 2C) PTDB 0.5.2, ls /opt/intel/oneapi/logs still returns v=0.5.1. When I re-install PTDB 0.5.2, it told me, PTDB 0.5.2 is already installed. So I assume PTDB is updated, Is there another way of verifying the PTDB version? If it truly is not PTDB 0.5.2, Rolling Agama 881/Agama 914.32 with PTDB 0.5.2 will be re-validate.
When on a new driver version, I think we might need to verify the cases in skiplists only, If one PASSED, it's better to remove and add it to test scope. How do you think? How about add an option to test skiplists only?

pbchekin commented 4 months ago

CI runners have been updated to use PTDB 0.5.2 in #1644. No new failed tests with Agama 914.32 and PTDB 0.5.2.

pbchekin commented 4 months ago

we might need to verify the cases in skiplists only

I don't think it is really needed. After big updates like the recent one, just use your "enable unskip" mode to see if there are tests that no longer fail and update the skip lists respectively.

intel / intel-xpu-backend-for-triton

Verify Agama Rolling & LTS with PTDB 0.5.2 #1630