torch.compile error on scaled_dot_product_attention in transformers.LlamaForCausalLM when providing attention_mask: RuntimeError: (*bias): last dimension must be contiguous
Hello there! From the DISABLED prefix in this issue title, it looks like you are attempting to disabled a job in PyTorch CI. The information I have parsed is below:
When I use register_full_backward_hook and register_forward_hook in nn.MultiheadAttention, The MultiheadAttention executed wrong conditional branch and get wrong backward grads.
The "register_module" function got a "key_description=" and reported that "Exception 0xc0000005 encountered at address 0x7ff7fb3dfbf3: Access violation reading location 0xffffffffffffffff"
Failed to load image Python extension: libc10_cuda.so: cannot open shared object file: No such file or directory warn(f"Failed to load image Python extension: {e}")
The dynamic library obtained by torch._export.aot_compile will cause cuda illegal memory access problem when running in parallel with multiple threads.
kiwik
commented
1 week ago
Issue Metrics
torch.compileerror onscaled_dot_product_attentionintransformers.LlamaForCausalLMwhen providingattention_mask:RuntimeError: (*bias): last dimension must be contiguousmagma-cudac10::ArrayRef::ArrayRef(const Container&)SFINAE bugtorch-2.6.0.dev20241029+cu121torch.fft.rffttorch.packagewarning --TypedStorageis deprecatedtransformsargument intorchvision.transforms.v2.Composetorch._numpy.ndarray.astype()does not accept Numpy Dtypes correctlyBatchNorm2d+torch.compilewithreduce-overhead+ DDPKeyError: 'cls'pkg_resourcesmodule is deprecated by setuptoolstorch.compilehas inconsistent numerical precision with eager modelistobject is not callablehostmaster+pytorch_adjust_num_blocks_and_indicesgives wrong adjusted block masktorch.compileerrors when inputs memory overlap.tensornot aFakeTensorunderFakeTensorModeanddevice('meta')dynamo_expected_failuresis silent on certain tests that ended up passingtorch.distributed.tensor.distribute_module.attn_maskinjagged_scaled_dot_product_attentiontest_constant_folding_abi_compatible_cpusurfacesCUDA error: invalid argumenton H100DataLoaderprobably shouldn't useforkby defaultpytorch/conda-builderDocker imagesdp_paramsand benchmark resultstorch.distributed._state_dict_utils._broadcast_tensorsdoes not properly support CPU tensors.run_decompositionsfails with pytree error onLlama-3.2-visionaten.istftaten::_make_dual.test_sp24_compileappears broken on sm80/sm90maybe_mark_dynamiccauses max recursion error when used with compile during tensordict consolidationlinux.aws.a100oninductor-perf-compare.ymllinux-binary-manywheel / manywheel-py3_10-xpu-build / buildis currently brokents_convert_method_to_trt_enginefunctiontorch.std_meanreturnsNaNas mean of aninfarray.*i8torch.special.zetaignoresnaninput whenother=-inf.torch.__config__.show()silently initialises CUDA (?); forked processes fail with uninitialised CUDAtruncorfloor_amp_foreach_non_finite_check_and_unscale_can be torch.compiled inside torch.amp, but not in identical code outside itdtypepromotion ofout=functions on meta inputs not consistent.out=meta device support.torch.nn.functional.softshrinkreturns 0 onNaNinput.aten::_trilinearnanmeantorch.packagebugcpuinfoerror on import(*bias): last dimension must be contiguouswhen running compiled SDPA on length 1 tensors