Wrong scalar type used in complex tests

[asmeurer]

I'm getting failures with torch in the asarray tests like

        else:
            assert kw["dtype"] == _dtype  # sanity check
            ph.assert_kw_dtype("asarray", kw_dtype=_dtype, out_dtype=out.dtype)
        ph.assert_shape("asarray", out_shape=out.shape, expected=shape)
        for idx, v_expect in zip(sh.ndindex(out.shape), _obj):
>           v = scalar_type(out[idx])
E           RuntimeError: value cannot be converted to type double without overflow
E           Falsifying example: test_asarray_scalars(
E               shape=(2, 3),
E               data=data(...),
E           )
E           Draw 1 (kw): {'dtype': torch.complex64}
E           Draw 2 (_obj): [complex(0.0, 0.0),
E            complex(0.0, 0.0),
E            complex(0.0, 0.0),
E            complex(0.0, 0.0),
E            complex(0.0, 0.0),
E            complex(0.0, 1.0)]
E           obj=[[0j, 0j, 0j], [0j, 0j, 1j]]

array_api_tests/test_creation_functions.py:253: RuntimeError

        if copy is not None:
            stype = dh.get_scalar_type(x.dtype)
            idx = data.draw(xps.indices(x.shape, max_dims=0), label="mutating idx")
>           old_value = stype(x[idx])
E           RuntimeError: value cannot be converted to type double without overflow
E           Falsifying example: test_asarray_arrays(
E               shape=(),
E               dtypes=OnewayPromotableDtypes(input_dtype=torch.complex64, result_dtype=torch.complex64),
E               data=data(...),
E           )
E           Draw 1 (x): tensor(0.+1.j)
E           Draw 2 (kw): {'copy': False}
E           Draw 3 (mutating idx): ()

array_api_tests/test_creation_functions.py:292: RuntimeError

From what I can tell, it is setting the scalar type as float with a complex array, so it is calling something like

>>> float(torch.asarray(1j))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: value cannot be converted to type double without overflow

The tests are surprisingly flaky (e.g., when I add a print statement they often start passing), and they also don't really seem to shrink properly, which suggests to me that there's some issue in the strategies themselves, or maybe even in hypothesis itself. I haven't investigated further yet.

[asmeurer]

Actually this failure is happening all over the place, not just in the asarray tests

FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_abs[abs] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_abs[__abs__] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_acos - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_acosh - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_add[add(x1, x2)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_add[__add__(x, s)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_add[__iadd__(x1, x2)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_asin - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_asinh - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_atan - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_atanh - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_conj - AttributeError: module 'array_api_tests._array_module' has no attribute 'conj'
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_cos - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_cosh - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_divide[divide(x1, x2)] - AssertionError: out.dtype=complex64, but should be complex128 [divide(complex128, complex64)]
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_equal[equal(x1, x2)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_equal[__eq__(x, s)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_exp - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_expm1 - RuntimeError: "expm1_vml_cpu" not implemented for 'ComplexFloat'
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_imag - AttributeError: module 'array_api_tests._array_module' has no attribute 'imag'
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_isfinite - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_isinf - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_isnan - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_log - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_log2 - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_log10 - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_multiply[multiply(x1, x2)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_multiply[__mul__(x, s)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_multiply[__imul__(x1, x2)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_multiply[__imul__(x, s)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_negative[negative] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_negative[__neg__] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_not_equal[not_equal(x1, x2)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_not_equal[__ne__(x, s)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_pow[pow(x1, x2)] - AssertionError: out.dtype=complex64, but should be complex128 [pow(complex128, complex64)]
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_real - AttributeError: module 'array_api_tests._array_module' has no attribute 'real'
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_round - RuntimeError: "round_vml_cpu" not implemented for 'ComplexFloat'
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_sign - RuntimeError: Unlike NumPy, torch.sign is not intended to support complex numbers. Please use torch.sgn instead.
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_sin - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_sinh - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_square - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_sqrt - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_subtract[subtract(x1, x2)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_subtract[__sub__(x, s)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_subtract[__isub__(x1, x2)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_subtract[__isub__(x, s)] - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_tan - RuntimeError: value cannot be converted to type double without overflow
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_tanh - RuntimeError: value cannot be converted to type double without overflow

[honno]

Sorry I can't seem to reproduce this, up-to-date on both this repo and array-api-compat, having tried both torch=2.0.0 and torch=1.13.1 with the following commands

$ ARRAY_API_TESTS_MODULE=torch ARRAY_API_TESTS_API_VERSION=2021.12 pytest array_api_tests/test_creation_functions.py::test_asarray_arrays
$ ARRAY_API_TESTS_MODULE=torch ARRAY_API_TESTS_API_VERSION=2022.12 pytest array_api_tests/test_creation_functions.py::test_asarray_arrays
$ ARRAY_API_TESTS_MODULE=array_api_compat.torch ARRAY_API_TESTS_API_VERSION=2021.12 pytest array_api_tests/test_creation_functions.py::test_asarray_arrays
$ ARRAY_API_TESTS_MODULE=array_api_compat.torch ARRAY_API_TESTS_API_VERSION=2022.12 pytest array_api_tests/test_creation_functions.py::test_asarray_arrays

[asmeurer]

Is there a way to make hypothesis spit out one of those reproduce decorators for the failure?

I'm using torch 2.0.0 and hypothesis 6.74.0.

[honno]

Is there a way to make hypothesis spit out one of those reproduce decorators for the failure?

~It's not great but there's seed. I think don't mind that, I just need to explore this more thoroughly.~ Managed to reproduce.