sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.48k stars 488 forks source link

Incorrect answers given for ranks of large matrices. #35885

Open elarson314 opened 1 year ago

elarson314 commented 1 year ago

Is there an existing issue for this?

Did you read the documentation and troubleshoot guide?

Environment

- **OS**: RedHat EL 7.7
- **Sage Version**: 9.5

Steps To Reproduce

On a machine with ample RAM, compute the rank of a matrix with more than 2^32 entries. For example, a Vandermonde matrix like so:

def vandermonde_rank(x, y):
   F = GF(8388593)
   M = matrix(F, x, y)

   for i in range(x):
      row = []
      i = F(i)
      ij = F(1)
      for j in range(y):
         row.append(ij)
         ij *= i
      M[i] = row

   M.echelonize()
   return M.rank()

print(vandermonde_rank(65537, 65537))

Expected Behavior

The rank should be computed correctly (or, failing that, an error should be raised saying that the matrix is too large).

Actual Behavior

The rank is computed incorrectly once there are more than 2^32 entries. For example, the above code prints 65536 for the rank of the 65537 x 65537 Vandermonde matrix. The user is not warned that the answer might not be correct.

Additional Information

No response

jhpalmieri commented 1 year ago

Here's something:

sage: M = identity_matrix(GF(8388593), 10)
sage: type(M.rank())
<class 'sage.rings.integer.Integer'>
sage: M.echelonize()
sage: type(M.rank())
<class 'int'>

The echelonize method caches the rank as part of its computations, but it caches a Python int rather than a Sage Integer.