search

AequilibraE / aequilibrae

aequilibrae - Python package for transportation modeling
https://www.aequilibrae.com
Other
166 stars 38 forks source link

Assignment large number of matrices at the same time consumes too much memory #544

Open pedrocamargo opened 3 months ago

pedrocamargo commented 3 months ago

When trying to assign many large matrices at the same time (225 x 11,700, 11,700) requires an excessive amount of memory (~230Gb).

There is no reason why we would not allow users to provide matrices in 32-bit and convert them only when feeding into the compiled code.

Art-Ev commented 2 months ago

@pedrocamargo, @Jake-Moss : coud this "problem" result in this kind of message ? (graph computation of only one 955 x 955 matrix on a 64 760 links network) image

Art-Ev commented 2 months ago

Same result with a 21x21 matrix on the same network :'( will investigate with another method

pedrocamargo commented 2 months ago

The problem here is a different one, @Art-Ev . I would guess that your problem here is that your link IDs are way too large. Are they as large as 2090000960?