speed / freeze?

[laurentbartholdi]

I would love to use MagmaCall to ship 100x100 matrices with polynomial coefficients to Magma, compute their nullspace, and pull them back. First step is to send polynomials to Magma, and I see very strange speed issues: most calls take the advertised 1ms, some take literally minutes. In julia 1.10 and the master MagmaCall, I get

julia> using MagmaCall

julia> R,q = magg.RationalFunctionField(:q,magf.Rationals());

julia> n = (q+1)^10

julia> for _=1:10000 @time magf.Coefficients(magf.Numerator(n)) end
  0.001652 seconds (327 allocations: 9.172 KiB)
  0.001623 seconds (327 allocations: 9.172 KiB)
  0.001602 seconds (327 allocations: 9.172 KiB)
# ... so far, so good; but time starts to creep up from 1ms to 3ms
 0.003863 seconds (317 allocations: 9.016 KiB)
  0.003900 seconds (317 allocations: 9.016 KiB)
  0.003888 seconds (317 allocations: 9.016 KiB)
  0.084517 seconds (24.46 k allocations: 2.293 MiB, 11.80% gc time)
143.700617 seconds (245.88 k allocations: 6.648 MiB)
# ... and then takes a huge time, before going back to the advertised 1ms, at which moment the whole cycle starts again
  0.001332 seconds (319 allocations: 8.984 KiB)
  0.001299 seconds (319 allocations: 8.984 KiB)
  0.001323 seconds (314 allocations: 8.906 KiB)
...

[cjdoris]

Oh wow I had no idea anyone was using this package! I haven't touched it for a few years.

My guess is that it's Julia's garbage collector causing the slow down.

If you call GC.gc() periodically through the loop, do you still get those large peaks? What about if you explicitly finalize the two Magma objects created in each iteration of your code?

[cjdoris]

Actually, the MagmaObject finalizer creates an async task. That's a lot of tasks. Maybe that's the slowdown.

Probably a better way would be for the finalizer to push the object into a channel and have a single global async task pulling stuff off the channel and dealing with it.

[laurentbartholdi]

Thanks a lot for the quick reply! I'm putting Tommy Hofmanand Ma Horn in the loop since they told me about MagmaCall.

I understand my usage of MagmaCall is off-label: I want to send a matrix to Magma and pull one back, ideally that should be only one call rather than one call per coefficient. Is there a way to serialize a matrix of polynomials through the magma object passing system, or otherwise send a matrix of coefficients, without converting them to string and back? Or, at least, does MagmaCall have a method of sending a string directly to Magma and get the reply? Something like a non-macro version of mag""?

Thanks again, four piece of code is already very useful!

On Fri, Mar 3, 2023, 19:54 Christopher Rowley @.***> wrote:

Actually, the MagmaObject finalizer creates an async task. That's a lot of tasks. Maybe that's the slowdown.

Probably a better way would be for the finalizer to push the object into a channel and have a single global async task pulling stuff off the channel and dealing with it.

— Reply to this email directly, view it on GitHub https://github.com/cjdoris/MagmaCall.jl/issues/1#issuecomment-1453966216, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARAQUF436DWWRQPXZMNXRTW2I46VANCNFSM6AAAAAAVOQJFUY . You are receiving this because you authored the thread.Message ID: @.***>

[cjdoris]

All the object-passing code is here: https://github.com/cjdoris/MagmaCall.jl/blob/master/src/motp.jl. It uses the "Magma Object Transfer Protocol" which is the format used by WriteObject and ReadObject in Magma. It's totally undocumented so I reverse-engineered it by inspecting the output.

Looking at the options for MAGOTP_TYPE_*, it looks like I got as far as implementing basic stuff like strings, integers, rationals, sequences and sets. This doesn't include matrices. However, if you convert your matrix of coefficients into sequences of sequences of integers (for example) then you should be able to transfer it in one go to Julia.

[cjdoris]

Alternatively, you could use string or magsprint to get a string representation of the object, then parse it yourself.