Pickle not working for large prime field...

[patniemeyer]

Initializing a large prime field for e.g. BLS12-381 takes several minutes. I was hoping to mitigate this by pickling the GF object to disk after first run, however this does not seem to work. Should it?

# BLS12-381 characteristic (prime)
p = 0x73EDA753299D7D483339D80809A1D80553BDA402FFFE5BFEFFFFFFFF00000001
GF = galois.GF(p, 1, verify=False)
with open('out.pkl', 'wb') as f:
    pickle.dump(GF, f)

and then trying to load it back:

with open('out.pkl', 'rb') as f:
    GF = pickle.load(f)

I get:

    GF = pickle.load(f)
         ^^^^^^^^^^^^^^
AttributeError: Can't get attribute 'FieldArray_52435875175126190479447740508185965837690552500527637822603658699938581184513_7' on <module 'galois._fields._factory' from '/Users/pat/.../venv/lib/python3.11/site-packages/galois/_fields/_factory.py'>

[mhostetter]

Thanks for the report. While we should look into this, here's a quick workaround.

Since your field is very large, the ufunc_mode will be "python-calculate", meaning no LUTs/etc will be created. So the only other difficult thing to find is the primitive root (the multiplicative generator of the field). That is what is taking several minutes. What you can do is manually run that once, and save/hardcode the result. Creating the Galois field, having provided the primitive element, is very fast. (You also need verify=False to be speedy. The verification step ensures the provided primitive root can generate the entire multiplicative group. In our case, we already know it does, so we can skip the costly verification.)

Here's an example.

In [1]: import galois

In [2]: p = 0x73EDA753299D7D483339D80809A1D80553BDA402FFFE5BFEFFFFFFFF00000001

In [3]: %time alpha = galois.primitive_root(p); alpha
CPU times: user 47.4 s, sys: 0 ns, total: 47.4 s
Wall time: 47.4 s
Out[3]: 7

In [4]: %time GF = galois.GF(p, 1, primitive_element=alpha, verify=False); print(GF.properties)
Galois Field:
  name: GF(52435875175126190479447740508185965837690552500527637822603658699938581184513)
  characteristic: 52435875175126190479447740508185965837690552500527637822603658699938581184513
  degree: 1
  order: 52435875175126190479447740508185965837690552500527637822603658699938581184513
  irreducible_poly: x + 52435875175126190479447740508185965837690552500527637822603658699938581184506
  is_primitive_poly: True
  primitive_element: 7
CPU times: user 788 µs, sys: 0 ns, total: 788 µs
Wall time: 795 µs

[mhostetter]

I can reproduce the bug.

Pickling/unpickling works inside the same Python session.

In [7]: with open('out.pkl', 'wb') as f:
   ...:     pickle.dump(GF, f)
   ...: 

In [8]: with open('out.pkl', 'rb') as f:
   ...:     GFF = pickle.load(f)
   ...: 

In [9]: GFF
Out[9]: <class 'galois.GF(52435875175126190479447740508185965837690552500527637822603658699938581184513)'>

But not in a new session (which is the point).

In [1]: import pickle

In [2]: import galois

In [3]: with open('out.pkl', 'rb') as f:
   ...:     GFF = pickle.load(f)
   ...: 
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-3-016b04d6b45d> in <module>
      1 with open('out.pkl', 'rb') as f:
----> 2     GFF = pickle.load(f)
      3 

AttributeError: Can't get attribute 'FieldArray_52435875175126190479447740508185965837690552500527637822603658699938581184513_7' on <module 'galois._fields._factory' from '/home/matt/.local/lib/python3.10/site-packages/galois/_fields/_factory.py'>

I tried to solve this before in #393, but clearly missed something. I believe I need to add some logic in __reduce__() and __setstate__() for this to work properly.

[patniemeyer]

Thanks for the report. While we should look into this, here's a quick workaround.

Thank you very much for the workaround (much better solution to the original problem really)! I should have investigated more closely where the time was going.

[nachocodexx]

I got the same issue trying to deserialize and object that contains and galois.GF attribute, but, I follow the instructions of @mhostetter and it worked well.

class IDAx(ActiveX):
    def __init__(self,k:int,n:int,p:int=2,verify:bool= False):

        # Define the Galois field
        alpha = galois.primitive_root(p)
        self.GF = galois.GF(p,1,primitive_element=alpha, verify=verify)

I deserialized the object using ```cloudpickle````