[feature] accessing the "order" argument of mpz_import() / mpz_export() in to_binary() / from_binary()

[mitshell]

Hello,

everything is in the title: from the source of gmpy2_binary.c, it looks impossible to access the "order" argument of mpz_import / export from Python. Is there any special tip which would allow to change this "order" parameter from Python ? If no, would it be possible to submit a patch to gmpy2_binary in order to make it accessible through the calling of to_binary() / from_binary(), e.g. as an argument with a default value of -1 ?

Thanks for your help, Benoit

[casevh]

Hi,

Just to clarify, in addition to calling mpz_import()/mpz_export(), to_binary()/from_binary() also adds a byte that identifies the sign and the result's original type: mpz, mpq, mpfr, etc.? Are you looking for direct access to mpz_import()/mpz_export() without the leading identifier/sign byte, or do you just want to change the layout of to_binary()/from_binary()?

Case

[mitshell]

Hello, ideally, having mpz_import() / mpz_export() available exposed within Python would be the best for me. On the other side, I realized that when I import a buffer to an mpz, the most significant bits that are null are removed from the internal representation ; for this reason, I am not sure gmpy2 will fit for my current project. Tell me if mpz_import / export can easily be accessed from Python, if it is not the case, I will just close the issue. Thanks for your quick feedback. Benoit

2016-02-05 15:04 GMT+01:00 casevh notifications@github.com:

Hi,

Just to clarify, in addition to calling mpz_import()/mpz_export(), to_binary()/from_binary() also adds a byte that identifies the sign and the result's original type: mpz, mpq, mpfr, etc.? Are you looking for direct access to mpz_import()/mpz_export() without the leading identifier/sign byte, or do you just want to change the layout of to_binary()/from_binary()?

Case

— Reply to this email directly or view it on GitHub https://github.com/aleaxit/gmpy/issues/104#issuecomment-180372360.

[casevh]

Hi,

I can add mpz_import()/mpz_export() to the current development version. Can you provide some sample Python code showing how you would use it? An example of a buffer filled with some data and passed to mpz_import() would be perfect.

Regarding the removal of the most significant zero bits, that is just the behavior of the underlying mpz_t type. It will expand and contract as necessary and does not have a fixed length. If you are trying to mimic a fixed-width integer type, you will need to do that in Python or create a new type in gmpy2.

Case

[mitshell]

Dear Case,

I had something in mind, in order to shift large buffers: today, I do this the ugly way, see https://github.com/mitshell/libmich/blob/master/libmich/core/shtr.py#L157. This helps me to process bitstream, like it is required with ASN.1 PER: fields are encoded / decoded bit by bit, in the minimum number of bits. I understand that the handling of most significant zero bits within mpz is limiting here, so I guess it won't easily help me for this case. I have another corner case, still with ASN.1, where it is possible to encode / decode unbounded integers (e.g. over 64kb, ...). I am not aware of any real-life protocol that would make use of such value, but the standard and encoding / decoding rules allow it ! Python starts to show its limit with such big integers, so using gmpy2 would help here. However, most significant bytes are leftmost with ASN.1, this would be a reason to get access at least to the "order" parameter, or even better to the raw mpz_import() / export() functions.

Best Regards Benoit

2016-02-07 20:02 GMT+01:00 casevh notifications@github.com:

Hi,

I can add mpz_import()/mpz_export() to the current development version. Can you provide some sample Python code showing how you would use it? An example of a buffer filled with some data and passed to mpz_import() would be perfect.

Regarding the removal of the most significant zero bits, that is just the behavior of the underlying mpz_t type. It will expand and contract as necessary and does not have a fixed length. If you are trying to mimic a fixed-width integer type, you will need to do that in Python or create a new type in gmpy2.

Case

— Reply to this email directly or view it on GitHub https://github.com/aleaxit/gmpy/issues/104#issuecomment-181083653.

[casevh]

Hi,

Interesting. My day job is a network engineer - primarily enterprise and data center LAN.

There are a couple of gmpy2 features that are a bit non-standard but they may help you with bit level manipulations. An mpz type can be indexed by the bit positions. For example:

>>> b=gmpy2.mpz(12345678)
>>> b.digits(2)
'101111000110000101001110'
>>> b[10:20].digits(2)
'1100011000'
>>> b[20:40].digits(2)
'1011'

If you try to index beyond the left-most bit, an infinite number of 0 bits is assumed.

There is also a mutable integer type called xmpz. With an xmpz, you can do in-place left/right shifts and modify bits - either individually or in slices.

I have written custom private functions for other libraries. Would an _xmpz_import() and _xmpz_export() functions be sufficient? (Or _mpz_import()/_mpz_export()?) I may change them to public functions in the future but keeping them as private functions for now will make it easier to change them if needed.

Regards, Case