search

P1sec / pycrate

A Python library to ease the development of encoders and decoders for various protocols and file formats; contains ASN.1 and CSN.1 compilers.
GNU Lesser General Public License v2.1
381 stars 132 forks source link

Publish on PyPI? #135

Closed benmaddison closed 3 years ago

benmaddison commented 3 years ago

Thank you for this codebase! This is the only OSS ASN.1 compiler that is feature-complete enough for what I needed.

I have written a library for prototyping RPKI signed objects (https://github.com/benmaddison/rpkimancer) that depends on the compiler and runtime.

I'd like to be able to publish on PyPI, but at the moment I have "direct" (git+https) dependency on pycrate, which is not allowed by PyPI.

Is there a reason that pycrate is not on PyPI? If you're open to it, I am happy to do a PR to add the CI/CD bits to automate the process.

p1-bmu commented 3 years ago

Hi, thank you for your message.

I never had the need for putting pycrate on pypi ; and never took the time to check how to do it. If you want to contribute some automation script that enables the project to be published on pypi, this would be highly appreciated !

benmaddison commented 3 years ago

@p1-bmu thanks for the quick response! I'll try and get a PR up in the next couple of days. A couple of questions, so that I know what kind of workflow to write:

p1-bmu commented 3 years ago

Supported Python versions are 2.7 and 3.4++. It would be excellent if all those are tested against automatically.

Regarding github or tagged releases, I do not use them a lot, the last one is 2 years old (I have a plan to tag a 0.5 one in the coming month however...). I don't even know the difference between those 2 kind of releases, I used git tag to do releases in the past, what seems considered as standard project release by github.

Sorry, I am not used to this kind of workflow and packaging routine. Feel free to keep asking in case you have any question, I will do my best to learn those aspects on my side too.

benmaddison commented 3 years ago

Supported Python versions are 2.7 and 3.4++. It would be excellent if all those are tested against automatically.

Sure, that's no problem, I'll be adding a testing step to the workflow.

Regarding github or tagged releases, I do not use them a lot, the last one is 2 years old (I have a plan to tag a 0.5 one in the coming month however...). I don't even know the difference between those 2 kind of releases, I used git tag to do releases in the past, what seems considered as standard project release by github.

A "release" is basically just a git tag with some GitHub-specific metadata attached. You can't create a GH release without a corresponding tag, but you can tag a commit without it being a GH release.

I'll make the workflow tag triggered, as that is easier to construct in relation to the test and build steps.

Sorry, I am not used to this kind of workflow and packaging routine. Feel free to keep asking in case you have any question, I will do my best to learn those aspects on my side too.

No problem at all, I will! Hopefully it will make more sense once you have a PR to review.

In the meantime, it would be useful if you could go and register an account on pypi.org and test.pypi.org, as you'll want to be in control of the published packages once they are there.

You'll also see I have copied you on an email to another pypi user that has a conflicting package using the pycrate name. Hopefully that won't be difficult to resolve.

benmaddison commented 3 years ago

Supported Python versions are 2.7 and 3.4++. It would be excellent if all those are tested against automatically.

Sure, that's no problem, I'll be adding a testing step to the workflow.

There are few issues with the test suite at the moment.

These should probably be dealt with in a separate issue, but recording them here for reference:

Happy to help with these items when the time comes.

p1-bmu commented 3 years ago

Thank you for your effort, Ben. Much appreciated !

OK for py34: old enough, do not bother with it if not supported as is by github actions. Regarding py27: I have no issue with running the test suite ; do you test the last tagged release (0.4) or the current master ? Maybe I should tag a 0.5 in the following days, so we can work with it instead of 0.4 for submitting to pypi. For the _pref test, no need to use them in integration, they are basically the same as the unittest in loops.

I have an account on pypi (and pypi test): https://pypi.org/user/p1-bmu/ So I should be able to upload the package when ready.

benmaddison commented 3 years ago

OK for py34: old enough, do not bother with it if not supported as is by github actions.

Ack

Regarding py27: I have no issue with running the test suite ; do you test the last tagged release (0.4) or the current master ?

OK, so tests under 2.7 pass on 0.4 and fail on master. I have bisected, and the commit that introduced the failure is: 7eff861370b3a0b06eda9b52e0bc37696e72023b

I can't see the issue just by inspecting the diff. Perhaps you have some insight?

I'll mark the 2.7 as being allowed to fail for now, and we can remove when that is fixed.

Maybe I should tag a 0.5 in the following days, so we can work with it instead of 0.4 for submitting to pypi.

I suggest waiting for the workflow to be merged: that way the publication will happen automatically when you push the new tag.

I would also suggest switching to semver-style version numbers. So the next release would be 0.5.0.

For the _pref test, no need to use them in integration, they are basically the same as the unittest in loops.

Noted

I have an account on pypi (and pypi test): https://pypi.org/user/p1-bmu/ So I should be able to upload the package when ready.

OK, great. You will need to have an API token created on pypi and pypi-test, and added to the github repo as a "secret". They should be named PYPI_TOKEN and TEST_PYPI_TOKEN respectively.

p1-bmu commented 3 years ago

This is strange that you get an failure with the unittests in Python2.7. The commit you identified was made recently, and nothing broke on my side...

Here are my results:

$ python2 --version && python2 -m unittest test.test_pycrate
Python 2.7.18
/usr/local/lib/python2.7/dist-packages/CryptoMobile/AES.py:61: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
  from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
[<>] testing pycrate_asn1c
[proc] module HardcoreSyntax (oid: []): 116 ASN.1 assignments found
--- compilation cycle ---
--- compilation cycle ---
--- compilation cycle ---
--- verifications ---
[proc] ASN.1 modules processed: ['HardcoreSyntax']
[proc] ASN.1 objects compiled: 75 types, 3 sets, 37 values
[proc] done
[<>] all ASN.1 modules generated to ./test_asn_todelete/
[<>] loading all compiled module
[<>] all ASN.1 modules loaded successfully from ./test_asn_delete/
.[<>] testing pycrate_asn1rt
.[<>] testing pycrate_core
.[<>] testing pycrate_csn1
.[<>] testing pycrate_ether
.[<>] testing GSM RR in pycrate_mobile
.[<>] testing pycrate_media
.[<>] testing pycrate_mobile
.
----------------------------------------------------------------------
Ran 8 tests in 7.163s

OK

Otherwise, OK for everything. Thanks.

Yevgnen commented 3 years ago

@p1-bmu @benmaddison

Thank you for your help: much appreciated!

Yes, please add p1-bmu as owner. Then he can generate an API token for me to complete the packaging workflow.

Please let us know when you have had a chance to do it.

Cheers,

Ben

Hi, I'm glad to help with the publish of pycrate to PyPi. As a beginning step, I've invited pi-bmu on PyPi to be the owner of project pycrate. Please check.

benmaddison commented 3 years ago

@p1-bmu we should be ready to go:

I will shortly push the last of my changes and mark #137 as ready for review and merging.

p1-bmu commented 3 years ago

Hi ! I created the 2 API tokens in pycrate for pypy and test-pypi. Regarding the issue with Cryptomobile, the library is not required for standard use of pycrate. I will check on my side regarding this. More generally, no specific dependencies are required to run most of the part of pycrate. Thanks for your help