search

kimgr / asn1ate

A Python library for translating ASN.1 into other forms.
Other
69 stars 41 forks source link

asn1ate support for IMPORTS and EXPORTS clauses #47

Closed vanrein closed 6 years ago

vanrein commented 8 years ago

Hello Kim,

Here are the changes that I am proposing for asn1ate as a library, so without my own asn2quickder.py included. I took the liberty of mentioning Quick DER in the README, which I assume you find useful too? I intend to include the asn2quickder script with Quick DER, and make that depend on asn1ate as a library.

After acceptance, I intend to dissolve my vanrein/asn2quickder project on GitHUB, and make its repository point to (1) your asn1ate work and (2) Quick DER.

In the end, I agree with you that this gives a clean cut between the projects, so thanks for making that decision.

Cheers, -Rick

vanrein commented 8 years ago

Your testdata/ already contain IMPORTS and EXPORTS statements, and they turned out to be useful tests. In fact, they used a form I had ignored (adding an OID after a ModuleRef) and which is now supported. As a result, basic_test.sh now reports precisely the same errors as it did before this PR was incorporated. (Namely, unknown names being referenced.)

vanrein commented 8 years ago

Hey,

The changes as posted are not valid Python, however, all tests fail with TabError: inconsistent use of tabs and spaces in indentation.

Must have happened during the merge, after my tests, sorry about that.

I think I'll just pick up your changes and apply them manually.

Thank you.

I'd love it if you could provide me with some isolated test cases, see the testdata directory for examples. Unfortunately some of those tests fail due to multiple modules, I intend to clean that up at some point.

You mean tests outside of Quick DER? Because I test it thoroughly, but indeed in a separate package. Let me see if I can isolate something out of that for you. I'll put it on my TODO list. (Please accept the PR nonetheless, I'd like to get that one off my list...?)

-Rick

kimgr commented 8 years ago

Sorry, my time is spreading thin. I spent an evening last week on this PR and ran into several quirks with the grammar.

I need small, representative test cases to make sure I'm parsing and generating a sema model for good data, and to get regression coverage for it. For now I'm making examples up as I read the ASN.1 spec, but I don't know enough about the construct to keep them real.

vanrein commented 8 years ago

Hey,

I now figured out what you do to test (basic_test.sh) and it looks like I'm breaking your already-present imports test -- perhaps because there is more to check now. I'll try to figure out what's going on, this must be fixed of course. I suspect there's more data available now, but not fully integrated into asn1ate's generator; I hope that I'm able to solve that riddle...

I updated the Pull Request; it now supports the test ASN.1 specs that you had prepared already in testdata/* (including the forms that go beyond what I've needed to port RFCs to headers).

-Rick

vanrein commented 8 years ago

Hmm,

I now figured out what you do to test (basic_test.sh) and it looks like I'm breaking your already-present imports test -- perhaps because there is more to check now. I'll try to figure out what's going on, this must be fixed of course. I suspect there's more data available now, but not fully integrated into asn1ate's generator; I hope that I'm able to solve that riddle...

-Rick

vanrein commented 8 years ago

Hi, you need small and representative... a bit of a contradiction ;-) but here is some ASN.1 that I distilled from an RFC:

https://github.com/vanrein/quick-der/blob/master/rfc/rfc2315.asn1

vanrein commented 7 years ago

Hey Kim,

Is there any news on this? We may be unknowingly waiting for each other?

I would love to clear out my fork in favour of your original (and move asn2quickder into the Quick DER package) but then I do need the sema.py patches for Imports and Exports checked in.

Thanks, -Rick

kimgr commented 7 years ago

Sorry, I've been swamped. Will try to come back to this, but I can't promise when.

vanrein commented 7 years ago

Thanks.

Powersource commented 7 years ago

cough

kimgr commented 7 years ago

Bless you. I'm hoping to look into this tonight (fingers crossed!)

vanrein commented 7 years ago

Hi kimgr,

You clearly have the intention to integrate this proposal, but it's been hanging for a year now...

This puts rather a strain on our packaging and distribution of asn2quickder and Quick DER, base package on which we build much of our open source software stack.

I would rather not fork, but if you think that is more practical than we would be prepared to take over the project; just let me know what you think the best approach would be.

Really all we're asking for is a repository for asn1ate that has the IMPORT / EXPORT patches installed, which we depend on. My ideal would be to see those integrated so we don't have to fork. We currently have an asn1ate project, and I'm itching to close it and fully rely on this one.

Let me know what your thoughts are, please!

kimgr commented 7 years ago

Hi vanrein,

I did some work on asn1ate this summer to get IMPORT and EXPORT working, but I ran into all sorts of subtleties with symbol resolution that drove me to update the parser. I think I ran out of time somewhere in the middle.

Oh yeah, I remember now -- the problem was to find a 1-1 mapping between IMPORTS and target language module imports (e.g. import in Python or #include in C). For Python I ran into several problems with asn1ate's test infrastructure, etc.

It could be that there's a small change on top of asn1ate master that carries IMPORTs through to sema, so you can build code generation on top of it.

Let me take another look right away, and if I can't figure it out in short order we can reevaluate.

kimgr commented 7 years ago

OK, so it turns out I had basic support for IMPORTS on master of my development box. I just pushed it to asn1ate/master now. Let me know if that works out for you.

vanrein commented 7 years ago

Hi Kim,

Thanks! I've forwarded it to Gijs, who is doing our packaging. This is =so= preferred over the alternative!

-Rick

gijzelaerr commented 7 years ago

Hi Kim,

did you forget to push? Last commit is from August 4.

kimgr commented 7 years ago

Gijs,

No, that's when the work happened. It just sat on my machine for three months :-|.

As far as I can see kimgr/asn1ate/master has IMPORTS support.

gijzelaerr commented 6 years ago

hi @kimgr this seems to work for us. Do you plan to do a 0.5.1 release soon?

kimgr commented 6 years ago

I was hoping to get some other stuff into the next release, but I'm not realistically going to be able to make that happen. I'll try to put something together this weekend.

kimgr commented 6 years ago

0.6.0 was just uploaded to PyPI: https://pypi.python.org/pypi/asn1ate/0.6.0

vanrein commented 6 years ago

Thanks Kim, that's a great enabler!

kimgr commented 6 years ago

Yeah, sorry it took so long. I'm closing this PR now, please follow up with separate issues/PRs if any problems come up with 0.6. Cheers.