Parser test for GraknClient

[erikedin]

It might be nice to see how many of the features files can be parsed, to track the progress, and see where work needs to be done. Just creates a simple script for that maybe?

[tk3369]

Hi, I have created a script in this fork: https://github.com/Humans-of-Julia/ExecutableSpecifications.jl/tree/parseonly

And, I just ran it against all Grakn feature files from https://github.com/graknlabs/behaviour.git. Nothing passes yet 😅 They all failed parsing with the same exception. From my prelim check, it seems that just a blank line before Feature would cause the following error. Or maybe it's the comments...

$ julia --project=. cmd/parseonly.jl /Users/tomkwong/Development/Grakn/graknlabs/behaviour/
Files failed parsing:
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/concept/rule/rule.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/concept/thing/attribute.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/concept/thing/entity.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/concept/thing/relation.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/concept/type/attributetype.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/concept/type/entitytype.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/concept/type/relationtype.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/concept/type/thingtype.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/connection/database.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/connection/session.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/connection/transaction.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/explanation/language.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/explanation/reasoner.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/language/compute.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/language/define.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/language/delete.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/language/get.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/language/insert.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/language/match.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/language/rule-validation.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/language/undefine.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/reasoner/arithmetic-equality.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/reasoner/attribute-attachment.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/reasoner/concept-inequality.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/reasoner/negation.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/reasoner/recursion.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/reasoner/relation-inference.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/reasoner/resolution-test-framework.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/reasoner/rule-interaction.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/reasoner/schema-queries.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/reasoner/type-hierarchy.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/reasoner/value-predicate.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
/Users/tomkwong/Development/Grakn/graknlabs/behaviour/graql/reasoner/variable-roles.feature
 reason: unexpected_construct
 expected: feature
 actual: scenario
Parsing failed: 33
Total number of files: 33

[erikedin]

Thanks, that's excellent! I'm working on getting Backgrounds recognized right now, but as soon as that's done, I'll look at getting some of these working.

[tk3369]

Great! Is there any specific issues that I can help out? I probably have few more hours to burn today but will get busy tomorrow.

[erikedin]

I think we need to create issues for the things in the roadmap, so we can track them in the collaboration project.

I expect that I'll rework large parts of the parser, so I'm not sure that's a good place to work right now. There's a good chance I'll step all over any changes you make. I'll try to get it in a more stable state before implementing more features, so we can grab any issues we want in there, without stepping on each others toes too much.

I think I need realistic tests on the data tables, comments, and Rules. Could you create a few of those?

[tk3369]

Sure. Let me create a few more tracking issues.

[erikedin]

Will you create an issue for parsing comments? I think it's the comments failing all of the feature files. I'll get to work on those right away. Famous last words, but it shouldn't be a major thing to just ignore those lines.

[tk3369]

Yes, just did. See #45

[tk3369]

I have just logged several more issues. I think it should be fairly complete for now. Once those new functionalities are implemented, we can re-run the parsing test and see what else falls out.

@mkschulze - I'm unable to edit issue #44 directly. Can you add a new set of checkboxes at the top that contains references to the pending issues? Please also replace the existing checkboxes with just a list and you can leave the assessment text just for reference.

[erikedin]

I just pushed a couple of fixes to master, and merged your parseonly script. It's mostly symbolic progress, but at least two feature files are properly parsed now.

[erikedin]

Two additional files can now be parsed properly, concept/thing/attribute.feature and concept/thing/entity.feature. I added the lenient parse options to parseonly.jl, so we allow out-of-order steps.

I made some progress on not requiring empty lines between sections, although it's not complete yet, I think. Some additional minor things I had to fix was allowing an empty description for a Background section.

At a quick glance, it looks like the rest of the feature files use data tables, so maybe that should be our next target. I think I don't have to overhaul as much of the parser as I imagined, so feel free to make changes if you want.

[erikedin]

As of the latest commits on master, and a recent clone of the behaviour repo, we have 6 files that we can't yet parse, out of 34. That seems like significant progress.