search

rlogiacco / Natural

Natural is a collection of Eclipse plugins to enable rich smart editing of acceptance tests using natural language definition files. It currently supports Cucumber and JBehave syntax.
Eclipse Public License 1.0
75 stars 37 forks source link

natural-next #73

Closed drkstr101 closed 4 years ago

drkstr101 commented 4 years ago

This branch is still under development. The WIP flag will be removed once it is ready for review.

drkstr101 commented 4 years ago

Here are the included updates so far

  1. Feature.scenarios now resolve to AbstractScenario rather than EObject
  2. Added AST support for additional tag placement on Background and Examples
  3. Added AST support for multiple example tables
  4. Upgraded to hamcrest-2.2 and added some additional test cases
drkstr101 commented 4 years ago
  1. Improved editor validation
    • Issue codes have been abstracted to better support localization
    • the step matching validation has been flagged as "Expensive". This means the step will only run when the user explicitly runs the full validation from the context menu. While degrading the experience slightly for some, it allows the user to continue editing while the validator attempts to locate matches. This is just a stopgap for improved stepmatching [pending further discussion].
    • Rather than displaying a cryptic parser error, new issue codes have been added for features that have no scenarios, and scenarios that have no steps.
    • Added new test scenario for validation
drkstr101 commented 4 years ago
  1. org.agileware.natural.common has been repurposed. The common bundle was renamed to stepmatcher. A new bundle named common was created in its place and given a major version bump (1.x -> 2.x). Both cucumber and jbehave depends on common, while only cucumber depends on stepmatcher.
drkstr101 commented 4 years ago

There are no longer any planned feature updates for this branch, and so WIP flag has been removed. However, there is still a full round of testing that needs to be done to ensure the updates do not have any unintended consequences. It may be a good idea if we can distribute a release candidate to some users outside of Eclipse Marketplace, as there are many diverse setups and use cases I'm entirely unaware of.

drkstr101 commented 4 years ago
  1. Added support for dubugging the lexer/parser The Xtext generator will now produce a debug .g file, for example src-gen/org/agileware/natural/cucumber/parser/antlr/internal/DebugInternalCucumber.g. This file may be loaded into ANTLRWorks for debugging. Just launch the executable jar antlrworks-1.5.1.jar located in the common bundle. Note utility jars located in commons are not on the classpath, however other bundles may put them on their own classpath by referencing the bundle path in the build.properties. See `.tests' bundles for example.
drkstr101 commented 4 years ago

Updated for conflicts.

drkstr101 commented 4 years ago

@rlogiacco Congrats on the new release!

All test cases in next are now passing, and I believe the coverage is sufficient enough to begin manual testing of a release package today. I will do the first round of this myself so as not to waste anybody's time.

There are still a couple test failures in Jbehave, or actually one real issue that bubbles up to 5 test failures overall. This is caused by specific variations of the GivenScenarios feature, which is in the spec, but I actually never found any usage of in my example stories. I am confident enough these issues in the ANTLR grammar are localized to the feature, and will not affect the overall structure of parsed AST.

rlogiacco commented 4 years ago

Should we list / reference the issues that should get fixed with this PR so that they get automatically close upon merge?

drkstr101 commented 4 years ago

Great idea!

I will start going through them and collecting them here.

rlogiacco commented 4 years ago

Are we ready to merge this and start wrapping up a new release?

rlogiacco commented 4 years ago

Are we aware each and every issue referencing this PR will get closed once we merge?

drkstr101 commented 4 years ago

Nope! I was not aware!

This was meant to be a preliminary list for my reference as I build out more tests. I didn't realize these would all be closed automatically. I need to go through and pull some of these out.

edit

All referenced issues are those which may have been affected by changes in next, which turned out to be pretty much all of them. I wanted to go through each one and see exactly what might have changed the behavior, and how/why.

drkstr101 commented 4 years ago

Are we ready to merge this and start wrapping up a new release?

I had accidentally included some unintended updates to the formatting2 when I merged from another branch which was causing test failures, but it should be good now. I just wanted to give it a quick run a few times to see if anything seems obviously broken.

Also, Cucumber lang is now fully compliant ANTLR grammar (EG. no lexer/parser warnings during MWE2 workflow). This may have unintended affects in the UI however, so I will need to spend a little time smoke-screening it.

drkstr101 commented 4 years ago

Seems ok at first glance: https://imgur.com/a/dsewf1I

But it seems formatting is still broken, even with the original formatter API. I may have done something to cause this however. I will investigate further.

drkstr101 commented 4 years ago

We seem to be facing an up-hill-battle trying to get original CucumberJavaFormatter working in latest Xtext, so rather than spinning cycles that will eventually be abandoned, they were spent on removing the last few blockers on the formatting2 api, which means it is now at least as capable as using the original formatter (in next), with a lot of room for custom fine tuning as needed/desired.

This implementation did however come with a slight trade off, in that the Xtext abstraction is now leaking into the AST. Or in other words, a slightly less ideal AST moel for the sole purpose of being easier to format.

An ideal implementation would likely come in the form of a core language model that had proper free-form multi-line text formatting support. This is unfortunately a project in and of itself.

Context: https://www.eclipse.org/forums/index.php?t=msg&th=1103866&goto=1828087&#msg_1828087

drkstr101 commented 4 years ago

Next up, figure out WTH happened to stepmatching!

~I tested this awhile ago and was working fine, so I havent tested it since, thinking nothing I did would have changed anything~... but as I type this my brain just debugged the problem in the background... Step model was slightly changes. Stepmatcher must also be changed to follow suit. Should be fairly minor update.

drkstr101 commented 4 years ago

@rlogiacco next has been reset and beta has been orphaned. I now have an alpha branch that contains the previous next, in case you would like me to open this PR back up. My intent however is to attempt to manually merge pieces from beta -> next, resubmitting the PR once I have a stable set of features in next.