search

sebastianbenz / Jnario

Executable specifications for Java
136 stars 38 forks source link

Make Jnario a pure Xbase language #160

Open borisbrodski opened 9 years ago

borisbrodski commented 9 years ago

Jnario as a set of pure Xbase languages

Motivation

Jnario currently depends directly on Xtend. Since Xtend provides no official public stable API, even each minor release of Xtend may break Jnario and this already happened in the past.

The suggestion is to make Jnario pure Xbase language dropping dependency on Xtend.

Pros:

Cons:

Progress:

sebastianbenz commented 9 years ago

This would definitely make sense. Unfortunately, I don't have the time to work on this - and most likely will have even less time in the future. If you would like to tackle this, I would be happy to support you.

borisbrodski commented 9 years ago

Great! Yes, I and @oehme (I hope) will look into it.

sebastianbenz commented 9 years ago

That would be fantastic!

oehme commented 9 years ago

I'd love to help.

In my opinion, this would only be feasible if we drop Xtend features like

Otherwise we would end up copy-pasting large amounts of code from Xtend, which no-one would really have time to maintain. Loosing features isn't nice, but I guess it's more important to have a stable Jnario. One of our customers already dropped it because of the upgrade problems.

What do you guys think?

sebastianbenz commented 9 years ago

That's great Stefan!

I personally have no problem to drop features, as the alternative is even worse - being always one step behind in the migration process (besides the required maintenance effort). Class definitions and anonymous classes are only recent additions anyway. However, anonymous classes are really helpful when writing tests, some kind of alternative would be nice at least.

Template expressions are no problem either, as I would say that you are doing something wrong if you need them in our specs. A simple multiline string terminal should be sufficient.

oehme commented 9 years ago

Well we still have closures for SAM types. And I'd say you can mostly get rid of anonymous classes using stubs with some mocking framework. What use cases do you have in mind which couldn't be solved like that?

2014-11-05 11:38 GMT+01:00 Sebastian Benz notifications@github.com:

That's great Stefan!

I personally have no problem to drop features, as the alternative is even worse - being always one step behind in the migration process (besides the required maintenance effort). Class definitions and anonymous classes are only recent additions anyway. However, anonymous classes are really helpful when writing tests, some kind of alternative would be nice at least.

Template expressions are no problem either, as I would say that you are doing something wrong if you need them in our specs. A simple multiline string terminal should be sufficient.

— Reply to this email directly or view it on GitHub https://github.com/sebastianbenz/Jnario/issues/160#issuecomment-61788539 .

sebastianbenz commented 9 years ago

Sometimes it is more convenient to write stubs on you own (e.g. when working with callbacks). But this shouldn't be a reason to not migrate to xbase.

borisbrodski commented 9 years ago

I think, that rich strings my actually be pretty think. Consider following example:

def table {
  | param1 | mytag  |
  | 1      | "arg1" |
  | 2      | "arg2" |
}

fact "it works" {
  table.forEach [
    val xml = '''
      <request>
          <do-something param="«param1»">
             <«mytag»/>
          </do-something>
      </request>
    '''
    process(xml) should be "ok"
  ]
}

What's wrong with this? As far as I know, rich string support isn't really a big deal to implement and maintain.

PS Why it's not a part of the Xbase at the first place?

borisbrodski commented 9 years ago

Removed Xtend dependency from EMF-models and Xtext grammars. I run didn't EMF- and Xtext- generators, so everythings compiles sofar.

https://github.com/borisbrodski/Jnario/commit/edbf311825988873f9b8dbec2e2bc01578f2f3ba

Please, review the new modell.

I removed "create" functions. I also removed RichStrings replacing it with a TODO. I think we may add it later, if needed.

PS Where are some weird problem with newlines. I tried to use Windows and Unix newlines. In both cases I got newline changes :( Please, compare ignoring whitespace and newline changes.

oehme commented 9 years ago

Could you please fix your git config so it doesn't change newline characters? What OS are you on? What does git config --list print?

borisbrodski commented 9 years ago

So, here is the same commit without newline changes: https://github.com/borisbrodski/Jnario/commit/edbf311825988873f9b8dbec2e2bc01578f2f3ba

The only reliable way to remove white space and newline changes I found is to use this alias

alias.addNoWhitespace=!sh -c 'git diff -w --no-color $@ | git apply --cached --ignore-whitespace' -
oehme commented 9 years ago

Just to let you know, I will be at Devoxx next week, so I won't have time to work on this until late November.

sebastianbenz commented 9 years ago

Same here. I will be on a Hackathon next week. Starting the week afterwards I am on vacation for the rest of the year with limited internet connection. In January will be busy with moving to London. Unfortunately, this means that I will have almost no time to work on this, but I will try to be available for feedback most of the time.

borisbrodski commented 9 years ago

Thanks for the info. I will try to move forward on this in the mean time.

It looks like ModelInferrers must be almost rewritten from scratch (combining Xtend and old-Jnario Inferrer parts together)

Any comments are welcome!

oehme commented 9 years ago

What do you mean by "combining Xtend and old-Jnario Inferrer parts together"? We don't want to copy Xtend, we want a slim and maintainable Jnario. Am 07.11.2014 11:57 schrieb "Boris Brodski" notifications@github.com:

Thanks for the info. I will try to move forward on this in the mean time.

My plan:

  • Regenerate EMF and Xtext code (xtend-free)
  • Comment out as much as possible (adding // TODO NO_EXTEND marker)
  • Get everything else compile
  • Get everything else run
  • Uncomment TODO NO_EXTEND parts and get it to work (can be easy parallelized)

It looks like ModelInferrers must be almost rewritten from scratch (combining Xtend and old-Jnario Inferrer parts together)

Any comments are welcome!

— Reply to this email directly or view it on GitHub https://github.com/sebastianbenz/Jnario/issues/160#issuecomment-62127403 .

borisbrodski commented 9 years ago

Yes, but there are some code needed in Jnario, that's not available in Xbase. (e.g. Validation of field annotations)

borisbrodski commented 9 years ago

@oehme Why actually Xbase (org.eclipse.xtext.xbase) has dependency on org.eclipse.xtend.lib?

oehme commented 9 years ago

It should only have an optional dependency, because we use Xtend code to implement Xbase. Am 11.11.2014 17:06 schrieb "Boris Brodski" notifications@github.com:

@oehme https://github.com/oehme Why actually Xbase ( org.eclipse.xtext.xbase) has dependency on org.eclipse.xtend.lib?

— Reply to this email directly or view it on GitHub https://github.com/sebastianbenz/Jnario/issues/160#issuecomment-62569275 .

borisbrodski commented 9 years ago

A dummy EMF question: How can I embed XxxImplCustom classes into class hierarchy?

- XtendMemberImpl
   - XtendMemberImplCustom
     - XtendTypeDeclarationImpl
       - XtendTypeDeclarationImplCustom
         - ...

My current approach is to modify Factories creating XxxImplCustom classes, but this seems to be the wrong one.

Thanks!

oehme commented 9 years ago

You shouldn't need to do anything special, just write the XxxImplCustom classes and they will be embedded into the factory.

2014-11-17 13:13 GMT+01:00 Boris Brodski notifications@github.com:

A dummy EMF question: How can I embed XxxImplCustom classes into class hierarchy?

  • XtendMemberImpl
    • XtendMemberImplCustom
    • XtendTypeDeclarationImpl
      • XtendTypeDeclarationImplCustom
      • ...

My current approach is to modify Factories creating XxxImplCustom classes, but this seems to be the wrong one.

Thanks!

— Reply to this email directly or view it on GitHub https://github.com/sebastianbenz/Jnario/issues/160#issuecomment-63296330 .

sebastianbenz commented 9 years ago

I had to patch the EcoreGenerator to get it to work (I think Xtend uses a different naming scheme). 

    component = org.jnario.PatchedEcoreGenerator {
        genModel = "platform:/resource/${projectName}/model/Feature.genmodel"
        srcPath ="platform:/resource/${projectName}/src"
        srcPath ="platform:/resource/org.eclipse.xtext.common.types/src"
        srcPath ="platform:/resource/org.eclipse.xtext.xbase/src"
        srcPath ="platform:/resource/org.eclipse.xtend.core/src"
        srcPath ="platform:/resource/org.jnario/src"
    }
borisbrodski commented 9 years ago

Migrated "no_xtend" branch to Xtext 2.8.1

sebastianbenz commented 9 years ago

This is fantastic! How is your progress?

borisbrodski commented 9 years ago

Pretty good I think. Check it out: https://github.com/borisbrodski/Jnario/tree/no_xtend

With Xtext 2.8.1 (and no dependencies to Xtend) I'm able to compile and run very simple feature, very simple spec and a corresponding suite. :)

sebastianbenz commented 9 years ago

Great! Do think that it will be possible to port all features to your version?

On Thu, Apr 2, 2015 at 11:59 AM Boris Brodski notifications@github.com wrote:

Pretty good I think. Check it out: https://github.com/borisbrodski/Jnario/tree/no_xtend

With Xtext 2.8.1 (and no dependencies to Xtend) I'm able to compile and run very simple feature, very simple spec and a corresponding suite. :)

— Reply to this email directly or view it on GitHub https://github.com/sebastianbenz/Jnario/issues/160#issuecomment-88865548 .

borisbrodski commented 9 years ago

I'm not sure about all features, but most features with no doubt. Currently I'm going through the tests to better understand the situation.

borisbrodski commented 9 years ago

By the way, could you please help me to remove dependency to xtend-maven-plugin from the jnario-maven-plugin project?

I'm not that experienced in maven plugins and it's not a requirement for my project here.

sebastianbenz commented 9 years ago

Are you sure that the jnario maven plugin has a dependency to the xtend plugin? I think I duplicated the code anyway.

borisbrodski commented 9 years ago

Yes, at least

import org.eclipse.xtend.core.XtendStandaloneSetupGenerated;
import org.eclipse.xtend.core.xtend.XtendPackage;
import org.eclipse.xtend.core.XtendRuntimeModule;
import org.eclipse.xtend.core.compiler.batch.XtendBatchCompiler;

Why do you need all this maven stuff any way? Xbase projects can be build from maven without any coding needed. Did you added some cool features to the maven build?

sebastianbenz commented 9 years ago

I think I copied the corresponding Xtend classes into the jnario plugin. I didn't add any new features, the only difference was the way the ResourceSet is loaded. However, I think all these moved into the JnarioBatchCompiler anyway.

On Tue, Apr 7, 2015 at 12:08 PM Boris Brodski notifications@github.com wrote:

Yes, at least

import org.eclipse.xtend.core.XtendStandaloneSetupGenerated; import org.eclipse.xtend.core.xtend.XtendPackage; import org.eclipse.xtend.core.XtendRuntimeModule; import org.eclipse.xtend.core.compiler.batch.XtendBatchCompiler;

Why do you need all this maven stuff any way? Xbase projects can be build from maven without any coding needed. Did you added some cool features to the maven build?

— Reply to this email directly or view it on GitHub https://github.com/sebastianbenz/Jnario/issues/160#issuecomment-90508398 .

borisbrodski commented 9 years ago

Do you have an idea, why I can't resolve references (by name) to specs and features from suite? (.*\ works)

In editor (during typing) references are resolved and autocompletion works as expected. But when I save the suite referencing a feature or a spec it generates an error "can't resolve reference to ...".

One of standalone compiler tests shows the same error.

Thank you!

sebastianbenz commented 9 years ago

I remember this being quite tricky, but I only had this problem with the standalone compiler.

borisbrodski commented 9 years ago

The error Couldn't resolve reference to Specification 'Example' occurs only if suite-resource get added to the ResourceSet prior to the spec-resource. If this is the case and the suite get loaded first, the spec "Example" isn't yet loaded and can't be resolved. The error message is now stored within the suite-resource.

After spec resource and all other resources are loaded the XtendBatchCompiler tries to redo resolving:

// install a fresh type provider for the second phase, so we
// clear all previously cached classes and misses.
installJvmTypeProvider(resourceSet, classDirectory, false);
EcoreUtil.resolveAll(resourceSet);

But this doesn't work. The old error message stays within the suite-resource, no scope and linking methods get called.

The error must be somewhere outside of the standalone-compiler stuff, since almost exactly the same happens within a runtime eclipse.

Some caching issues? Any ideas? @oehme What would you do to find the problem?

Thank you!

oehme commented 9 years ago

Sounds like cross references are resolved during indexing, which is generally not possible. The most likely candidates are the inferer and resourcedescriptionstrategy. Am 08.04.2015 13:31 schrieb "Boris Brodski" notifications@github.com:

The error Couldn't resolve reference to Specification 'Example' occurs only if suite-resource get added to the ResourceSet prior to the spec-resource. If this is the case and the suite get loaded first, the spec "Example" isn't yet loaded and can't be resolved. The error message is now stored within the suite-resource.

After spec resource and all other resources are loaded the XtendBatchCompiler tries to redo resolving:

// install a fresh type provider for the second phase, so we// clear all previously cached classes and misses. installJvmTypeProvider(resourceSet, classDirectory, false);EcoreUtil.resolveAll(resourceSet);

But this doesn't work. The old error message stays within the suite-resource, no scope and linking methods get called.

The error must be somewhere outside of the standalone-compiler stuff, since almost exactly the same happens within a runtime eclipse.

Some caching issues? Any ideas? @oehme https://github.com/oehme What would you do to find the problem?

Thank you!

— Reply to this email directly or view it on GitHub https://github.com/sebastianbenz/Jnario/issues/160#issuecomment-90884863 .

borisbrodski commented 9 years ago

Got it! Thank you very much!! This was the Linker instead of LazyLinker :/

borisbrodski commented 9 years ago

A lot of Jnario tests looks like this one:

    fact "returns steps"{
        '''
        Feature: My feature
        Scenario: My first Scenario
            Given something
        '''.parseScenario
        scenario("Scenario: My first Scenario").executables => list(step("Given something"))
    }

Since we don't have the rich string support yet, I see following four possibilities:

Any ideas?

sebastianbenz commented 9 years ago

I would go with option two (Add new string type ''' ... ''). Having proper multi-line strings is very important - however I seldom use actual rich strings in my specs. This way you also have the possibility to add proper rich string support later.

oehme commented 9 years ago

Please note that the default string literals ("...") in Xbase are already multiline. Am 10.04.2015 13:01 schrieb "Sebastian Benz" notifications@github.com:

I would go with option two (Add new string type ''' ... ''). Having proper multi-line strings is very important - however I seldom use actual rich strings in my specs. This way you also have the possibility to add proper rich string support later.

— Reply to this email directly or view it on GitHub https://github.com/sebastianbenz/Jnario/issues/160#issuecomment-91517664 .

sebastianbenz commented 9 years ago

I still would prefer having '''...''' as well. You no longer need to care about quotes and double quotes in your text and it makes it easier to migrate tests.

oehme commented 9 years ago

Yes, you can do that by overriding the STRING terminal rule and its value converter. No need for the huge amount of code related to template expressions. =)

Also, while you're talking about Strings: You can overwrite XbaseTypeComputer#_computeTypes(XStringLiteral) so that a single character in single quotes is always a char and never a String. We had to do that conversion for backwards compatibility, but it causes major headaches for newcomers, especially when using assertEquals/shouldBe.

2015-04-10 14:16 GMT+02:00 Sebastian Benz notifications@github.com:

I still would prefer having '''...''' as well. You no longer need to care about quotes and double quotes in your text and it makes it easier to migrate tests.

— Reply to this email directly or view it on GitHub https://github.com/sebastianbenz/Jnario/issues/160#issuecomment-91535109 .

borisbrodski commented 9 years ago

Is there any particular reason why you create jvm object through TypesFactory like here 

def toClass(JnarioClass xtendClass, List<JvmGenericType> scenarios, IJvmDeclaredTypeAcceptor acceptor, List<Runnable> doLater, boolean preIndexingPhase){
    val javaType = typesFactory.createJvmGenericType
    setNameAndAssociate(xtendClass.jnarioFile, xtendClass, javaType)
    acceptor.accept(javaType)
    if (!preIndexingPhase) {
        doLater.add([|init(xtendClass, javaType, scenarios)]);
    }
    javaType
}

Why not just to use XBase eObject.toClass(...) approach?

PS The toClass method above produces an error ... cannot be exported as the target is not contained in a resource. Why did it worked earlier?

sebastianbenz commented 9 years ago

The scenarios are passed in as well. I think I needed them later in the process. The implementation should be the same as in xbase. Did it change in 2.8? On Mon, 13 Apr 2015 at 09:47 Boris Brodski notifications@github.com wrote:

Is there any particular reason why you create jvm object through TypesFactory like here

def toClass(JnarioClass xtendClass, List scenarios, IJvmDeclaredTypeAcceptor acceptor, List doLater, boolean preIndexingPhase){ val javaType = typesFactory.createJvmGenericType setNameAndAssociate(xtendClass.jnarioFile, xtendClass, javaType) acceptor.accept(javaType) if (!preIndexingPhase) { doLater.add([|init(xtendClass, javaType, scenarios)]); } javaType }

Why not just to use XBase eObject.toClass(...) approach?

PS The toClass method above produces an error ... cannot be exported as the target is not contained in a resource. Why did it worked earlier?

— Reply to this email directly or view it on GitHub https://github.com/sebastianbenz/Jnario/issues/160#issuecomment-92273997 .

borisbrodski commented 9 years ago

In your Jnario tests you use pretty often the construct

  /**
   * Docs
   * Docs
   * Docs
   */
  fact "description" {
    "not implemented"
  }

Unfortunately this produces now (with Xbase 2.8.1) the error 

This expression is not allowed in this context, since it doesn't cause any side effects.

How should I deal with it:

sebastianbenz commented 9 years ago

The reason why I add the string is that the spec is not marked as pending - to fix the formatting of the spec. When I think about it, the correct solution would be to only format pending specs differently if we generate the docs after an actual test run (including failed and passed specs). This way we could simply convert this into a normal comment.

borisbrodski commented 9 years ago

Ok, so I just remove "not implemented" strings for now and we will fix the formatting later on. Ok?

sebastianbenz commented 9 years ago

SGTM

riederm commented 9 years ago

Hi guys, does anybody have something like an updated list of open TODOs. @ghaith me are planning to join the efforts. So from this thread and the current state of the noXtend branch I can tell that following things are still todo(?)

borisbrodski commented 9 years ago

Hi everyone,

back from the vacation I continue my work on Jnario. This is the only reasonable way to migrate my customer projects to the new Xtext.

At first I will update the TODO list at the top of this page adding elements from @riederm and some new ones. (How can I give permissions to others to check/uncheck checkboxes in the TODO list?)

For now I will concentrate on getting Jnario tests green. It would be really great, if someone could get the maven build to work. Just get mvn clean install -DskipTests to work.

I'm pretty sure, together we will release first usable Xtend-only version in the near future. Thank you!

riederm commented 9 years ago

hi boris, great news, same situation for me, we're stuck at xtext 2.6.3 until we're successful

i tried to fix some minor bugs, nothing spectecular, but I found myself most of the time in a situation that tests would pass, if the rich strings used in the facts would work as expected. I spent a night trying to extract the rich-string classes from xtend. I stopped when I realized that the xtend rich-strings have a lot of features (like the <> stuff, the <> stuff, special highlighting provider, etc.) ... it was too much for that particular night :-)

The maven clean build runs for me except for the Jnario Maven Plugin which still uses some XtendCompilerXXX classes. I'll take a look at these next week.

Keep me up to date if you start some work on the rich strings, I'm interested in seeing your approach.

oehme commented 9 years ago

You would save yourselves a lot of trouble if you dropped rich strings. They are about 5000 lines of code that you would have to maintain.

I looked at a few tests and most of them have a RichStrings without any replacement variables. You can replace those with normal Strings. Normal Strings in Xbase are multi-line-capable.

2015-08-19 20:30 GMT+02:00 Mathias Rieder notifications@github.com:

hi boris, great news, same situation for me, we're stuck at xtext 2.6.3 until we're successful

i tried to fix some minor bugs, nothing spectecular, but I found myself most of the time in a situation that tests would pass, if the rich strings used in the facts would work as expected. I spent a night trying to extract the rich-string classes from xtend. I stopped when I realized that the xtend rich-strings have a lot of features (like the <> stuff, the <> stuff, special highlighting provider, etc.) ... it was too much for that particular night :-)

The maven clean build runs for me except for the Jnario Maven Plugin which still uses some XtendCompilerXXX classes. I'll take a look at these next week.

Keep me up to date if you start some work on the rich strings, I'm interested in seeing your approach.

— Reply to this email directly or view it on GitHub https://github.com/sebastianbenz/Jnario/issues/160#issuecomment-132733911 .