2.0.1 snapshot with test

[magomimmo]

Hi Creighton, I tried to follow my way and it seems to me to be much simpler. I did not use deftemplate, because all I need (a js/document) to create your fixture is already created by the Chas's script. You can test that everything works as expected by issuing:

lein do clean, compile, test
...
Running all ClojureScript tests.

Testing enfocus.core-test
...
Ran 26 tests containing 50 assertions.
1 failures, 0 errors.
{:test 26, :pass 49, :fail 1, :error 0, :type :summary}
...
Ran 26 tests containing 50 assertions.
1 failures, 0 errors.
{:test 26, :pass 49, :fail 1, :error 0, :type :summary}
...
Ran 26 tests containing 50 assertions.
1 failures, 0 errors.
{:test 26, :pass 49, :fail 1, :error 0, :type :summary}
Subprocess failed

My personal opinion is that we should now add other runners to clojurescript.test to cover others js engines. I'm not a JS guy (to me JS is a bad lisp in C dress), but you can start trying with slimerjs or others JS engine.

Hope you like it ;-)

My best

[magomimmo]

ah, I forgot to say that the lein jar produces exactly what we want.

HIH

[ckirkendall]

@magomimmo I completely agree with the separations of concerns and intend on moving the whole reporting code out into a library. Removing it all together really isn't an option though because we have requirements that dictate it. If it will make you feel better I can pull it out sooner than later. I also agree that each-fixture should not be in reporting-generator.

On this pull request and supporting other JS/Engines. JS Engines like phantom and gekco are special because they are full browser implementations that allow you to load in resources and work with a live dom (required for any testing with enfocus). They are great for development testing but even they are limited. They are not able to tell you if a given browser will test correctly. This is true even if it implements that js/engine. Enfocus must be able to state what browsers are supported. The only way I know how to accomplish this is to have to clean easy way to view the tests in a browser. This needs to be a visual view using the live dom because not all browsers we need to support have a js/console. When thinking about testing tools like domina or Enfocus you have to think about a way to structure it so that if someone says does it work in this browser you can point them to a URL that will tell them for sure. A good example is mobile browsers. I been asked in the past if I supported a given mobile browser. I had no way of even accessing this browser as I didn't have phone that supported it. I was able to give this person directions on how to bring up the test suite in that browser to see what worked and what didn''t, so they could decide if enfocus met their needs.

I know the setup seem complex but once its pulled out into a separate library I don't think it will be an issue. I hope my explanation help you understand why the visual representation in the browser is so important to what we have to do. Thank you again for helping me out with this. Having you to discuss all this stuff with really helps me. Please don't stop challenging things where you feel they need to change I really appreciate it a lot.

CK

[magomimmo]

@ckirkendall I think we need to have both type of tests. A standard unit test approach to be run against anything that does not really need a visual confirmation. And that kind of reporting you implemented to be run against the things needing a visual confirmation in more browsers. The ones that do not need visual confirmation still need JS engine validation. So if we test them against phantomjs only, we can only say that they succeeded in a "web-kit" based browser. This is why we should extend the clojurescript.test lib to cover more headless-browsers runners as asked by Chas.

That said I agree that there are a lot of stuff that can't be tested on an headless-browser and need the visual confirmation in more browsers.

They are two different categories of testing and we should keep them separated one from the other. If you're able to externalise the reporting facility in a separated lib to be re-used with other cljs lib (e.g. domina), this is great.

My best

[ckirkendall]

@magomimmo I think I need to understand a few things. I having a hard time understanding your objections to extending the reporting for all tests, so we can cover both uses cases with one strategy. If I separate the two (not really possible the way clojurescript.test is designed) I have to worry about if a tests needs visual confirmation vs just script. The current setup I have, works in both cases and still can be extended to work with other JS engines as long as they are headless browsers. Maybe the two of us could get one a google hangout and discuss this because I am confused by what is in this pull request and what you are saying. In this pull request you removed the ability to cover the other use cases and I can't see why you would want to do that.

Right now the setup allows the following:

1. running lein test -> all results reporting against phantomjs
2. running in the browser -> all results reported js/console
3. A visual representation is created in the live dom allowing for quick inspection of test on various browsers
4. ability to bring a browser up and have a faster test cycle than through lein test.

I would love to hop on a google hangout to discuss this if you have time.

CK

[magomimmo]

On Oct 23, 2013, at 12:21 PM, Creighton Kirkendall notifications@github.com wrote:

Maybe the two of us could get one a google hangout

sure. I'll let you know if this afternoon I'll be available around 4 pm - http://www.timeanddate.com/library/abbreviations/timezones/eu/cest.html

In this pull request you removed the ability to cover the other use cases and I can't see why you would want to do that.

Sorry about that. It was just because I wanted to be sure to have a clean state. It was not because they are not useful.

Right now the setup allows the following:

1. running lein test -> all results reporting against phantomjs

ok

2. running in the browser -> all results reported js/console

ok

3. A visual representation is created in the live dom allowing for quick inspection of test on various browsers
4. ability to bring a browser up and have a faster test cycle than through lein test.

I would love to hop on a google hangout to discuss this if you have time.

I'm envisioning something like this:

a. let clojurescript.test lib works as usual with the embedded runner.js and accepting a *.js argument b. try to add slimerjs for supporting FF JS Engine/HTML/CSS renders c. try to classify which enfocus API needs programmatic unit tests only and which API needs a visual representation in any browser. d. keep the programmatic unit test class separated from the ones that needs a visual representation and treat them differently. e. eventually, the unit tests which needs a visual feedback could call the underlying programatic tests f. modify the project.clj to keep the things separated.

The above is just a very first plan and needs to be unfolded in a real plan

I let you know for this afternoon…..

my best

mimmo

CK

— Reply to this email directly or view it on GitHub.

[ckirkendall]

@magomimmo to give you a view of where my vision for all of this is. I want to move to a contentious build using travis-ci and possibly sauce labs for automated testing. That way as more people get involved it will be easy to manage the testing verification.

http://about.travis-ci.org/docs/user/gui-and-headless-browsers/ https://saucelabs.com/

CK

[magomimmo]

:-). I'' take a look at them.

thanks

On Oct 23, 2013, at 12:43 PM, Creighton Kirkendall notifications@github.com wrote:

@magomimmo to give you a view of where my vision for all of this is. I want to move to a contentious build using travis-ci and possibly sauce labs for automated testing. That way as more people get involved it will be easy to manage the testing verification.

http://about.travis-ci.org/docs/user/gui-and-headless-browsers/ https://saucelabs.com/

CK

— Reply to this email directly or view it on GitHub.

[magomimmo]

I just added the slimerjs test commands. Now you can run all the unit tests on both webkit and gecko. I'm available at 4.00 pm GMT + 2 for google hangout

[ckirkendall]

@magomimmo we will have connect up later this week. I have meetings all day today. What city do you live in? Do you sleep. ;) Can you split out the slimjs changes into a separate pull request that doesn't delete the reporting structure. That way I can get those in while we decide on how to proceed.

[magomimmo]

ok. no problem. I leave in Milan (Italy - GMT + 2). yes, I'll separate the pull requests later

On Oct 23, 2013, at 2:42 PM, Creighton Kirkendall notifications@github.com wrote:

@magomimmo we will have connect up later this week. I have meetings all day today. What city do you live in? Do you sleep. ;) Can you split out the slimjs changes into a separate pull request that doesn't delete the reporting structure. That way I can get those in while we decide on how to proceed.

— Reply to this email directly or view it on GitHub.

[magomimmo]

I'm closing this pull request to submit the next one which incorporates both the standard unit testing and the reporting unit testing.