Could hooks return something?

[hakunin]

Hi, I am just starting to write test for a JS heavy app and hooks seem super useful.

Only I was suprised to find that I can't have return value going back to client side, that would be super userful I think.

For instance, I use FactoryGirl, so it would make sense to create a setup database state with it and get that state as a returned value from the callback.

[jejacks0n]

I understand the desire, but it seems unlikely that your application would be able to load that data via a similar method. Can you provide me with some reasoning for why, and how you would use that information returned from the server -- and a hook implementation that would not give you reliable and known values?

[mikepack]

If I understand correctly, here is a use case:

1. Fire the hook in the spec
2. Create a record on the server
3. Return the record (as JSON to the client)
4. Use the returned data to make an assertion in the test

@hakunin using hooks is a smell. In retrospect it should not have been added. Hooks tie your JS tests to your server, making the tests much more stateful. Once tied to the server, making the tests server-less would be difficult.

There are great acceptance testing tools in Ruby (eg Capybara), and Teaspoon does not try to compete with these tools, meaning that Teaspoon should be used for unit testing only. The recommended approach is to stub your AJAX responses with a tool like Sinon, and generally not rely on the state of the server to get your tests passing.

That said, if you decide to use hooks, you can pass arguments to the hooks to control the data on your server:

Teaspoon.hook('createRecord', {name: 'Mike Pack'})
# Do something to fetch the data on the server
expect(something).to.eq('Mike Pack')

On the server

suite.hook :createRecord do |data|
  create(:user, name: data['name'])
end

[jejacks0n]

Yeah, pretty much what @mikepack said. We discussed it, and I knew the PR to introduce hooks was something of a smell, but there were some compelling reasons for it, and at the time I was unsure the direction Teaspoon might take. Over time, what I've determined is that hooks are a last resort sort of thing -- and providing other tooling has proved more useful. The customizable boot partial for instance was another way to provide similar functionality where the server IS actually useful, like with loading initial configuration or state.

[hakunin]

If Teaspoon is only unit testing framework, then I understand you don't want to go that direction.

What I don't understand is why I should add another framework/testing setup just so I can do both unit and end-to-end tests of the same codebase and furthermore write them in a different language - Ruby instead of JavaScript.

What makes you think Teaspoon should be unit only? Maybe there is something I am missing.

[jejacks0n]

Well, it's not a Teaspoon limitation, it's a limitation of jasmine, mocha, and qunit.. And the inevitable breakdown of JavaScript limitations imposed by the browser. It also creates a coupling of your client tests to your server implementation, which in the long run is a bad idea -- but a lesson well learned yourself.

The JavaScript limitations for instance are, you can't navigate to a different url from within teaspoon. You can't answer a confirmation, or close an alert without hacking their real implementations. You can't select a file in a file input. Those are just a few of the limitations of integration testing from within JavaScript. You can sometimes work around these things, but what you'll end up with is a simulation that can never be 100% real and may have to run in an iframe or similar.

On top of that there's very little tooling in any of the test frameworks for integration level testing. Yes, you can do integration like tests because you happen to be within the context of the "view" layer by having access to the DOM, but it will eventually break down.

All that's to explain the reasons why teaspoon is not an integration testing framework. You can use it for integration like tests, but from my experiences it's ill advised to take that too far. And using the hooks heavily is where I've chosen to draw that line.

You also asked why you would write them in a different language -- but you're already doing that by writing your hooks in ruby. If you're testing ruby through JavaScript or JavaScript through ruby, you've got the same scenario either way.

At the end of the day though, what you're doing isn't particularly bad, but for fundamental reasons I have to decline because I wouldn't want to lead others down a road that I've seen fail long term. For now at least.

Thanks for your thoughts, and the idea.