search

luckyframework / lucky_flow

Automated browser tests for web applications. Similar to Ruby's Capybara.
https://luckyframework.github.io/lucky_flow/
MIT License
51 stars 8 forks source link

Consider switching to Spectator #115

Open alexanderadam opened 3 years ago

alexanderadam commented 3 years ago

So at the panel discussion @matthewmcgarvey mentioned that he would love to have mocking and additional features available in specs. I guess by "missing" he meant in reference to RSpec.

Luckily (!), @icy-arctic-fox wrote Spectator, which is a testing framework based on RSpec (> 3). It has an incredible amount of RSpec features already available.

I guess one of the main decisions whether to switch from an stdlib testing library to an external one, is usually the amount of dependencies but in Lucky's case we are talking about a fully fledged web framework that should allow developers to develop apps as secure, convenient and fast as possible. If someone is looking for a solution that's as tiny as possible, Lucky might not be the optimal choice. There are other tiny web frameworks in Crystal that are focussing on that.

Thus, while the standard Crystal spec is functional and intentionally kept tiny, this doesn't mean that Lucky has to stuck with it. Ruby's RSpec is also not the standard test library in Ruby but it is yet one of the (or the?) most popular test library.

Also Matthew is most likely not the only web dev who's missing convenient test features that would be solved by switching to Spectator (see here, here, here or various comments in this issue). Which is absolutely understandable, since tests/specs are giving security and allow us to develop better applications that are as stable as possible.

PS: Also using the non-intrusive expect syntax might be an advantage. PPS: I absolutely understand that this might be a hot topic but I'm sure it it much better to have this discussion at this point than at a much later point where a full developed ecosystem might have been evolved that would make it even harder to switch. I guess a (slow) migration at this point might still be a realistic option and Lucky's users would gain a lot. PPPS: Thank you so much for Lucky. :pray: Really. Lucky is wonderful!

thoran commented 3 years ago

The team I'm working in has been using Spectator with Lucky for a few months now. Switching LuckyFlow over would be good for us at least...

jwoertink commented 3 years ago

I think this is more of a Lucky related topic than LuckyFlow. LuckyFlow is specifically for interacting with front-end components with a few test helpers. With that said, one of the internal Lucky "guide/rules" is that we avoid including any shard outside of the Lucky org that we can't directly control. That could change in the future as shards become updated more frequently, and releases become more stable. The main reason for that is just when a shard we use breaks, then it requires us to submit an issue / PR, and then wait for a release of that shard which causes blocks for us.

I think instead of adding spectator directly in to the Lucky ecosystem, we should just add some guides to our testing section on how to include it. Now, I've never actually used this shard before, so I'm probably not the best one to write the guides, but if anyone else would like to submit a small section to our testing guides on how to best utilize spectator to really beef up your testing in Lucky, I think that would at least be a great start!

alexanderadam commented 3 years ago

I think this is more of a Lucky related topic than LuckyFlow.

This absolutely makes sense. I wasn't sure which would be the better place for this. But I guess moving this issue won't be the major issue here‽ :wink:

we avoid including any shard outside of the Lucky org that we can't directly control

While I understand the general idea, I think that this should not hinder improve the development experience. And as mentioned earlier, the stdlib spec is intentionally kept tiny. So in this way Spectator relates to stdlib spec like Lucky to the integrated HTTP::Server.

Apart from the avoiding external shards thing: did you never miss more advanced spec features so far? Did you never see any room for improvement in this regards?

The main reason for that is just when a shard we use breaks, then it requires us to submit an issue / PR, and then wait for a release of that shard which causes blocks for us.

Of course. I mean this is true for many other dependencies in many other languages, frameworks and libraries and many of them are doing great although they are often using external libraries. Otherwise there might also be some solutions like adding a thin abstraction (lucky_spec?) that ensures compatibility. Or maybe it would make sense to invite @icy-arctic-fox to join the core team to oversee the testing functionality?

I absolutely understand the abstract desire to keep out external dependencies but I won't lose hope yet that those two wonderful projects will find their way to each other. :wink:

jwoertink commented 3 years ago

Apart from the avoiding external shards thing: did you never miss more advanced spec features so far?

The only thing I'm really missing is mocks and stubs which I see spectator supports, but we have a project started to solve this! Just a matter of getting it fleshed out and having time to work on it. Maybe we can just bring over that portion in to our project we started 🤔

We should probably move this to a discussion https://github.com/luckyframework/lucky/discussions and see what others think on the testing front.