Crazy feature request: enable extending RSpec to run minitest tests

[e2]

I though rspec-core was the best place to ask first, because it's related to syntax/DSL.

Also, while I have a ridiculously strong preference for RSpec (hence the idea I guess), I'm dead serious about being constructive here - so I won't go into anything not related to actually getting this done (or strongly considered).

Example use case

If I encounter a project using minitest (lets say it's even without mocks - which is actually not uncommon anyway)...

... I'd like to run the tests, but using RSpec - along with it's settings, infrastructure, formatters, options and plugins (I'm talking about development tools integrating with RSpec, not extensions to DSL, matchers, or the like).

I'm not talking about "converting" files to RSpec, but actually using them unchanged - with everything seeming like RSpec except the code.

Implementation approaches

In Minitest's parallel mode basically every "test" is run in a single thread, which means we can assume properly written test in Minitest are isolated enough to be treated as independent "blocks" (so if they're not, refactoring benefits the tests anyway, whether run by Minitest or RSpec).

Mocks libs in Minitest usually expire when the block ends, so they can be though of as implicit RSpec before/after/around blocks.

And the obvious improvements, such as formatting test names as if they were it blocks, etc., assertions to format like expectations, etc.

Obviously this would be more suited for a separate gem - I was just wondering how much of what RSpec's DSL offers (which without monkey-patching is very well isolated and elegant) can be "married" with typical minitest/ruby test classes.

Basically, since Minitest uses introspection behind the scenes, I'm assuming this would be more trivial than expected (just mapping classes and methods through introspection into RSpec's DSL calls).

I'm not afraid of monkey-patching anything on the Minitest/TestCase side, since it seems to be the "norm" in the test/minitest world anyway.

Questions

So my 2 questions are: 1) does this make sense (technically in terms of what can be done with RSpec's DSL and what it exposes)? 2) what minimal changes in RSpec (if any) would be necessary? (I admit I haven't researched yet)

(Note: please don't get me started on why I'm asking for this...)

[myronmarston]

This is definitely something that would belong in a separate extension gem if it ever got built. I personally don't have the time to work on such a gem (and I doubt any of the core team would either), given that it would take time away from other issues that affect more users. You're the only user who has ever expressed interest in something like this.

As for the how -- having never used minitest (besides the occasional drive-by contribution to a project that uses minitest), I really have no idea what would be involved as I don't know how minitest models things internally. Regardless, things like this have been done before -- see https://github.com/glv/rspec-unit for example.

If you (or anyone) wanted to build an extension gem to do this, we'd be glad to add any new necessary public APIs to RSpec to make this possible. I don't know what those would be -- really, the requests for those would have to come from the person building the extension gem.

I'm going to close this as I don't think it makes sense to put this in rspec-core (and thus, there's nothing actionable for us to do here) but feel free to continue the conversation and I'll keep responding, and as I said, we'd be glad to support you (or anyone) doing this in the form of new APIs if needed -- so please ask for what you need.

[e2]

Excellent! Thanks.

I wanted to suggest closing the issue, anyway - and I wouldn't expect the core team to work on anything other than improving RSpec.

Also, rspec-unit is great starting point - thanks for pointing me there!

[e2]

RSpec is so awesome and well thought out, this was even easier than I thought.

Proof of concept: https://gist.github.com/e2/bcd2be81b4ac28c85ea0

The code is ugly, quick and dirty - but the result is beautiful, thanks!

As for supporting mocks, this seems relevant: https://github.com/rspec/rspec-core/issues/683, but it's unlikely there will be more than one mock framework used at a time.

Thanks for making RSpec so marvelous!

[e2]

The most work will probably be with rewriting the backtraces and example line numbers.

I'm guessing RSpec isn't internally prepared to handle stuff e.g. cat foo.rb | rspec or rspec <(cat foo.rb). I'm thinking along the lines of how instance_eval takes a filename and line number as a parameter.

[myronmarston]

Nice to see you got something working really quickly :).

I noticed that you defined your own assertions module to provide a tiny subset of minitest's assertions. As it turns out, you don't need to do that -- you can just use minitest's assertion module directly from RSpec:

https://relishapp.com/rspec/rspec-core/v/3-1/docs/expectation-framework-integration/configure-expectation-framework#configure-minitest-assertions

[e2]

Thanks, it works great (including backtrace)!

Side note: I actually realized it took me less time to write this than it ever took me to get colors working in minitest.

I think I'll check out how to handle passing locations to rspec in a compatible way - and if I get that working, I can release a gem. I'd love to see this working with e.g. guard-rspec.

I do get a test summary with "No examples found." before the actual tests - I'm wondering if that's due to RSpec's at_exit hook firing for the formatters or something. But I'll work it out when I get back to it.

[splattael]

@JoshCheek created mrspec and gives @e2 credit for his initial idea pointing to this issue.

I'd like to link back to help future interwebs users: https://github.com/JoshCheek/mrspec

[JoshCheek]

Thanks for the idea, @e2, I wouldn't have even tried it if not for your gist :)

@splattael thanks for linking :) I was intending to come back here and make a comment, but was head down in the code all night, and found out this morning that I was scheduled to teach today. So had to get it completeish quick and run off to work >.<

On the upside, the lesson was How Testing Works, and I'm always excited to talk about how to think about testing so you achieve the benefits. It was really nice to have the runner, today, we start the students off with Minitest because they can reason about classes and methods better than blocks. Then, once they have some momentum, we only require them to test their code, but don't dictate which libs to use (though I give feedback on how to improve their tests, and use their tests to improve their design) so some switch to RSpec, and some continue using Minitest. The runner worked well, with the lesson, and I got their ~/.rspec setup, and we got to try a lot more iterations of the test than I expected, because it was expressive enough, and quick enough that we could try out lots of ideas quickly :) I decided not to go into features of the runner beyond "run this file", because they're only 3 days in, and needed me to talk for a while about classes, methods, namespacing, why they need require, etc. I figured today wasn't the day to focus on command-line args.

@myronmarston man, I saw that Twitter post had threads I was unaware of because I hadn't been mentioned in a bit and my attention was on coding. Didn't even realize it until lunch time, and only just got to reading it. I wasn't trying to draw any lines in the sand or criticize anyone/thing with it, I just thought it was a cool idea. I'm a big fan of RSpec, and I especially appreciate your rigor and investment to it (there are other projects where my PRs lie fallow for months, even while a majority of users are affected). The detailed and thoughtful feedback has been really nice, too. It's really nice to have a project that is improving (I really like the direction RSpec has moved for v3), with maintainers who care, and to have an informed opinion. A lot of the stuff I do, I'm alone and ignorant, so the environment around RSpec has been very appreciated.

Also, I know I get wrapped up my own childishness from time to time and then get annoyed or confrontational (esp with expressions of authority / rules). If I do that to y'all, or you see me do it elsewhere, just point it out, my super-ego will kick in (I say that b/c it's a convenient idea, not as an affirmation of Freud), I'll contemplate who I want to be and what kind of world I want to live in, and then I'll find a way think and behave in a kinder more positive manner. Anyway, I really enjoyed the honesty in your responses ^_^ It takes a courage to expose vulnerabilities publicly.

So, just to be super clear: poke me when I get whiny or snide, b/c at the end of the day, I want a more positive inspiring world too :) And I use RSpec heavily and appreciate all the work y'all are doing on this!

[myronmarston]

@JoshCheek -- :heart: :heart: :heart: Thanks for the note! Nothing you said bothered me one bit, and I definitely want constructive cricitism of my work, as long as it's given respectfully---which is the case with virtually all the feedback I hear about RSpec (and has definitely been the case from you). Ryan's feedback has never felt respectful to me. That may just be my experiences and insecurities coloring by perception. It dates back to my very first ruby conference, when Ryan gave a talk which essentially felt like "RSpec sucks; Minitest is better and here's why" and which included a dissection of some of my code as part of his talk. I felt humiliated, frankly. He later invited me to seattle.rb, and I went, but frankly didn't feel welcome (my coworker came along with me and later asked me, "Was it just me, or was there an anti-RSpec bent to the entire evening?"). I thanked Ryan for the evening, but told him I didn't feel welcome and that I wasn't going to be spending my limited free time returning...which Ryan now presents as "tucking talk". Apparently choosing not to attend a group he runs is an act of cowardice or something.

Last night a lot of that background just kinda boiled over. I actually wasn't nearly as upset as my tweets probably made it seem, but it felt like something I needed to get off my chest.

[e2]

@splattael - thanks for linking!

@JoshCheek - awesome! I never got to finishing the gem, but I wouldn't have been as good anyway. Thanks! As for students, I'd suggest (at some point?) using guard-rspec and/or guard-minitest (avoids the discouragement of running the tests manually). Let me know if you have any issues with those.

@myronmarston - I saw the tweets, I watched the talk, and read a bit here and there. What I got from the "Size doesn't matter" is: my values/priorities are in conflict with those of the Seattle.rb group. There's no point in discussing "respect", "I am not my code", etc. - since emotions are not based on logic and reasoning.

I never had any good experiences with MiniTest. Whether it's because "I suck at Ruby" is irrelevant - while I'd agree the speed is nice, I'd actually care less about the "code size" and "no magic" arguments, but ... not having a configurable runner is a deal-breaker for me.

You're not going to change the "anti-RSpec" approach ever, because of a emotional investment in specific values/priorities that camp has. I don't find the "I am not my code" relevant at all. Instead, "I am not my beliefs/values" is where it's hardest to separate "ourselves" from "what's important", because we all want to feel important to begin with.

I'm not even willing to hang around people who have such a different set of values - there's no upside.

To avoid a long rant, I'll just say we all lack "social education" - I'd blame schools (if blaming actually worked), because we're all poorly prepared to cooperate with each other.

E.g. I was never taught that "disrespect" has to first be treated and "accepted" as such, and, "there's a good intention behind every negative behavior". And, to completely avoid social conflicts, at least one side has to have both inhuman self-control and inhuman empathy - at the same time (which is why conflicts are so common).

So while I'd say debating with Ryan makes no sense (because of a difference of values to begin with), I'm quite sure you both understand "respect" differently.

Personal example: I used to believe people who were "nice" about what I did wrong were "disrespectful" towards my professionalism. Nowadays, I believe people will always produce crap - compared to what they are ultimately capable of. This turns the "I am(not) my code" debate into simply embracing the emotional conflict/duality behind it (that's there's no "single" "correct" emotion here).

This suggests: there's a way of looking at things, where Ryan's responses aren't disrespectful. They may actually be respectful.

Friction regarding values/priorities is very different than friction towards getting work done.

E.g. I don't want to find solutions to my MiniTest problems, because I don't like the experience of doing so (no matter how trivial or "standard" the workarounds for fixing my issues) - therefore, I like mrspec very much, because I no longer have to deal with experiences I don't want (well, except that tests still use the inferior syntax). And I don't care about listing specifics in order to improve MiniTest (even if other users could avoid the problems I had).

I'm pretty sure there are analogies with how Ryan feels about RSpec. So instead of "thick/thin skin" debates, it's better to be clear about the differences in values first (because it avoids dead-end disputes).