search

php-pds / skeleton

Standard PHP package skeleton.
Creative Commons Attribution Share Alike 4.0 International
2.32k stars 167 forks source link

Tests dir won't work for some devs #2

Closed garrettw closed 7 years ago

garrettw commented 7 years ago

I know that PHPUnit is the most widely used testing suite out there, but I (and some others here and there) have made a conscious choice to use PHPSpec, which accomplishes basically the same thing but with a different (and in my opinion, better) grammar. It doesn't call it's code "tests" but rather "specs" to point out that what you write are specifications of what output is expected for certain input. And as such, in contrast to PHPUnit, it automatically uses a "specs" directory to hold that code. So because of that, none of my projects will have a "tests" directory.

I would also mention Behat, which uses a "features" directory for its own set of reasons.

Is there some way an allowance for this kind of thing could be worked into the standard?

pmjones commented 7 years ago

Before we continue, I want to point out a subtlety of the pds/skeleton language. It says if you have a top-level directory/file for a certain purpose, then it must be named per the standard. This has two ramifications related to your point above:

  1. The standard does not require a tests directory. It says only that if you have a testing-related directory, then it has to be named tests/. Since what you have are "specs" and not "tests", it does not appear to be covered by that particular part of the draft standard.

  2. The standard allows for additional directories/files for purposes not already covered by the standard. Since "specs" is not covered, having a "specs" directory is not disallowed.

Does that help to clarify?

garrettw commented 7 years ago

I suppose. I guess I thought PHPSpec and Behat stuff would be considered testing-related.

pmjones commented 7 years ago

I guess I thought PHPSpec and Behat stuff would be considered testing-related.

Maybe? You made the case above that they're "specifications" and not "tests". Under that argument, it strikes me there is room for them in the draft as it stands now.

As a side note, the directories "spec/", "specs/", "features/", etc. do not appear frequently in the sample (combined, they appear fewer than 1500 times across 65000 packages, cf. https://github.com/php-pds/skeleton_research/blob/1.x/results/01-dirs.txt). So it's hard, merely from a relative frequency point of view, to say there's enough usage to warrant explicit inclusion in the draft.

I know that's unsatisfying. :-(

pmjones commented 7 years ago

Thinking more about this, there may be some way to loosen the language a little, to allow for tools that require a specific directory name at the top level. Alternatively, it could become more specific, and mention only particular types of testing. Either way, it would have to stay away from mention of specific tooling. Looking more closely into the contents of existing "tests" directories would be informative here.

garrettw commented 7 years ago

You made the case above that they're "specifications" and not "tests".

Haha. Yes, I guess I did, but the keyword is "related" - as they serve a similar purpose. However, being that it's not a common practice as you say, what about maybe just changing the language from "MUST" to "SHOULD" for the item in question?

pmjones commented 7 years ago

Hi again!

I've been reading http://www.phpspec.net/en/stable/cookbook/configuration.html#spec-and-source-locations and it appears that while "spec/" is the default for PHPSpec, it can be configured relatively easily to be some other directory.

Does that modify your feelings about the tests/ directory? Users of multiple testing systems might split their tests into subdirectories there, a la tests/unit/, tests/spec/, tests/features/ etc. (I am not suggesting that for the standard, only that a top-level tests/ directory can accommodate that kind of structure.)

Revisor commented 7 years ago

I agree with Paul. tests is very intuitive unlike specs, and various types of tests can reside under the tests directory (unit, integration, specs, acceptance...).

That is also how Codeception works: http://codeception.com/docs/02-GettingStarted

garrettw commented 7 years ago

Interesting. Ok, I might could get on board with that.

pmjones commented 7 years ago

Cool! Think about it some, and let me know where you end up.

And thanks for taking the time to raise the issue in the first place. :-)

pmjones commented 7 years ago

@garrettw Any further thoughts on this? If not, and you if feel this has been resolved, you can close the issue at your convenience.

garrettw commented 7 years ago

I like the idea of using tests/spec because I agree that "tests" is intuitive, plus I can easily see how different kinds of tests should be grouped under one directory. Makes sense.