v4.6.0 -> v4.6.1 broke ability to run individual feature scenarios

[ovgray]

If I try to run an individual feature scenario like this:

vendor/bin/behat --suite=example features/example/first.feature:30

I now get

In FeatureNode.php line 83:

The file should be an absolute path.

I also get this error if I run

vendor/bin/behat features/example/first.feature:30

This runs:

vendor/bin/behat --suite=example features/example/first.feature

But I get the above error with:

vendor/bin/behat features/example/first.feature

I get these errors even if I specify the full path for the feature file.

This does not happen with v4.6.0

I am using this on Windows 10 with:

behat/behat v3.6.1 behat/mink dev-master a534fe7
behat/mink-browserkit-driver 1.3.3 behat/mink-extension 2.3.1 behat/mink-selenium2-driver v1.3.1 behat/symfony2-extension 2.1.5 behat/transliterator v1.3.0

[meshenka]

Same issue here with --tags ~notesting

[ciaranmcnulty]

Thanks for the report; @pfrenssen do you have the same issues on the codebase that prompted the original fix?

[pfrenssen]

Can we get a backtrace? Should be output when passing the -vvv flag. It seems that somewhere in Behat a relative path is passed instead of the absolute path that is expected when FeatureNode is instantiated.

[pfrenssen]

@meshenka can you give some more information? Are you also on Windows?

[meshenka]

@pfrenssen i will find you a stack trace tomorrow.

No we are on php7.3 linux

[ovgray]

Can we get a backtrace? Should be output when passing the -vvv flag. It seems that somewhere in Behat a relative path is passed instead of the absolute path that is expected when FeatureNode is instantiated.

Here is the output of vendor/bin/behat -vvv when trying to run a single scenario:

at C:\xampp\htdocs\adrexpedite\vendor\behat\gherkin\src\Behat\Gherkin\Node\FeatureNode.php:83 Behat\Gherkin\Node\FeatureNode->__construct() at C:\xampp\htdocs\adrexpedite\vendor\behat\gherkin\src\Behat\Gherkin\Filter\LineFilter.php:119 Behat\Gherkin\Filter\LineFilter->filterFeature() at C:\xampp\htdocs\adrexpedite\vendor\behat\gherkin\src\Behat\Gherkin\Gherkin.php:112 Behat\Gherkin\Gherkin->load() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Behat\Gherkin\Specification\LazyFeatureIterator.php:205 Behat\Behat\Gherkin\Specification\LazyFeatureIterator->parseFeature() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Behat\Gherkin\Specification\LazyFeatureIterator.php:189 Behat\Behat\Gherkin\Specification\LazyFeatureIterator->moveToNextAvailableFeature() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Behat\Gherkin\Specification\LazyFeatureIterator.php:90 Behat\Behat\Gherkin\Specification\LazyFeatureIterator->rewind() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Testwork\Specification\GroupedSpecificationIterator.php:84 Behat\Testwork\Specification\GroupedSpecificationIterator->rewind() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Testwork\Tester\Runtime\RuntimeSuiteTester.php:60 Behat\Testwork\Tester\Runtime\RuntimeSuiteTester->test() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Testwork\Hook\Tester\HookableSuiteTester.php:73 Behat\Testwork\Hook\Tester\HookableSuiteTester->test() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Testwork\EventDispatcher\Tester\EventDispatchingSuiteTester.php:83 Behat\Testwork\EventDispatcher\Tester\EventDispatchingSuiteTester->test() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Testwork\Tester\Runtime\RuntimeExercise.php:71 Behat\Testwork\Tester\Runtime\RuntimeExercise->test() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Testwork\EventDispatcher\Tester\EventDispatchingExercise.php:81 Behat\Testwork\EventDispatcher\Tester\EventDispatchingExercise->test() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Testwork\Ordering\OrderedExercise.php:80 Behat\Testwork\Ordering\OrderedExercise->test() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Testwork\Tester\Cli\ExerciseController.php:149 Behat\Testwork\Tester\Cli\ExerciseController->testSpecifications() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Testwork\Tester\Cli\ExerciseController.php:108 Behat\Testwork\Tester\Cli\ExerciseController->execute() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Testwork\Cli\Command.php:63 Behat\Testwork\Cli\Command->execute() at C:\xampp\htdocs\adrexpedite\vendor\symfony\console\Command\Command.php:255 Symfony\Component\Console\Command\Command->run() at C:\xampp\htdocs\adrexpedite\vendor\symfony\console\Application.php:1001 Symfony\Component\Console\Application->doRunCommand() at C:\xampp\htdocs\adrexpedite\vendor\symfony\console\Application.php:271 Symfony\Component\Console\Application->doRun() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\src\Behat\Testwork\Cli\Application.php:124 Behat\Testwork\Cli\Application->doRun() at C:\xampp\htdocs\adrexpedite\vendor\symfony\console\Application.php:147 Symfony\Component\Console\Application->run() at C:\xampp\htdocs\adrexpedite\vendor\behat\behat\bin\behat:34

[pfrenssen]

Thanks for the stack trace, this narrows down the scope of the problem a lot. One of the possible failure paths I see is in the instantiation of the LazyFeatureIterator class. This is passed in an array of paths which should be absolute paths, but this is not enforced.

I see that one possible way where this can go wrong is in the configuration of the test suites. Can you please check that all paths in your behat.yml are absolute? If you see that you have any relative paths, can you either make them absolute, or prefix them with %paths.base% like in this example:

default:
    suites:
        core_features:
            # All relative paths should be prefixed with %paths.base%
            paths:    [ %paths.base%/features/core ]

If you are still having the problem, it could help if you copy the contents of your behat.yml file so I can try to further narrow down the problem.

[ovgray]

Thanks for the stack trace, this narrows down the scope of the problem a lot. One of the possible failure paths I see is in the instantiation of the LazyFeatureIterator class. This is passed in an array of paths which should be absolute paths, but this is not enforced.

I see that one possible way where this can go wrong is in the configuration of the test suites. Can you please check that all paths in your behat.yml are absolute? If you see that you have any relative paths, can you either make them absolute, or prefix them with %paths.base% like in this example:

default:
    suites:
        core_features:
            # All relative paths should be prefixed with %paths.base%
            paths:    [ %paths.base%/features/core ]

If you are still having the problem, it could help if you copy the contents of your behat.yml file so I can try to further narrow down the problem.

The paths entry for each of the suites in my behat.yaml looks like this:

paths: [ '%paths.base%/features/command' ]

If I remove the single quotes, as in your example, I get warned that this is improper syntax:

In Inline.php line 305:

The reserved indicator "%" cannot start a plain scalar; you need to quote the scalar at line 38 (near "paths: [ %paths.base%/features/command ]").

Other than the paths settings, what else in my behat.yml would be pertinent to this problem?

[pfrenssen]

I took this example straight from the documentation at https://docs.behat.org/en/latest/user_guide/configuration/suites.html but I see there is a PR to update it with single quotes: https://github.com/Behat/docs/pull/130

I suppose you don't have any other relative paths in your behat.yml file?

It's a bit difficult to guess what is going wrong. Would it be possible for you to debug it? When I try to replicate it I can see in the debugger by setting a breakpoint in FilesystemFeatureLocator::locateSpecifications() that the $locator variable is a relative path at the start of the method, but then it gets converted to an absolute path inside ::findAbsolutePath() (called by $this->findFeatureFiles() right before it is passed to the LazyFeatureIterator). From this moment on the path is absolute and everything is working as expected.

There must be something different in your case, either this code path is not followed, or ::findAbsolutePath() returns false for some reason. It is hard to say for sure.

Perhaps the base path is not correct? You can see it by running ./vendor/bin/behat --debug - it will be output as the directory for the configuration file.

[ovgray]

@pfrenssen

I'm not very competent with a debugger, and tend to just stick in dd() to see whats going on.

It appears to me that when behat is given a file line argument like "features/example/example.feature:10", what gets passed to FilesystemFeatureLocator::locateSpecifications() as $locator is "features/example/example.feature:10", which is not really a path. But $this->findFeatureFiles($locator) seems to expect $locator to be a path. When $locator it is passed to ::findAbsolutePath as its $path argument, that function adds the base path to it, but it is still not a valid file or directory name because of the ":10" at the end. So that function returns false because is_file returns false in line 168 and a relative path is passed to LazyFeatureIterator. It looks like that is what was intended in that Behat file. Does v4.6.1 of Behat/Gherkin prevent LazyFeatureIterator from handling the relative path?

[pfrenssen]

OK that makes sense. This is something that should be fixed in Behat and not in Gherkin. I will open a sister ticket and try to roll a patch for this as soon as possible. For the moment you can lock version 4.6.0 of Gherkin.

[pfrenssen]

I found a comment that already hints at this problem back in 2016: https://github.com/Behat/Behat/issues/731#issuecomment-172925851

[pfrenssen]

Digging a bit deeper circles me back to Gherkin. The splitting of a locator like features/example/example.feature:10 is happening in Gherkin::load() which will split it into a "resource" (features/example/example.feature) and adds a separate LineFilter which is passed the line number 10. Then the "resource" will pass through GherkinFileLoader::load() which has another instance of the relative path -> absolute path conversion (AbstractFileLoader::findAbsolutePath()). Now I'm seeing that in this code my relative path is correctly converted to an absolute path.

Could you perhaps do some dd() debugging in AbstractFileLoader::findAbsolutePath() to see what is going on? Does it also return false here?

Also, on a hunch, could you try to delete your Gherkin cache files? This has been known in the past to cause hard to debug problems:

$ rm -rf /tmp/behat_*

[talisto]

Also, on a hunch, could you try to delete your Gherkin cache files? This has been known in the past to cause hard to debug problems:

I was also running into the The file should be an absolute path error, and deleting the cache files resolved the issue for me! Thanks for the tip!

[ovgray]

@pfrenssen @talisto

Also, on a hunch, could you try to delete your Gherkin cache files? This has been known in the past to cause hard to debug problems:

$ rm -rf /tmp/behat_*

For those with this same problem in Windows 10, I found the behat_gherkin_cache and behat_rerun_cache folders in "%USERPROFILE%\AppData\Local\".

Deleting the folders and their contents seems to have resolved this issue for me.

[pfrenssen]

@talisto and @adrx that's great!

We should make a mention in the release notes, and possibly on the README that it is recommended to clear the cache after updating. I can make a PR to update the README. @ciaranmcnulty can you put a mention in the release notes?

[pfrenssen]

I just looked into the caching code and there is a mechanism in there to avoid problems with the cache when releasing a new Gherkin version (ref. https://github.com/Behat/Gherkin/blob/master/src/Behat/Gherkin/Cache/FileCache.php#L36). The cache is located in a subfolder that uses the current Gherkin version string as the filename. The version string is a constant Gherkin::VERSION.

Unfortunately this version string has been set to 4.4-dev (see #97 and #116) but we have not been updating this version string to the actual version when preparing a new release. We need to define a release procedure that includes updating this string and setting it back to 4.x-dev after the release is made.

@ciaranmcnulty @stof any idea if any of the other Behat projects have defined a release procedure so we can follow their example?

[pfrenssen]

For the moment it would be great to get a message in the release notes to alert people that might be experiencing this problem. Here is a draft message we can include:

## Known problems

In case error messages such as `The file should be an absolute path` appear after updating to this release, please clear your Gherkin and Behat cache folders.

### On UNIX/linux:
```
$ rm -rf /tmp/behat_*
```

### On Windows:
Remove the `behat_gherkin_cache` and `behat_rerun_cache` folders in `%USERPROFILE%\AppData\Local`

[ciaranmcnulty]

Seems like we should update this and do a patch release?

[ciaranmcnulty]

Hopefully 4.6.2 resolves this

[phil-davis]

Should be fixed by #166 and release 4.6.2

@pfrenssen @ovgray close this issue?

[ovgray]

Closed

[stof]

I suggest that once Composer v2 is out, we rely on the new composer-runtime-api to get the version of behat/gherkin being installed rather than relying on a constant that needs to be maintained manually.