Windows compatibility issues

[fritzmg]

On my local system under Windows 7 x64 and XAMPP 1.8.3 (PHP 5.5.11) the Contao 4.0.0.-beta1 installation via composer create-project contao/standard-edition contao4 4.0.0-beta1 fails, or rather the app/console command contao:symlinks fails with the message

[Symfony\Component\Filesystem\Exception\IOException]
Failed to create symbolic link from "../../C:\xampp\htdocs\contao4\vendor\contao\core-bundle\src/Resources/contao/themes\flexible" to "C:\xampp\htdocs\contao4/system/themes/flexible".

Only the theme symlink fails initially, the other symlinks before are successfully generated by Symfony. The problem seems to be the leading '../../' which creates a path like ../../C:\xampp\htdocs\…. When those are removed from

$this->symlink("../../$path", 'system/themes/' . basename($path), $rootDir, $output);

then the symlink is created successfully. However, all following symlinks cannot be generated as well, i.e.

$this->symlink('../assets', 'web/assets', $rootDir, $output);
$this->symlink('../../system/themes', 'web/system/themes', $rootDir, $output);
$this->symlink('../app/logs', 'system/logs', $rootDir, $output);

Though I haven't figured out yet why.

[bezin]

I have the exact same issue on Windows 8.1 / Apache 2.4.9 / PHP 5.5.12.

[leofeyer]

It seems that the absolute paths is not replaced here:

https://github.com/contao/core-bundle/blob/develop/src/Command/SymlinksCommand.php#L140

Can you debug this?

[fritzmg]

Ah, yes it's because of the forward slash of "$rootDir/". On Windows, $rootDirwill be:

C:\xampp\htdocs\contao4

$theme->getPathname() will be:

C:\xampp\htdocs\contao4\vendor\contao\core-bundle\src/Resources/contao/themes\flexible

str_replace will search for:

C:\xampp\htdocs\contao4/

which is not present.

Maybe you could use $rootDir . DIRECTORY_SEPARATOR instead, but isn't there a better solution to these kinds of problems?

[fritzmg]

Hm, ok, even with the change I get

[Symfony\Component\Filesystem\Exception\IOException]
Failed to create symbolic link from "../../vendor\contao\core-bundle\src/Resources/contao/themes\flexible" to "C:\xampp\htdocs\contao4/system/themes/flexible".

If I remove the ../../ again from "../../$path" it will work again. Why are the ../../ necessary in the first place?

[bytehead]

@fritzmg then it's probably related to #215 now...

[bytehead]

Does it work with the change in e907f1449c7ebd1f550f76d99ea9e3f27bcb704a?

[fritzmg]

Does it work with the change in e907f14?

No. symlinkThemes() does not use relativeSymlink().

[bytehead]

You're right. The used DIRECTORY_SEPARATOR (/) is some kind of wrong...

[bytehead]

The str_replace on line 140 should use the DIRECTORY_SEPARATOR.

(see also http://alanhogan.com/tips/php/directory-separator-not-necessary)

(Note that DIRECTORY_SEPARATOR is still useful for things like explode-ing a path that the system gave you. ...)

[leofeyer]

It still should not be necessary to use DIRECTORY_SEPARATOR …

[leofeyer]

Does it work if you change line 140 to the following?

$path = str_replace($rootDir . DIRECTORY_SEPARATOR, '', $theme->getPathname());

[fritzmg]

I already tried that here https://github.com/contao/core-bundle/issues/208#issuecomment-93775626 But it doesn't matter, the problem is still ../../$path later on. Even if the root directory does not get removed from the path, the problem is only the relative path used in

$this->symlink("../../$path", 'system/themes/' . basename($path), $rootDir, $output);

I don't understand why ../../ is needed in the first place? At least on my system, under windows, the path will be correct without the ../../, no matter if the root directory is prepended or not.

[leofeyer]

It is supposed to be a relative symlink: "go two directories up (from /web/themes to /) and then add the path (from / to /system/themes)".

[leofeyer]

Could be this:

Symlinks on windows are created by Symlink() which accept only absolute paths but not relative paths .relative paths on windows are not supported for symlinks

:(

[fritzmg]

It is supposed to be a relative symlink: "go two directories up (from /web/themes to /) and then add the path (from / to /system/themes)".

Hm, yes, but in case of the themes, the path generated by Symlinks::symlinkThemes still seems weird to me. Even if you remove the root directory, the code will want to generate a symlink from

../../vendor\contao\core-bundle\src/Resources/contao/themes\flexible

to

system/themes/flexible

That's what I do not understand. The leading ../../ would point outside the working directory, or am I missing something?

Could be this:

Symlinks on windows are created by Symlink() which accept only absolute paths but not relative paths .relative paths on windows are not supported for symlinks

:(

No, I just tried, relative symlinks work, in general. However, the problem is, that all these source paths:

$this->symlink('../assets', 'web/assets', $rootDir, $output);
$this->symlink('../../system/themes', 'web/system/themes', $rootDir, $output);
$this->symlink('../app/logs', 'system/logs', $rootDir, $output);

simply do not exist

[leofeyer]

That's what I do not understand. The leading ../../ would point outside the root directory, or am I missing something?

They are not pointing outside the root directory.

system/themes/
web/system/themes -> ../../system/themes

[fritzmg]

Aaaah, now I get it. Indeed, Windows does not support relative symlinks... well, it does, in a way - but not as expected. I'll try to explain:

If you do this on Linux

$this->symlink('../app/logs', 'system/logs', $rootDir, $output);

it will create a symlink at the location system/logs and the symlink points to ../app/logs relative to that location, as you expect.

If you do the same in Windows, the symlink creation will fail with the message

[Symfony\Component\Filesystem\Exception\IOException]
Failed to create symbolic link from "../app/logs" to "C:\xampp\htdocs\contao4/system/logs".

as already said. But now comes the weird part:

If you do this on Windows:

$this->symlink('../contao4/app/logs', 'system/logs', $rootDir, $output);

(contao4/ is the folder that contains the Contao installation) it will successfully create a symlink at the system/logs location, however, the symlink will be invalid, since it points to ../contao4/app/logs.

o_0

[leofeyer]

That does not really make sense, does it?

[fritzmg]

Yeah, indeed it does not. The reason I tried it at all is that I misunderstood the $source parameter of the SymlinksCommand::symlink function. I thought the path would have to be relative to the current root folder or working directory. That's why I tried ../contao4/app/logs, because that would be a valid location relative to the root directory (../ steps out of the root directory, and contao4/ back in).

I am currently trying to test the symlink function of PHP to see what is going on.

[leofeyer]

Never mind. We just have to use absolute links on Windows.

[fritzmg]

I think I found the problem, within PHP. PHP's symlink function checks if the first parameter ($target i.e. topath in C) does in fact exist: https://github.com/php/php-src/blob/master/ext/standard/link_win32.c#L169

if ((attr = GetFileAttributes(topath)) == INVALID_FILE_ATTRIBUTES) {
        php_error_docref(NULL, E_WARNING, "Could not fetch file information(error %d)", GetLastError());
        RETURN_FALSE;
}

However, the Win32 API function GetFileAttributes expects the path to be either absolute or relative to the working directory. But that's a problem of course if you are using a relative symlink - the symlink will be relative to the $source's directory, not the working directory, and thus this call will always fail with the Could not fetch file information error, when you are using an otherwise valid relative file path.

Relative symlinks would work under Windows in principle - it's just that there is a bug in PHP's symlink function, that prevents this - from my analysis.

But in any case, yes, we have to stick with absolute links on Windows for now...

[leofeyer]

Thanks for the detailed explanation.

[fritzmg]

I've also made a bug report on php.net: https://bugs.php.net/bug.php?id=69473

[leofeyer]

Should be fixed in 77c1c106110aaf75152be5deccdc03f3ea093393. Please test and give feedback :)

[fritzmg]

@leofeyer SymlinksCommand.php#L139 still contains

$path = str_replace("$rootDir/", '', $theme->getPathname());

instead of

$path = str_replace($rootDir . DIRECTORY_SEPARATOR, '', $theme->getPathname());

Thus the theme symlinks still fail on windows:

  [Symfony\Component\Filesystem\Exception\IOException]
  Failed to create symbolic link from "C:\xampp\htdocs\contao4/C:\xampp\htdocs\contao4\vendor\con
  tao\core-bundle\src/Resources/contao/themes\flexible" to "C:\xampp\htdocs\contao4/system/themes
  /flexible".

You don't need DIRECTORY_SEPARATOR for any functions related to the file system, but you still need it for pure string functions like str_replace.

[leofeyer]

Really? I have made the necessary changes in e023e7b9a70a9229296479f62f7d16265283d2b9 and realized that half of them also affect Contao 3. So how does Contao 3 work at all on your machine?

[fritzmg]

I don't know :). Can you give me an example in Contao 3? Then I can check what is going on.

[leofeyer]

Just look at my changes in e023e7b9a70a9229296479f62f7d16265283d2b9. E.g.

• BackendInstall::importExampleWebsite()
• DC_Folder::delete()
• DC_Folder::generateTree()
• Dbafs::addResource()
• Filter\SyncExclude::accept()
• Folder::getHash()
• ModuleSearch::compile()
• FileSelector::renderFiletree()

All of them are using TL_ROOT . '/', so I wonder why it is working.

[fritzmg]

You are right. I have skimmed over the Contao 3 code and there are a few places, where the root directory of the Contao installation + '/' is (supposed to be) removed from an absolute path string - and it does indeed not work on Windows for the same reason. However, none of these instances seem to have any severe effect on the functionality of Contao on Windows. It's mostly used for logging and the like.

One example: functions.php#L64 - on Windows, you always see the full path in the error message - since str_replace(TL_ROOT . '/', '', …) has no effect.

Some extensions are affected as well.

[leofeyer]

Then we should back port the changes to Contao 3. Fixed in 32537a8e84b406dbaed89df15b9d1961bc354f2b.

[leofeyer]

Ok, so this did not work so well: contao/core#7847

I've spent some more time debugging and I found out that we are passing the FilesystemIterator::UNIX_PATHS flag to the directory iterators, therefore everything works fine in Contao 3.5. The Symfony finder, however, does not automatically convert the directory separator (and it cannot be configured to do so either?), therefore we have to use DIRECTORY_SEPARATOR wherever we use the finder.

[fritzmg]

There is a similar problem to these issues in Dbafs::addResource(…). Look at the following code:

// The resource does not exist or lies outside the upload directory
if ($strResource == '' || strncmp($strResource,  $strUploadPath, strlen($strUploadPath)) !== 0 || !file_exists(TL_ROOT . '/' . $strResource))
{
    echo "foo";
    throw new \InvalidArgumentException("Invalid resource $strResource");
}

If $strResource happens to contain backward slashes instead of forward slashes, for example

files\syncCto_backups\database\2015-06-04_1152_DB-Backup.zip

the condition

strncmp($strResource,  $strUploadPath, strlen($strUploadPath)) !== 0

will fail, because it compares files\ with files/.

This problem currently arises in the newest syncCto version (see menatwork/syncCto#236), because syncCto happens to pass the file path to Dbafs::addResource(…) with backward slashes instead of forward slashes.

[leofeyer]

Fixed in 3261c642dcc99c53bca773c8cd3e3d55428da1eb.

[fritzmg]

Will this also be backported to Contao 3.5?

[leofeyer]

Yes.

[leofeyer]

Back-ported in contao/core@d01d94fca33b7582b26c861722aeb2abd72e5f86.

[fritzmg]

This problem is back in Contao 4.0.2, or at least when installing Contao with the new web/install.php script. If you run this new Install Tool under Windows, the following error will occur:

Fatal error: Uncaught exception 'Symfony\Component\Filesystem\Exception\IOException' with message 'Failed to create symbolic link from "C:/xampp/htdocs/contao-4.0.2/C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src/Resources/contao/themes\flexible" to "C:/xampp/htdocs/contao-4.0.2/system/themes/flexible".' in C:\xampp\htdocs\contao-4.0.2\vendor\symfony\symfony\src\Symfony\Component\Filesystem\Filesystem.php:314 
Stack trace: 
#0 C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src\Command\SymlinksCommand.php(168): Symfony\Component\Filesystem\Filesystem->symlink('C:/xampp/htdocs...', 'C:/xampp/htdocs...') 
#1 C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src\Command\SymlinksCommand.php(128): Contao\CoreBundle\Command\SymlinksCommand->symlink('C:\\xampp\\htdocs...', 'system/themes/f...') 
#2 C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src\Command\SymlinksCommand.php(77): Contao\CoreBundle\Command\SymlinksCommand->symlinkThemes() 
#3 C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src\Co in C:\xampp\htdocs\contao-4.0.2\vendor\symfony\symfony\src\Symfony\Component\Filesystem\Filesystem.php on line 314

This time the problem is the other way around, sort of. For instance, in symlinkThemes() the variable $this->rootDir will be

C:/xampp/htdocs/contao-4.0.2

and $theme->getPathname() will be

C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src/Resources/contao/themes\flexible

which will cause

str_replace($this->rootDir . DIRECTORY_SEPARATOR, '', $theme->getPathname());

not to remove the rootDir from the theme's pathname, since it will be

str_replace('C:/xampp/htdocs/contao-4.0.2\\', '', 'C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src/Resources/contao/themes\flexible');

[fritzmg]

just fyi

$this->container->getParameter('kernel.root_dir')

will always return forward slashes: https://github.com/symfony/symfony/blob/v2.7.3/src/Symfony/Component/HttpKernel/Kernel.php#L355

Whereas Symfony\Component\Finder and the following getPathname call, which is used here:

$themes = $this->getContainer()->get('contao.resource_finder')->findIn('themes')->depth(0)->directories();
…
$path = str_replace($this->rootDir . DIRECTORY_SEPARATOR, '', $theme->getPathname());

seems to either return backward or forward slashes, depending on the OS.

Maybe it's best to always normalize paths (i.e. always convert \ to /). Then you could also omit the DIRECTORY_SEPARATOR constant.

[leofeyer]

This seems to be a Symfony issue: https://github.com/symfony/symfony/issues/15474

I'll add a workaround to the installation-bundle.

[leofeyer]

Fixed in contao/standard-edition@30d06354942129250296798bf683903a5b1c35ee.

[fritzmg]

On Windows, the /install.php still generates Fatal Errors because the generated symlinks are not detected as symlinks:

Fatal error: Uncaught exception 'LogicException' with message 'The symlink target "system/themes/flexible" exists and is not a symlink.' in C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src\Command\SymlinksCommand.php:204
Stack trace:
#0 C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src\Command\SymlinksCommand.php(158): Contao\CoreBundle\Command\SymlinksCommand->validateSymlink('vendor\\contao\\c...', 'system/themes/f...')
#1 C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src\Command\SymlinksCommand.php(128): Contao\CoreBundle\Command\SymlinksCommand->symlink('vendor\\contao\\c...', 'system/themes/f...')
#2 C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src\Command\SymlinksCommand.php(77): Contao\CoreBundle\Command\SymlinksCommand->symlinkThemes()
#3 C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src\Command\SymlinksCommand.php(57): Contao\CoreBundle\Command\SymlinksCommand->generateSymlinks()
#4 C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src\Command\AbstractLockedComm in C:\xampp\htdocs\contao-4.0.2\vendor\contao\core-bundle\src\Command\SymlinksCommand.php on line 204

is_link($this->rootDir . '/' . $target)

returns false, even though

C:\xampp\htdocs\contao-4.0.2/system/themes/flexible

is in fact a <SYMLINKD>:

 Directory of C:\xampp\htdocs\contao-4.0.2\system\themes

08.08.2015  23:42    <DIR>          .
08.08.2015  23:42    <DIR>          ..
06.08.2015  00:26                57 .gitignore
08.08.2015  23:42    <SYMLINKD>     flexible [C:\xampp\htdocs\contao-4.0.2/vendor\contao\core-bundle\src/Resources/contao/themes\flexible]
               1 File(s)             57 bytes
               3 Dir(s)  29.926.965.248 bytes free

[leofeyer]

Are you sure? is_link() should recognize <SYMLINKD>. It just cannot handle junctions.

[fritzmg]

I know, but it returns false for these symlinks for some reason.

[leofeyer]

This sounds like a problem on your machine?

[fritzmg]

Well at least someone else is running into the same problem: https://community.contao.org/de/showthread.php?58543-Contao-4-Installation-XAMPP&p=380294&viewfull=1#post380294

I am on vacation right now, so I can't analyse it yet.

[leofeyer]

Which Windows version are you using?

[fritzmg]

Still Windows 7 x64, the system is the same as in the first post in this issue.

Some user on reddit is having the same problem (see #327 above).

[jamesdevine]

I'm also having the same problem on Windows 10 with the latest install tool + Contao 4.0.2 on WAMP, PHP 5.5.12. Exactly as described above.

The symlink looks good:

12/08/2015  17:26    <SYMLINKD>     flexible [F:\Work\Development-sites\contao4/vendor\contao\core-bundle\src/Resources/contao/themes\flexible]

However is_link() is not returning true for the path:

var_dump(is_link('F:/Work/Development-sites/contao4/system/themes/flexible'));

boolean false

Update: infact I am not able to get is_link() to return true for any symlinks including symlinks made using MKLINK.

[leofeyer]

I am trying to debug the issue, but I just don't succeed. The error message is symlink(): Could not fetch file information(error 3). There are other people out there with exactly the same problem:

http://stackoverflow.com/questions/26600247/how-to-make-windows-symlinks-work-with-mod-php

[leofeyer]

There is a similar issue here: https://github.com/contao-community-alliance/composer-plugin/issues/34