Feature: Get absolute/evaluated path from package `resource://`

[mhsdesign]

I wanted to find out the realpath of a resource://Foo.Bar/... path. But the thing is phps realpath is not compatible with custom stream wrappers like resource://. Its just a small wrapper around some c code.

So how about allowing to easily find out the absolute/evaluated ressource path (the path with flow root and co - not necessary the realpath) of such links. Those are then compatible with realpath.

There are also a few questions about this in slack - but no clear flow way solution, and it should be IMO.

Since there is no implementation api to handle this the php way: there is no resolved_path_for_file_stream_wrapper respectively to file_get_contents.

I build a custom solution by copying and pasting some code from the ResourceStreamWrapper: And I think this should land one way or another into the core. Maybe one could call StreamWrapperAdapter::getResolvedPathForFileStreamWrapper() https://github.com/neos/flow-development-collection/blob/c8c3d7e5da927624d13f10f01bf1ec8a5b1f5b9e/Neos.Flow/Classes/ResourceManagement/Streams/StreamWrapperAdapter.php#L28

class ResourceStreamWrapperHelper
{
    /**
     * @Flow\Inject(lazy = false)
     * @var PackageManager
     */
    protected $packageManager;

    public function getAbsolutePath(string $requestedPath): string
    {
        $requestPathParts = explode('://', $requestedPath, 2);
        if ($requestPathParts[0] !== ResourceStreamWrapper::getScheme()) {
            throw new \InvalidArgumentException('The ' . __CLASS__ . ' only supports the \'' . ResourceStreamWrapper::getScheme() . '\' scheme.', 1643540543);
        }

        if (isset($requestPathParts[1]) === false) {
            throw new \InvalidArgumentException('Incomplete $requestedPath', 1643540538);
        }

        $resourceUriWithoutScheme = $requestPathParts[1];

        if (strpos($resourceUriWithoutScheme, '/') === false && preg_match('/^[0-9a-f]{40}$/i', $resourceUriWithoutScheme) === 1) {
            throw new \InvalidArgumentException('Cant process resource', 1643540539);
        }

        list($packageName, $path) = explode('/', $resourceUriWithoutScheme, 2);

        try {
            $package = $this->packageManager->getPackage($packageName);
        } catch (\Neos\Flow\Package\Exception\UnknownPackageException $packageException) {
            throw new \Exception(sprintf('Invalid resource URI "%s": Package "%s" is not available.', $requestedPath, $packageName), 123, $packageException);
        }

        if ($package instanceof FlowPackageInterface === false) {
            return false;
        }

        return Files::concatenatePaths([$package->getResourcesPath(), $path]);
    }
}

mostly extracted logic of: https://github.com/neos/flow-development-collection/blob/2988e50e2c4d3e42e92524cc9e08473852ba9443/Neos.Flow/Classes/ResourceManagement/Streams/ResourceStreamWrapper.php#L490

[mhsdesign]

I could also imagine a custom method in the flow Files utility: https://github.com/neos/flow-development-collection/blob/d559aca053fd64a3b2d8ed3e3c19942f61f3b9ee/Neos.Utility.Files/Classes/Files.php

Files::getResolvedPathForFileStreamWrapper('resource://foo.bar').

[mhsdesign]

maybe one should introduce an additional interface for flow stream wrapper and then let the ResourceStreamWrapperHelper call the getAbsolutePath on these flow stream wrapper ...

[kitsunet]

No, absolute paths are not a thing for resources, the wohle point of resources is to abstract away the filesystem. Which means you cannot expect an absolute path to even exist in case of eg. cloud storage. what would you need it for?

[kitsunet]

I would clsoe this issue in 24h unless thre is a good argument made.

[mhsdesign]

Im not talking about persistent resources but only the local files from a package in the resources folder. (which are not stored externally)

My problem is:

i need to compare filepaths and check if they point to the same file. I cant use the content hash as this could be different over time.

usecase (for the new fusion parser wich uses an internal AST (not the array object tree)):

i want to cache each included fusion file and invalidate the cached ast, when the file has changed. So i want to create a cache entry by filepath.

but the same fusion file might be presented by different filepaths,

eg. in fusion one could include a file via include: resource://My.Site/Private/Fusion/Foo/Bar.fusion but the flow filesHaveChanged will give me an absolute path like absoluteFlowPathRoot/Packages/Sites/My.Package/Resources/Private/Fusion/Foo/Bar.fusion

all point to the same file:

• resource://My.Site/Private/Fusion/Foo/Bar.fusion
• resource://My.Site/Private/Fusion/Foo/Subfolder/Nested/../../Bar.fusion
• absoluteFlowPathRoot/Packages/Sites/My.Package/Resources/Private/Fusion/Foo/Bar.fusion

or they do too:

• resource://My.Site/../NodeTypes/Image/Buz.fusion
• absoluteFlowPathRoot/Packages/Sites/My.Package/NodeTypes/Image/Buz.fusion

so i would need to resolve them to a common base to use them as cache indentifier.

realpath would take care of resolving ../ and so on but it doesnt work on ressource stream wrappers. Thats why i need this resolve resource wrapper functionality

[mhsdesign]

in slack there are many issues about trying to use realpath() on a package resource path (what doesnt work because realpath doesnt know about stream wrapper) ... (search for realpath resource)

one could also rebuild it much smaller like @bwaidelich did: https://neos-project.slack.com/archives/C04V4C6B0/p1516725927000381 https://github.com/bwaidelich/Wwwision.GraphQL/blob/e37443915bd08af976b1d4a17f1d6775fb740fc5/Classes/Package.php#L50

or even better:

[$packageKey, $relativePathAndFilename] = $this->resourceManager->getPackageAndPathByPublicPath($resourceUri);
$package = $packageManager->getPackage($packageKey);

return realpath(Files::concatenatePaths([$package->getResourcesPath(), $relativePathAndFilename]));

maybe put this into a flow Files function?

[kitsunet]

Ok understood now, that does make sense I guess. We just need to find a way to make that work in various environments and without too many assumptions. I guess what happens in \Neos\Flow\ResourceManagement\Streams\ResourceStreamWrapper::evaluateResourcePath is what we would need, BUT $package->getResourcesPath() is on my internal deprecation/make better list since ages. it assumes a whole lot, as we just iterate the Packages folder and decide any place we find a composer.json in is a package and that's the path to it. Which works well enough as we can see, but I am not super happy about it. Either way removing that takes a lot more effort anyways, so implementing this new functionality based on that now should be fine...

[mhsdesign]

then its even better to abstract it away to not have the need to call $package->getResourcesPath() directly ...

maybe we should put it on the $resourceManager as here lives already the functionality getPackageAndPathByPublicPath?https://github.com/neos/flow-development-collection/blob/24297f3ab69357855d243a52a16bc0e06f725720/Neos.Flow/Classes/ResourceManagement/ResourceManager.php#L456

something like would be nice: but maybe its not the $resourceManagers task to do that but more the streamwrappers task ...

realpath($resourceManager->getAbsolutePackageResourcePath('resource://Your.Package/Private/Foo/Bar.html'));

(i guess whatever we come up with it wouldnt have helped @bwaidelich since he used it at a really early stage where no di is available, maybe one would need a method to injectPackageManager on the object which implements this functionality.)

[mhsdesign]

i now build this myself (for neos.fusion), making a feature out of this still makes sense - its just hard to get right. https://github.com/neos/neos-development-collection/pull/3659/files#diff-2bddda138583e813fb9494e8dd1a33a0749c9c7ad85b2764b340225299ad5ec4R132

[fcool]

A major idea to make it "easier" to allow realpath, etc. in some cases would be to drop "Package" - Resources. Maybe through introduction of another resource wrapper. Because these "resources" will always be accessable - which is fundamentely different from PersistentResources

[mhsdesign]

yes i discussed somewhere a package:// resource wrapper, which can be used to acess the Resource folder too https://github.com/neos/neos-development-collection/issues/3507#issuecomment-1025468376

[mhsdesign]

with https://github.com/neos/neos-development-collection/pull/3903 we now have a second stream wrapper:

nodetypes:// which we also need to reverse externally for caching purposes. see https://github.com/neos/neos-development-collection/pull/3903/files#diff-2bddda138583e813fb9494e8dd1a33a0749c9c7ad85b2764b340225299ad5ec4R112-R134

its time to introduce a flow way of reversing streamwrappers i think.

[jonnitto]

@mhsdesign I think we could close this one, right?