filesize remains pending after marie accepts the federated shared file.

[navid-shokri]

when Marie accepts the shared folder/file the pending file remains beside it.

[thepeak99]

Blocked by #194 and #193

[navid-shokri]

:information_source: So, I could not reproduce the #194 , #193 but when I shared my file with the recipient group and group member accepted the share I was facing with this error when I was trying to open the file. I guess it can help you.

{"reqId":"5a7AEXWHMwgAGAe99L4R","level":3,"time":"2023-06-14T10:53:24+00:00","remoteAddr":"172.18.0.6","user":"marie","app":"PHP","method":"POST","url":"\/index.php\/apps\/files_sharing\/api\/externalShares","message":"Undefined index: password at \/var\/www\/html\/apps\/files_sharing\/lib\/External\/Storage.php#92"}

Also, the pending flag is shown just for the folder and the files can indicate their size as you can see below image:

[yasharpm]

After researching #68 it is clear to me that the file size remains pending if the webda PROPFIND call to the share provider fails. So this is most likely same as #169

[yasharpm]

After a debugging session with @thepeak99 here is our findings:

• This problem exists independent from our code. There is something wrong with our dev environment setup but we need to know what it is in order to make sure it won't happen in a real world setup and also to be able to test our code properly.
• All webdav calls result in connection refused error.
• But the provider actually receives the call and returns the expected result.
• If we make the same call that OC makes to webdav using curl, we get the response no problem.
• If the call is modified to use http instead of https, it works fine.

The piece of code we were specifically starting at is this.

[yasharpm]

Our setup has no problem with webdav on NextCloud. I am comparing what is different between the 2 regarding the use of webdav:

• OC uses php version 7.4 but NC uses php version 8.0 . Our dev environment (on the oc-opencloudmesh-testing branch that we are currently using) uses php version 7.4 for OC and php version 8.0 for NC.
• OC uses guzzlehttp/guzzle version ^7.5 in composer.json and version 7.5.0 in composer.lock. NC uses guzzlehttp/guzzle version ^7.5.0 in composer.json and ^6.5.8 || ^7.4.5 in composer.lock
• The code that gets the shared file content is identical for both OC and NC.
• The ClientService that both OC and NC use for opening connections are functionally identical. Both of them make use of a CertificateManager which comparing the code of the two under the path lib/private/Security/CertificateManager.php seems like they basically do the same thing.

[yasharpm]

I tried to run OC on php 8.0 by changing the docker file header to FROM apache-php-8.0 but I got this error:

This version of ownCloud is not compatible with PHP 8.0
You are currently running PHP 8.1.2-1ubuntu2.11.

[yasharpm]

OwnCloud installation manual states here:

Ubuntu 22.04 only provides PHP 8.x which is currently not supported by ownCloud. Therefore PHP 7.4 is installed via PPA. 

The greatest difference between OC and NC that I detected about the webdav was the php version. :-(

[yasharpm]

Trying to inspect the source of connection refused exception in the guzzlehttp library.

The stack trace for connection refused up to the point the stack reaches OC is as follows:

#0 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/Handler\\\/Proxy.php(48): GuzzleHttp\\\\Handler\\\\StreamHandler->__invoke()
#1 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/PrepareBodyMiddleware.php(35): GuzzleHttp\\\\Handler\\\\Proxy::GuzzleHttp\\\\Handler\\\\{closure}(*** sensitive parameters replaced ***)
#2 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/Middleware.php(31): GuzzleHttp\\\\PrepareBodyMiddleware->__invoke()
#3 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/RedirectMiddleware.php(71): GuzzleHttp\\\\Middleware::GuzzleHttp\\\\{closure}(*** sensitive parameters replaced ***)
#4 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/Middleware.php(63): GuzzleHttp\\\\RedirectMiddleware->__invoke()
#5 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/HandlerStack.php(75): GuzzleHttp\\\\Middleware::GuzzleHttp\\\\{closure}(*** sensitive parameters replaced ***)
#6 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/Client.php(331): GuzzleHttp\\\\HandlerStack->__invoke()
#7 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/Client.php(168): GuzzleHttp\\\\Client->transfer()
#8 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/Client.php(187): GuzzleHttp\\\\Client->requestAsync()
#9 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/ClientTrait.php(44): GuzzleHttp\\\\Client->request()
#10 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Http\\\/Client\\\/Client.php(184): GuzzleHttp\\\\Client->get()

[yasharpm]

Local path inside the docker image to the guzzlehttp source is: /var/www/html/lib/composer/guzzlehttp/guzzle/src

The github repository for guzzlehttp version 7.5.1 that is present in composer.lock of the container is: https://github.com/guzzle/guzzle/tree/b964ca597e86b752cd994f27293e9fa6b6a95ed9/src

[yasharpm]

The first trace Handler/Proxy.php(48) is ambiguous as it is a callable call to a closure which we don't know what it is. I put a log on the call that creates the closure that makes the close on Proxy.php(47) to print the stack trace and figure out what is being called. It is quite a wraps happening but from the logs, I can tell the first call in the queue is failing so fingers crossed the failing first call is calling wrap here:

called wrap here: 
#0 /var/www/html/lib/composer/guzzlehttp/guzzle/src/Utils.php(102): GuzzleHttp\\Handler\\Proxy::wrapStreaming()\n
#1 /var/www/html/lib/composer/guzzlehttp/guzzle/src/HandlerStack.php(49): GuzzleHttp\\Utils::chooseHandler()\n
#2 /var/www/html/lib/composer/guzzlehttp/guzzle/src/Client.php(60): GuzzleHttp\\HandlerStack::create()\n
#3 /var/www/html/lib/private/Http/Client/ClientService.php(56): GuzzleHttp\\Client->__construct()\n
#4 /var/www/html/apps/federatedfilesharing/lib/DiscoveryManager.php(57): OC\\Http\\Client\\ClientService->newClient()\n
#5 /var/www/html/apps/federatedfilesharing/lib/AppInfo/Application.php(210): OCA\\FederatedFileSharing\\DiscoveryManager->__construct()\n
#6 /var/www/html/apps/federatedfilesharing/lib/AppInfo/Application.php(195): OCA\\FederatedFileSharing\\AppInfo\\Application->initFederatedShareProvider()\n
#7 /var/www/html/apps/federatedfilesharing/lib/AppInfo/Application.php(281): OCA\\FederatedFileSharing\\AppInfo\\Application->getFederatedShareProvider()\n
#8 /var/www/html/apps/federatedfilesharing/appinfo/app.php(41): OCA\\FederatedFileSharing\\AppInfo\\Application->registerListeners()\n
#9 /var/www/html/lib/private/legacy/app.php(253): require_once('/var/www/html/a...')\n
#10 /var/www/html/lib/private/legacy/app.php(192): OC_App::requireAppFile()\n
#11 /var/www/html/lib/private/legacy/app.php(125): OC_App::loadApp()\n
#12 /var/www/html/lib/private/legacy/util.php(148): OC_App::loadApps()\n
#13 /var/www/html/lib/private/Files/Filesystem.php(262): OC_Util::setupFS()\n
#14 /var/www/html/apps/oc-opencloudmesh/opencloudmesh/lib/Files_Sharing/Hooks.php(52): OC\\Files\\Filesystem::getMountManager()\n
#15 /var/www/html/apps/oc-opencloudmesh/opencloudmesh/lib/AppInfo/Application.php(210): OCA\\OpenCloudMesh\\Files_Sharing\\Hooks->__construct()\n
#16 /var/www/html/lib/composer/pimple/pimple/src/Pimple/Container.php(122): OCA\\OpenCloudMesh\\AppInfo\\Application->OCA\\OpenCloudMesh\\AppInfo\\{closure}()\n
#17 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(107): Pimple\\Container->offsetGet()\n
#18 /var/www/html/apps/oc-opencloudmesh/opencloudmesh/lib/AppInfo/Application.php(344): OC\\AppFramework\\Utility\\SimpleContainer->query()\n
#19 /var/www/html/apps/oc-opencloudmesh/opencloudmesh/appinfo/app.php(27): OCA\\OpenCloudMesh\\AppInfo\\Application->registerEvents()\n
#20 /var/www/html/lib/private/legacy/app.php(253): require_once('/var/www/html/a...')\n
#21 /var/www/html/lib/private/legacy/app.php(192): OC_App::requireAppFile()\n
#22 /var/www/html/lib/private/legacy/app.php(125): OC_App::loadApp()\n
#23 /var/www/html/lib/base.php(883): OC_App::loadApps()\n
#24 /var/www/html/index.php(54): OC::handleRequest()\n
#25 {main}

[michielbdejong]

I could not reproduce this with latest dev-stock main branch.

Be careful! There is the word 'pending' in the 'file size' column! Don't be distracted by that :)

Please reopen if you have tested this on the latest dev-stock main branch and you still see it happening there

[michielbdejong]

ah sorry, after reading the comments I realised that this issue is actually about the filesize! :) Confirmed that this is still happening, both for ocm-to-user and ocm-to-group, in dev-stock main branch:

[yasharpm]

The checkRedirect function in the RedirectMiddleware class seems to be the source of our problems. I could verify that the receiver's get call to the provider is successful and gets a response. This response then gets further processed by the redirect middle and somehow it ends up with a connection refused error. The thread enters the function but it never leaves it!

Here is the function code: https://github.com/guzzle/guzzle/blob/b964ca597e86b752cd994f27293e9fa6b6a95ed9/src/RedirectMiddleware.php#L80

[yasharpm]

I was wrong. The thread safely exits the redirect middleware.

[yasharpm]

My best guess atm is that the await() function here throws the exception.

[yasharpm]

My guess seems to be correct. In which case the await call on the FulfilledPromise class throws an exception the second(?) time the code tries to send a request.

FulfilledPromise.php is located at /var/www/html/lib/composer/guzzlehttp/promises/src on the local server and https://github.com/guzzle/promises/blob/2.0/src/FulfilledPromise.php on Github.

[yasharpm]

A FullfilledPromise does nothing in the wait function. It already has the value and just returns it.

[yasharpm]

I confirmed. Upon refreshing the page to get the received file content. The first call is GET https://oc1.docker/ocm-provider/ which is successful and the second call is GET https://oc1.docker/public.php/webdav/ which returns a RejectedPromise and throws the exception on wait.

[yasharpm]

Finally closed down on the culprit: https://github.com/guzzle/guzzle/blob/b964ca597e86b752cd994f27293e9fa6b6a95ed9/src/Handler/StreamHandler.php#L328

[yasharpm]

This means that the root of the issue is:

$context = [
    "http" =>
        [
            "method" => "GET",
            "header" => "Authorization: Basic UzZxd0MwdDdPYVpNRU12Og==\\r\\nHost: oc1.docker\\r\\nUser-Agent: ownCloud Server Crawler\\r\\nContent-Length: 0\\r\\nConnection: close",
            "protocol_version" => "1.1",
            "ignore_errors" => true,
            "follow_location" => 0
        ],
    "ssl" =>
        [
            "peer_name" => "oc1.docker",
            "cafile" => "\\/var\\/www\\/html\\/data\\/files_external\\/rootcerts.crt",
            "verify_peer" => true,
            "verify_peer_name" => true,
            "allow_self_signed" => false
        ]
];
$params = [];
$contextResource = \stream_context_create($context, $params);
return @\fopen('https://oc1.docker/public.php/webdav/', 'r', false, $contextResource);

This returns false while it shouldn't. Going to verify.

[yasharpm]

Progress! Setting cafile config to point to /etc/ssl/certs/ca-certificates.crt fixes the issue. Next is to figure out where the current config comes from and how to change it.

[yasharpm]

So adding

$context['ssl']['cafile'] = '/etc/ssl/certs/ca-certificates.crt';

to line 318 of /Handler/StreamHandler.php fixes the issue.

[yasharpm]

Here is where in OC code that the ca file option is being set:

        if ($this->certificateManager->listCertificates() !== []) {
            $options[RequestOptions::VERIFY] = $this->certificateManager->getAbsoluteBundlePath();
        } else {
            // If the instance is not yet setup we need to use the static path as
            // $this->certificateManager->getAbsoluteBundlePath() tries to instantiate
            // a view
            if ($this->config->getSystemValue('installed', false) && !\OCP\Util::needUpgrade()) {
                $options[RequestOptions::VERIFY] = $this->certificateManager->getAbsoluteBundlePath(null);
            } else {
                $options[RequestOptions::VERIFY] = \OC::$SERVERROOT . '/resources/config/ca-bundle.crt';
            }
        }

[yasharpm]

The certificate manager is located at lib/private/Security/CertificateManager.php. It gathers certificates from the /files_external/uploads/ folder and /var/www/html/resources/config/ca-bundle.crt and /files_external/rootcerts.crt files.

[yasharpm]

I fixed it by appending the contents of /etc/ssl/certs/ca-certificates.crt to /var/www/html/resources/config/ca-bundle.crt inside the docker image on dev-stock in this commit.

[michielbdejong]

Apparently this is happening in production now:

[yasharpm]

I was not able to reproduce this on our test environment. However what got my attention is that apparently the share provider is not actually sharing. From my understanding, the left image is a screenshot from the provider and the right one is from the receiver. But this is how the share provider should see their shared files: Notice the "shared" label next to the share icon. This symptom is probably has been caused by something far different from what we resolved on this issue. My guess is that our SRAM controller is calling the wrong functions to create a share. So the OCM notifications are being send to the receiver but share rows are not being created on the provider side. This is highly unusual. Maybe is it another case of master slave database architecture that SURF is using on their OwnCloud servers.

@michielbdejong the following questions are vital to resolving this issue:

1. The share label the receiver side says "tomw@...". Is it a user share?
2. Is the share created via the SRAM?
3. I could not reproduce this error. Is it possible to wait for SURF to provide their test environment to us first?

[michielbdejong]

Thanks! That helps, I'll talk to Tom!

[michielbdejong]

The share label the receiver side says "tomw@...". Is it a user share?

That would be the sender, right? In our dev env it would usually say "einstein@oc1.docker" there when Maria views it, right?

[yasharpm]

Correct!