Files located on files_external mounted as Nextcloud or Webdav will not extract

[bluesky-ca]

I tried to extract the same file in 5 locations

• local disk: OK
• External Files - sftp: OK
• External Files - Local ( remote disk mounted using fuse.sshfs ): OK ( best performance for remote systems )
• External Files - Webdav: FAIL, top level folder created, no file inside
• External Files - Nextcloud: FAIL, top level folder created, no file inside

If indeed it is not possible to extract files on some External Mounts like S3 Webdav, etc it would be good to do a check ( if the API allows for that ) and return an error to the user - even better the ... context menu for the archive should not show 'Extract Here' option if the file system is not supported or should give the user an option to place the files in another location - this would be good, as it would work with R/O mounted file systems.

Unrelated to this problem I have been doing performance benchmarks on NC external file systems, and just looking at the depth of the call stack tells us why some of the External Mounts are as slow as they are - again nothing to do with this app, and thanks to Paul for creating and maintaining it !!!

For those interested to copy the same file using fuse.sshfs took 7 sec. vs 35 sec. for sftp which in theory is the same protocol - of course one uses a binary (fuse.sshfs) the other PHP library to access the files. Pure scp took 5.679 sec.

System details: NC v21.0.4 php 7.3.29 Apache 2.4 CentOS 7

[no app in context] Warning: GuzzleHttp\Exception\ConnectException: cURL error 28: Operation timed out after 30000 milliseconds with 0 out of -1 bytes received (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://host.example.com/remote.php/webdav/junk2/junk2 at <>

0. /var/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php line 158 GuzzleHttp\Handler\CurlFactory::createRejection(GuzzleHttp\Handl ... l}, {0: "And 25 more ... l})
1. /var/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php line 110 GuzzleHttp\Handler\CurlFactory::finishError(GuzzleHttp\Handler\CurlHandler {}, GuzzleHttp\Handl ... l}, GuzzleHttp\Handler\CurlFactory {})
2. /var/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlHandler.php line 47 GuzzleHttp\Handler\CurlFactory::finish(GuzzleHttp\Handler\CurlHandler {}, GuzzleHttp\Handl ... l}, GuzzleHttp\Handler\CurlFactory {})
3. /var/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/Proxy.php line 28 GuzzleHttp\Handler\CurlHandler->__invoke(" sensitive parameter replaced ", " sensitive parameter replaced ")
4. /var/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/Proxy.php line 48 GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler{closure}(" sensitive parameters replaced ")
5. /var/nextcloud/3rdparty/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php line 64 GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler{closure}(" sensitive parameters replaced ")
6. /var/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php line 31 GuzzleHttp\PrepareBodyMiddleware->__invoke(" sensitive parameter replaced ", " sensitive parameter replaced ")
7. /var/nextcloud/3rdparty/guzzlehttp/guzzle/src/RedirectMiddleware.php line 71 GuzzleHttp\Middleware::GuzzleHttp{closure}(" sensitive parameters replaced ")
8. /var/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php line 63 GuzzleHttp\RedirectMiddleware->__invoke(" sensitive parameter replaced ", " sensitive parameter replaced ")
9. /var/nextcloud/3rdparty/guzzlehttp/guzzle/src/HandlerStack.php line 75 GuzzleHttp\Middleware::GuzzleHttp{closure}(" sensitive parameters replaced ")
   10. /var/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php line 331 GuzzleHttp\HandlerStack->__invoke(" sensitive parameter replaced ", " sensitive parameter replaced ")
   11. /var/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php line 168 GuzzleHttp\Client->transfer(" sensitive parameter replaced ", " sensitive parameter replaced ")
   12. /var/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php line 187 GuzzleHttp\Client->requestAsync("put", GuzzleHttp\Psr7\Uri {}, {0: "And 7 more ... e})
   13. /var/nextcloud/lib/private/Http/Client/Client.php line 342 GuzzleHttp\Client->request("put", "https://host.e ... 2", {0: "And 2 more ... ]})
   14. /var/nextcloud/lib/private/Files/Storage/DAV.php line 516 OC\Http\Client\Client->put("https://host.e ... 2", {body: null,auth: ["USER","XXXXXXXXXXXXXXXX"]})
   15. /var/nextcloud/lib/private/Files/Storage/DAV.php line 423 OC\Files\Storage\DAV->uploadFile("/tmp/oc_tmp_QUDtuk", "junk2/junk2")
   16. /var/nextcloud/lib/private/Files/Storage/DAV.php line 414 OC\Files\Storage\DAV->writeBack("/tmp/oc_tmp_QUDtuk", "junk2/junk2")
   17. <> OC\Files\Storage\DAV->OC\Files\Storage{closure}(" sensitive parameters replaced ")
   18. /var/nextcloud/apps/files_external/3rdparty/icewind/streams/src/CallbackWrapper.php line 119 call_user_func(Closure {})
   19. <> Icewind\Streams\CallbackWrapper->stream_close()
   20. /var/nextcloud/lib/private/Files/Storage/Common.php line 871 fclose(null)
   21. /var/nextcloud/lib/private/Files/Storage/Common.php line 626 OC\Files\Storage\Common->writeStream("junk2/junk2", null)
   22. /var/nextcloud/lib/private/Files/Storage/Common.php line 617 OC\Files\Storage\Common->copyFromStorage(OCA\Files_Trashb ... l}, "files/extract_tmp/junk2/junk2", "junk2/junk2")
   23. /var/nextcloud/lib/private/Files/Storage/Common.php line 689 OC\Files\Storage\Common->copyFromStorage(OCA\Files_Trashb ... l}, "files/extract_tmp/junk2", "junk2", true)
   24. /var/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php line 577 OC\Files\Storage\Common->moveFromStorage(OCA\Files_Trashb ... l}, "files/extract_tmp/junk2", "junk2")
   25. /var/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php line 577 OC\Files\Storage\Wrapper\Wrapper->moveFromStorage(OCA\Files_Trashb ... l}, "files/extract_tmp/junk2", "junk2")
   26. /var/nextcloud/lib/private/Files/Storage/Wrapper/Availability.php line 434 OC\Files\Storage\Wrapper\Wrapper->moveFromStorage(OCA\Files_Trashb ... l}, "files/extract_tmp/junk2", "junk2")
   27. /var/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php line 577 OC\Files\Storage\Wrapper\Availability->moveFromStorage(OCA\Files_Trashb ... l}, "files/extract_tmp/junk2", "junk2")
   28. /var/nextcloud/apps/files_trashbin/lib/Storage.php line 240 OC\Files\Storage\Wrapper\Wrapper->moveFromStorage(OCA\Files_Trashb ... l}, "files/extract_tmp/junk2", "junk2")
   29. /var/nextcloud/lib/private/Files/View.php line 833 OCA\Files_Trashbin\Storage->moveFromStorage(OCA\Files_Trashb ... l}, "files/extract_tmp/junk2", "junk2")
   30. /var/nextcloud/lib/private/Files/Filesystem.php line 728 OC\Files\View->rename("/extract_tmp/junk2", "/Storage/Nextcloud/junk2")
   31. /var/nextcloud/apps/extract/lib/Controller/ExtractionController.php line 169 OC\Files\Filesystem::rename("/extract_tmp/junk2", "/Storage/Nextcloud/junk2")
   32. /var/nextcloud/apps/extract/lib/Controller/ExtractionController.php line 104 OCA\Extract\Controller\ExtractionController->postExtract("junk2", "/Storage/Nextcloud", "/extract_tmp/junk2", "1")
   33. /var/nextcloud/lib/private/AppFramework/Http/Dispatcher.php line 218 OCA\Extract\Controller\ExtractionController->extract("junk2.gz", "/Storage/Nextcloud", "1", "other")
   34. /var/nextcloud/lib/private/AppFramework/Http/Dispatcher.php line 127 OC\AppFramework\Http\Dispatcher->executeController(OCA\Extract\Cont ... {}, "extract")
   35. /var/nextcloud/lib/private/AppFramework/App.php line 157 OC\AppFramework\Http\Dispatcher->dispatch(OCA\Extract\Cont ... {}, "extract")
   36. /var/nextcloud/lib/private/Route/Router.php line 302 OC\AppFramework\App::main("OCA\Extract\C ... r", "extract", OC\AppFramework\ ... {}, {_route: "extract.extraction.extract"})
   37. /var/nextcloud/lib/base.php line 993 OC\Route\Router->match("/apps/extract/ajax/extract.php")
   38. /var/nextcloud/index.php line 37 OC::handleRequest()

POST /index.php/apps/extract/ajax/extract.php

[bluesky-ca]

I have done more testing, and looks to me like the problem is with the files_external app for webdav/Nextcloud connections . I will look some more to understand the issue - here is the log in case somebody spots something interesting - for now I will keep the issue open until further confirmation. The log is from trying to upload the file from my desktop, to local nextcloud that mounts the remote nextcloud using different methods - see the original post for all the methods. If I upload the file directly to the remote site, it works fine, but uploading to anything mounted using http protocol seems to have an issue.

[PHP] Error: Error: Cannot modify header information - headers already sent by (output started at /var/nextcloud/3rdparty/sabre/http/lib/Sapi.php:132) at /var/nextcloud/apps/dav/lib/Connector/Sabre/File.php#691 at <>

0. <> OC\Log\ErrorHandler::onError(2, "Cannot modify h ... )", "/var/nextcloud/ ... p", 691, {string: "X-Hash ... "})
1. /var/nextcloud/apps/dav/lib/Connector/Sabre/File.php line 691 header("X-Hash-MD5: a46 ... 0")
2. /var/nextcloud/apps/dav/lib/Connector/Sabre/File.php line 189 OCA\DAV\Connector\Sabre\File->header("X-Hash-MD5: a46 ... 0")
3. <> OCA\DAV\Connector\Sabre\File->OCA\DAV\Connector\Sabre{closure}(" sensitive parameters replaced ")
4. /var/nextcloud/lib/private/Files/Stream/HashWrapper.php line 71 call_user_func(Closure {}, " sensitive parameter replaced ")
5. <> OC\Files\Stream\HashWrapper->stream_close()
6. /var/nextcloud/apps/files_external/3rdparty/icewind/streams/src/Wrapper.php line 95 fclose(null)
7. /var/nextcloud/lib/private/Files/Stream/HashWrapper.php line 75 Icewind\Streams\Wrapper->stream_close()
8. <> OC\Files\Stream\HashWrapper->stream_close()
9. /var/nextcloud/apps/files_external/3rdparty/icewind/streams/src/Wrapper.php line 95 fclose(null)
   10. /var/nextcloud/lib/private/Files/Stream/HashWrapper.php line 75 Icewind\Streams\Wrapper->stream_close()
   11. <> OC\Files\Stream\HashWrapper->stream_close()
   12. /var/nextcloud/apps/files_external/3rdparty/icewind/streams/src/Wrapper.php line 95 fclose(null)
   13. /var/nextcloud/apps/files_external/3rdparty/icewind/streams/src/CallbackWrapper.php line 117 Icewind\Streams\Wrapper->stream_close()
   14. /var/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 321 Icewind\Streams\CallbackWrapper->stream_close()
   15. /var/nextcloud/apps/dav/appinfo/v1/webdav.php line 84 Sabre\DAV\Server->exec()
   16. /var/nextcloud/remote.php line 167 require_once("/var/nextcloud/ ... p")

PUT /remote.php/webdav/Storage/Nextcloud/junk2.gz

[bluesky-ca]

I have satisfied myself that the issue is in another part of NC - when I upload small files, I can uncompress them on webdav or Nextcloud mounted shares - my original test file was 98MB and somehow something is coming into play - since I can upload the file fine to either Nextcloud site directly it should not be upload limits - I am using php-fpm on my local site ( the one that does the remote mount ) and regular php on the target system ( the one that exports its local disk as Webdav or Nextcloud ).