Open tuxArg opened 1 month ago
So ... I suppose then that it is something with wrong or better unexpected mime-types.
I would suggest to start with uncompressed tar archives, although both -- .tar.[ADDITIONAL EXTENSION]
and .tar
should work. I have attached two screenshots with the details view of a compressed and an uncompressed tar archive.
First: do you have that archive information in the details tab at all? If so, what is the mime type and archive format, and which archive backend is in use? Please note that .tar.xz
is not a "mime-extension", this is just a compressed format .xz
which is "correct".
If there is no archive tab in the details view, then the issue is that the mime type is not accepted by files_archive. Then it would probably good to augment the settings sections by the supported mime types and archive backends to ease debugging (would probably a good idea anyway).
Here's the screenshot of a zip file that works:
And a tar file that doesn't:
When I click on the Archive tab there's a 400 http code response. URL: /apps/files_archive/archive/info/%252FDocuments%252FnoBackup%252FtestFiles%252Ftest.tar Here's the output:
{
"messages": [
"Error: Entry xt does not exist"
],
"archiveStatus": 0,
"archiveInfo": {
"defaultMountPoint": "test",
"defaultTargetBaseName": "test"
}
}
Well, this test.tar
, does it contain an archive member name xt
?
Well, this
test.tar
, does it contain an archive member namext
?
No, Here I attach it. (unfortunately github doesn't allow tar files, so it's zipped, inside there's the test.tar file) test.tar.zip
Perhaps the xt
is some spurious remnant from .txt
... anyhow. I will now first inspect the tar file and then we will see. Are there any other apps which use pear/archive_tar? Could perhaps be checked with the Unix find
command in the apps folder ...
Perhaps the
xt
is some spurious remnant from.txt
... anyhow.
Indeed! I've tried with a test.php.tar, and the message was "Entry hp does not exist".
Are there any other apps which use pear/archive_tar? Could perhaps be checked with the Unix
find
command in the apps folder ...
I don't think so:
me@nextcloud:/var/www/nextcloud/apps$ find ./ -name archive_tar
./files_archive/vendor/pear/archive_tar
I am trying to find the piece of code stating 'Entry xt does not exist' but am not able to find it. The exception should also be logged to the Nextcloud log, can you please look it up? I am not sure were this exception is thrown . Thanks.
I am trying to find the piece of code stating 'Entry xt does not exist' but am not able to find it. The exception should also be logged to the Nextcloud log, can you please look it up? I am not sure were this exception is thrown . Thanks.
Please see my previous message, the xt
is a substring of test.txt
.
Here it is in the logs: (note that /var/www/nextcloud/apps is a symlink to /mnt/nc_data/apps)
{
"reqId": "zkpBqJn5SeSA73NdhFCF",
"level": 3,
"time": "2024-05-27T12:15:27 00:00",
"remoteAddr": "10.8.0.100",
"user": "testUser",
"app": "files_archive",
"method": "POST",
"url": "/index.phps_archive/archive/info/%2FDocuments%2FnoBackup%2FtestFiles%2Ftest.php.tar",
"message": "/mnt/nc_data/apps/files_archive/lib/Controller/ArchiveController.php:157: OCA\\FilesArchive\\Controller\\ArchiveController::info: Caught an Exception",
"userAgent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36",
"version": "29.0.1.1",
"exception": {
"Exception": "BadMethodCallException",
"Message": "Entry hp does not exist",
"Code": 0,
"Trace": [
{
"file": "/mnt/nc_data/apps/files_archive/vendor/wapmorgan/unified-archive/src/Drivers/TarByPhar.php",
"line": 181,
"function": "offsetGet",
"class": "PharData",
"type": "->"
},
{
"file": "/mnt/nc_data/apps/files_archive/vendor/wapmorgan/unified-archive/src/UnifiedArchive.php",
"line": 498,
"function": "getFileData",
"class": "wapmorgan\\UnifiedArchive\\Drivers\\TarByPhar",
"type": "->"
},
{
"file": "/mnt/nc_data/apps/files_archive/lib/Toolkit/Service/ArchiveService.php",
"line": 378,
"function": "getFileData",
"class": "wapmorgan\\UnifiedArchive\\UnifiedArchive",
"type": "->"
},
{
"file": "/mnt/nc_data/apps/files_archive/lib/Toolkit/Service/ArchiveService.php",
"line": 365,
"function": "getFiles",
"class": "OCA\\FilesArchive\\Toolkit\\Service\\ArchiveService",
"type": "->"
},
{
"file": "/mnt/nc_data/apps/files_archive/lib/Toolkit/Service/ArchiveService.php",
"line": 329,
"function": "getCommonDirectoryPrefix",
"class": "OCA\\FilesArchive\\Toolkit\\Service\\ArchiveService",
"type": "->"
},
{
"file": "/mnt/nc_data/apps/files_archive/lib/Toolkit/Service/ArchiveService.php",
"line": 286,
"function": "getArchiveInfo",
"class": "OCA\\FilesArchive\\Toolkit\\Service\\ArchiveService",
"type": "->"
},
{
"file": "/mnt/nc_data/apps/files_archive/lib/Controller/ArchiveController.php",
"line": 146,
"function": "open",
"class": "OCA\\FilesArchive\\Toolkit\\Service\\ArchiveService",
"type": "->"
},
{
"file": "/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php",
"line": 232,
"function": "info",
"class": "OCA\\FilesArchive\\Controller\\ArchiveController",
"type": "->"
},
{
"file": "/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php",
"line": 138,
"function": "executeController",
"class": "OC\\AppFramework\\Http\\Dispatcher",
"type": "->"
},
{
"file": "/var/www/nextcloud/lib/private/AppFramework/App.php",
"line": 184,
"function": "dispatch",
"class": "OC\\AppFramework\\Http\\Dispatcher",
"type": "->"
},
{
"file": "/var/www/nextcloud/lib/private/Route/Router.php",
"line": 338,
"function": "main",
"class": "OC\\AppFramework\\App",
"type": "::"
},
{
"file": "/var/www/nextcloud/lib/base.php",
"line": 1050,
"function": "match",
"class": "OC\\Route\\Router",
"type": "->"
},
{
"file": "/var/www/nextcloud/index.php",
"line": 49,
"function": "handleRequest",
"class": "OC",
"type": "::"
}
],
"File": "/mnt/nc_data/apps/files_archive/vendor/wapmorgan/unified-archive/src/Drivers/TarByPhar.php",
"Line": 181,
"message": "/mnt/nc_data/apps/files_archive/lib/Controller/ArchiveController.php:157: OCA\\FilesArchive\\Controller\\ArchiveController::info: Caught an Exception",
"exception": {},
"CustomMessage": "/mnt/nc_data/apps/files_archive/lib/Controller/ArchiveController.php:157: OCA\\FilesArchive\\Controller\\ArchiveController::info: Caught an Exception"
}
}
Oh well. It is TarByPhar
, not TarByPear
(so the requested find of tar_archive was unnecessary, sorry for that).
However, this means the error messages is generated by the phar extension.
Ok. The way I would go would be to edit that TarByPhar.php and throw out log messages in order to check where things break. E.g. in line 181 it would be interesting to know whether the there mentioned parametere $fileName
is already truncated. Same for ArchiveService::getFiles() and so on.
Is the data directory also a symlink which maybe confuses the phar extension?
Yes. That is the problem. Or the cause of the manifestation of whatever problem we really have. I have just now replaced my data directory by a symlink like this:
mv nextcloud-git-29 nextcloud-git-29-blah
ln -s nextcloud-git-29-blah nextcloud-git-29
Then everything else still works, but the phar-extension does not seem to like this change. The error message is slightly different:
Entry .tar/test.txt does not exist
You could try to replace your symlink by a bind mount, just to check if this changes anything. The phar-extension problems could perhaps be cured by using realpath
before feeding the archive file into UnifiedArchive. Still it would be good to inspect the situation further and decide whether to escalate this problem to e.g. the authors of the phar extension.
The realpath()
approach would consist of the following: replace
by
private static function getLocalPath(File $fileNode):string
{
return realpath($fileNode->getStorage()->getLocalFile($fileNode->getInternalPath()));
}
You could try to replace your symlink by a bind mount,
I was already doing that! And it works now!!!! Thanks a lot! I hope there's no side effect.
Now, I can't mount [.tar].xz
and [.tar].gz
files, should I open a new issue?
You could try to replace your symlink by a bind mount,
I was already doing that! And it works now!!!! Thanks a lot! I hope there's no side effect.
Now, I can't mount
[.tar].xz
and[.tar].gz
files, should I open a new issue?
Oops. Still? Neither can I on my laptop test setup, but it works @home in another instance. The actual backend driver in use is then indeed TarByPear, no longer TarByPhar. Please leave the issue open for the moment; the topic is correct although different things may cause the problems.
Ok, @home I can mount .tar.xz, .tar.bz2, but .tar.gz is not recognized. @work with my notebook neither of the archives works, and .tar.gz is not recognized. This means I have one system where I can reproduce the problem locally, so it should not be too difficult to solve. However, not now. I'll come back to it later.
Nextcloud (at least my instance) likes to classify .gz
as application/gzip
while the app -- or UnifiedArchive -- expects application/x-gzip
. More funny is .tgz
which is classified as application/x-compressed
. Strange enough, but of course, simple thing of a translation between Nextcloud mime-type flavour and UnifiedArchive mime-type flavour.
This does not address the problem that on some systems .tar.WHATEVER
does not work. Will probably have to wait until tomorrow.
Well, according to https://www.rfc-editor.org/rfc/rfc9110.html#name-gzip-coding "x-gzip" and "gzip" are equivalent.
Well, according to https://www.rfc-editor.org/rfc/rfc9110.html#name-gzip-coding "x-gzip" and "gzip" are equivalent.
Yes "of course", but that does not help if Nextcloud returns one flavour and your app expects the other. However, I think that the latest pre-release fixes the archive-not-detected problems (.tar.gz
, .tgz
). I did not yet address the problem that on my laptop I still cannot open those archives.
Oops. I missed that realpath() change in the latest pre-release.
Ok, about .tar.bz2
: on my Laptop UA likes to use its AlchemyZippy backend which has a bug; I'll shortly will place an issue about it. On my desktop PC @home UA likes to use the PHAR backend (where the PHP extension is buggy, but can be worked around using realpath()).
The alchemy/zippy bug got its work around in 761b4bf until wapmorgan/UnifiedArchive#44 has been merged "upstream". Can you please re-check whether perhaps now it works for you also? Why UA chooses AlchemyZippy on my laptop for .tar.bz
and ext-phar on my desktop remains an open question -- but maybe I do not want to search for the answer ;)
Hi, .tar.bz2
are working fine using TarByPhar for me after bind mounting.
Do you mean .tar.gz
?
Mmh. Ok then: are there any problems remaining? (restricting ourselves to the problems you mentioned in this issue ;) )
Of the file formats I've tried: .tar.gz
and .tar.xz
still fail. The .xz
one has the Archive tab with all fields "unknown" and the .gz
one hasn't. But you can reproduce that, right?
No, the xz stuff just also works. But please let me double check this later.
No, the xz stuff just also works. But please let me double check this later.
Ok. At home I have lzma installed, not on the laptop. If we look at the support matrix locally (no on the web, I must filed a bug report wapmorgan/UnifiedArchive#45) we have the following output:
claus@anaxagoras /var/www/dev2/htdocs/nextcloud-git-29/apps/files_archive $ php8.3 vendor/bin/cam system:formats
+----------------------+------------------------+------------+------+------+------+------+--------+--------+--------+-----------+-----+-----+-----+------+-----+-----+-----+-----+-----+-----+-----+------+-------+
| driver / format | driver type | zip | rar | gz | bz2 | xz | tar | tgz | tbz2 | 7z | cab | iso | arj | uefi | gpt | mbr | msi | dmg | rpm | deb | udf | txz | tar.z |
+----------------------+------------------------+------------+------+------+------+------+--------+--------+--------+-----------+-----+-----+-----+------+-----+-----+-----+-----+-----+-----+-----+------+-------+
| Zip | php extension | oOtxsadTcC | | | | | | | | | | | | | | | | | | | | | |
| Rar | php extension | | oOxs | | | | | | | | | | | | | | | | | | | | |
| Gzip | php extension | | | oxsc | | | | | | | | | | | | | | | | | | | |
| Bzip | php extension | | | | oxsc | | | | | | | | | | | | | | | | | | |
| Lzma | php extension | | | | | oxsc | | | | | | | | | | | | | | | | | |
| TarByPhar | php extension | oxsadc | | | | | oxsadc | oxsadc | oxsadc | | | | | | | | | | | | | | |
| SevenZip | utilities + php bridge | oOxadcC | | | | | oxadc | | | oOtxadTcC | ox | ox | ox | ox | ox | ox | ox | ox | ox | ox | ox | | |
| AlchemyZippy | utilities + php bridge | oxadc | | | | | oxadc | oxadc | oxadc | | | | | | | | | | | | | | |
| SplitbrainPhpArchive | pure php | | | | | | | | | | | | | | | | | | | | | | |
| NelexaZip | pure php | oOtxadTcCS | | | | | | | | | | | | | | | | | | | | | |
| TarByPear | pure php | | | | | | oxac | oxac | oxac | | | | | | | | | | | | | oxac | oxac |
| Iso | pure php | | | | | | | | | | | ox | | | | | | | | | | | |
| Cab | pure php | | | | | | | | | | ox | | | | | | | | | | | | |
+----------------------+------------------------+------------+------+------+------+------+--------+--------+--------+-----------+-----+-----+-----+------+-----+-----+-----+-----+-----+-----+-----+------+-------+
o - open
O - open (+password)
t - get comment
x - extract
s - stream
a - append
d - delete
T - set comment
c - create
C - create (+password)
S - create (as string)
Actually going back to you initial issue #42 it would perhaps be good to either document the use of cam
a bit better or point the users more "drastically" to wapmorgen/UnifiedArchive. In principle the installation suggestions are listed by
heinecj@nbnmh110:/var/www/html/nextcloud-git-29/apps/files_archive$ vendor/bin/cam system:drivers
driver type description / installation
Zip php extension adapter for ext-zip (1.7.3)
Rar php extension install [rar] extension.
Can be installed with pecl: pecl install rar
Gzip php extension adapter for ext-zlib (1.2.11)
Bzip php extension install [bz2] php extension
Lzma php extension install [xz] extension
For 5.x: https://github.com/payden/php-xz
For 7.x/8.x: https://github.com/codemasher/php-ext-xz
TarByPhar php extension adapter for ext-phar
SevenZip utilities + php bridge php-library and console program (16.02)
AlchemyZippy utilities + php bridge php-library and console programs
SplitbrainPhpArchive pure php install library [splitbrain/php-archive]: composer require splitbrain/php-archive
NelexaZip pure php nelexa/zip driver
TarByPear pure php php-library for tar
Iso pure php iso archives reader
Cab pure php php-library
Installing the lzma php extension seems to be the only option at the moment.
... and yes, there must be a test and a suitable error message for this case ...
6d4d4c1 is an attempt to have a less unfriendly error handling. The error text is displayed as toast and also contains some installation instructions. Perhaps one could also provide the summary generated by the system:drivers and system:formats command on the admin settings page.
Ok, installing xz
extension from https://github.com/codemasher/php-ext-xz
made it work!
How all this "mounting" works. Is it really extracted on demand or it's extracted on a temp dir until it's unmounted?
Perhaps one could also provide the summary generated by the system:drivers and system:formats command on the admin settings page.
I couldn't run php vendor/bin/cam system:formats
in my box. It complained about symfony/console
being missing.
Perhaps one could also provide the summary generated by the system:drivers and system:formats command on the admin settings page.
I couldn't run
php vendor/bin/cam system:formats
in my box. It complained aboutsymfony/console
being missing.
composer install or require could have helped for that. Anyhow, the current state of the affair is that I have hopefully improved the error message:
and in the admin settings there is the support matrix and driver overview (by executing the console commands without console, Symfony supports this via its BufferedOutput class, https://symfony.com/doc/current/console/command_in_controller.html:
Hi, openning new issue here and closing #42 .
Basically I am unable to mount any .tar (compressed or not) file but it works using directly the command line.
I've tried this that works ok:
When trying mounting in nextcloud, it results in a 500 http code.
zip files work ok and can be mounted. .tar.xz files don't work for me, even using UnifiedArchive directly.