search

shaarli / Shaarli

The personal, minimalist, super-fast, database free, bookmarking service - community repo
https://shaarli.readthedocs.io/
Other
3.44k stars 294 forks source link

"File bookmarks_all_xxx.html has an unknown file format. Nothing was imported" #969

Closed TheNomad11 closed 6 years ago

TheNomad11 commented 7 years ago

I exported my bookmarks from shaarli (newest version) and tried to import it in another shaarli instance. Then I got this error message

"File bookmarks_all_20170917_231302.html (1139620 bytes) has an unknown file format. Nothing was imported".

The file is 1,1 MB big.

I then made the file smaller, including only one entry. Then the import worked fine.

Are there issues with larger files? Why do I get an error message about the file format being invalid?

virtualtam commented 7 years ago

Hi @TheNomad11 !

What are the respective Shaarli versions and HTTP server configurations?

Also, are the HTTP server and PHP properly configured to allow uploading files >= 1 MiB?

izumitelj commented 7 years ago

Hi @virtualtam I think I have the same problem.

My original file is 1072 KB, exported from Diigo. When importing I got the same message File xyz.html (1097148 bytes) has an unknown file format. Nothing was imported

After trimming the file (while trying to figure if this is the problem with syntax or specific characters) I've found that the problem is really in file size. File trimmed to 100221 bytes imports OK while file with 100222 bytes throws the error.

My server configuration:

Apache Version  2.4.25
PHP Version     5.6.30
Architecture    x86_64
Operating System    linux

Some of my PHP settings:

max_execution_time   90 
max_input_time   60 
memory_limit   128M 
upload_max_filesize   2M

I'm not administrator of this server but I can ask for changes if that is the problem.

virtualtam commented 7 years ago

Hi @izumitelj !

What is the value for the PHP post_max_size setting?

izumitelj commented 7 years ago

@virtualtam here it is:

post_max_size  64M
upload_max_filesize  32M

and here are other settings

izumitelj commented 7 years ago

@virtualtam It seems like this has to do with PHP version. The issue is present with PHP 5.6, while with PHP 7 it works as expected without errors.

Unfortunately I can't switch to PHP 7 yet :(.

Can someone look into this issue, probably NetscapeBookmarkUtils.php, or at least revisit the documentation.

Thanks

virtualtam commented 7 years ago

OK, just to sum things up, could you provide us with the following information so we can try to reproduce the issue?

Note:

PHP provides different configuration files (one per environment: apache2, cli, fpm, etc.), be sure to check values from the one used by the actual HTTP server (Apache2 / Nginx+FPM, etc.)

Example on a dedicated server running Debian 8 Jessie, using Apache 2.4 and PHP 5.6:

$ uname -a
Linux myserver 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u5 (2015-10-09) x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 8.2 (jessie)
Release:    8.2
Codename:   jessie

$ dpkg -l apache2 | grep ^i
ii  apache2        2.4.10-10+deb8u10 amd64        Apache HTTP Server

$ dpkg -l php5 | grep ^i
ii  php5           5.6.30+dfsg-0+deb8u1 all          server-side, HTML-embedded scripting language (metapackage)

$ grep post_max /etc/php5/apache2/php.ini
post_max_size = 501M

$ grep upload_max /etc/php5/apache2/php.ini
upload_max_filesize = 500M

In the meantime, I'll write a tool to generate dummy bookmark files of configurable size, to run tests against different environments.

izumitelj commented 7 years ago

Hmm... maybe I've made wrong assumption about incompatible PHP version. System admins said that they have checked all relevant variables and even tried switching Mod Security off. According to them the only thing that fixed the problem was switching to PHP 7 (which I couldn't do because of the other services).

In the mean time they gave me chance to migrate to newer server that supports multiple PHP versions per subdomain.

Now I've tested it and it works OK on both versions PHP 5.6.31 and 7.0.23, so the problem might be somewhere else! Unfortunately I can't do any testing on ex-server but I still have some info saved.

Old server:

Linux servername 2.6.32-696.6.3.el6.x86_64 #1 SMP Wed Jul 12 14:17:22 UTC 2017 x86_64
Apache Version  2.4.25
PHP Version     5.6.27
post_max_size  64M
upload_max_filesize  32M

New server:

Linux servername 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 
Apache Version  2.4.27
PHP Version     5.6.31
PHP Version     7.0.23
post_max_size  48M
upload_max_filesize  32M
izumitelj commented 7 years ago

With release 0.9.2 I've successfully imported my bookmarks (1.098.716 bytes, 4583 links) without need to split the file in half. It took some time to process but worked.