search

joomla-docker / docker-joomla

Docker Images for Joomla!
https://hub.docker.com/_/joomla
GNU General Public License v2.0
102 stars 85 forks source link

Upgrade of Joomla from 3.x to 4.x Media Functions do not work #138

Closed MelbosaRW closed 1 year ago

MelbosaRW commented 2 years ago

I upgraded a docker install of Joomla from 3.x to 4.x within the application, and then updated docker php-fpm version to a newer image (tested both 7.4 & 8.0 tested).

NOTE: Prior to upgrade, this functionality was working fine.

Image Tag Used: joomla:php8.0-fpm-alpine

Within Joomla 4.1.0 when using docker image, if you go to Content > Media, the page loads fine, and I can navigate through exiting folders/files just fine. But I am unable to:

Logs show these errors: 

joomla-php-fpm_1  | 172.18.0.4 -  16/Feb/2022:21:36:13 +0000 "POST /administrator/index.php" 500

joomla-php-fpm_1  | NOTICE: PHP message: PHP Warning:  iconv(): Wrong encoding, conversion from "UTF-8" to "ASCII//TRANSLIT//IGNORE" is not allowed in /var/www/html/libraries/src/Filesystem/File.php on line 95

joomla-nginx_1    | 2022/02/16 21:36:13 [error] 31#31: *69 FastCGI sent in stderr: "PHP message: PHP Warning:  iconv(): Wrong encoding, conversion from "UTF-8" to "ASCII//TRANSLIT//IGNORE" is not allowed in /var/www/html/libraries/src/Filesystem/File.php on line 95" while reading response header from upstream, client: 172.19.0.4, server: www.rwoc.ca, request: "POST /administrator/index.php?option=com_media&format=json&mediatypes=0,1,2,3&task=api.files&path=local-images:/ HTTP/1.1", upstream: "fastcgi://172.18.0.6:9000", host: "www.rwoc.ca", referrer: "https://www.rwoc.ca/administrator/index.php?option=com_media&path=local-images:/"

joomla-nginx_1    | 172.19.0.4 - - [16/Feb/2022:21:36:13 +0000] "POST /administrator/index.php?option=com_media&format=json&mediatypes=0,1,2,3&task=api.files&path=local-images:/ HTTP/1.1" 500 422 "https://www.rwoc.ca/administrator/index.php?option=com_media&path=local-images:/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36" "192.197.128.19"

Researching error indicates iconv php module may be missing in image, but from what I can see the modules needed are loaded.

The line of Code that throws the error is:

// Using iconv to ignore characters that can't be transliterated

$file = iconv("UTF-8", "ASCII//TRANSLIT//IGNORE", transliterator_transliterate('Any-Latin; Latin-ASCII', $file));

I should point out I have tested this upgrade on two 3.x to 4.x and same result.

MelbosaRW commented 2 years ago

You can review the PHP Information here: https://www.rwoc.ca/phpinfo.php (NOTE: this file is only temporarily enabled)

JoMandalor commented 2 years ago

This looks like a Joomla/PHP issue and not docker related. This file: /var/www/html/libraries/src/Filesystem/File.php on line 95 changed two months ago.

How long ago did you update your site?

MelbosaRW commented 2 years ago

This looks like a Joomla/PHP issue and not docker related. This file: /var/www/html/libraries/src/Filesystem/File.php on line 95 changed two months ago.

How long ago did you update your site?

I updated just this week. It does appear to use the iconv extension, but while it shows enabled in PHP Info, it also DOES NOT show in the loaded Configure Command on PHP Info:

'./configure' '--build=x86_64-linux-musl' '--with-config-file-path=/usr/local/etc/php' '--with-config-file-scan-dir=/usr/local/etc/php/conf.d' '--enable-option-checking=fatal' '--with-mhash' '--with-pic' '--enable-ftp' '--enable-mbstring' '--enable-mysqlnd' '--with-password-argon2' '--with-sodium=shared' '--with-pdo-sqlite=/usr' '--with-sqlite3=/usr' '--with-curl' '--with-openssl' '--with-readline' '--with-zlib' '--with-pear' '--disable-cgi' '--enable-fpm' '--with-fpm-user=www-data' '--with-fpm-group=www-data' 'build_alias=x86_64-linux-musl'

Wondering if it is missing from the docker side? Something that would then show like in Configure Command:

--with-iconv

JoMandalor commented 2 years ago

The iconv function seems to exits because:

if (function_exists('transliterator_transliterate') && function_exists('iconv'))

Means that if its not there it will not try to use it, the fact that it return true on both those means its there alright.

MelbosaRW commented 2 years ago

The iconv function seems to exits because:

if (function_exists('transliterator_transliterate') && function_exists('iconv'))

Means that if its not there it will not try to use it, the fact that it return true on both those means its there alright.

So then, why this error? Do I have to raise the issue on the Joomla GitHub? I have people not running this in a docker and it does work.

JoMandalor commented 2 years ago

I just ran a few tests and it seems like the size limitation is not being handled correctly, and gives an error in the console but nothing in the browser (foxfire). But I could create folders upload images (below the size limit) and that with both php8.0 and php8.1 of Joomla 4.1

Did you check the image size limit?

MelbosaRW commented 2 years ago

Yes, I have overrides for that. I am going to try a fresh install and see if I can just copy over the files to get it to work.

MelbosaRW commented 2 years ago

I did a fresh container, with fresh files and fresh install. Still get the error:

test-joomla-php-fpm_1  | 172.18.0.7 -  17/Feb/2022:00:23:47 +0000 "POST /administrator/index.php" 500
test-joomla-php-fpm_1  | NOTICE: PHP message: PHP Warning:  iconv(): Wrong encoding, conversion from "UTF-8" to "ASCII//TRANSLIT//IGNORE" is not allowed in /var/www/html/libraries/src/Filesystem/File.php on line 95
test-joomla-nginx_1    | 2022/02/17 00:23:47 [error] 31#31: *7 FastCGI sent in stderr: "PHP message: PHP Warning:  iconv(): Wrong encoding, conversion from "UTF-8" to "ASCII//TRANSLIT//IGNORE" is not allowed in /var/www/html/libraries/src/Filesystem/File.php on line 95" while reading response header from upstream, client: 172.19.0.3, server: test.rwoc.ca, request: "POST /administrator/index.php?option=com_media&format=json&mediatypes=0,1,2,3&task=api.files&path=local-images:/ HTTP/1.1", upstream: "fastcgi://172.18.0.6:9000", host: "test.rwoc.ca", referrer: "https://test.rwoc.ca/administrator/index.php?option=com_media&path=local-images:/"
test-joomla-nginx_1    | 172.19.0.3 - - [17/Feb/2022:00:23:47 +0000] "POST /administrator/index.php?option=com_media&format=json&mediatypes=0,1,2,3&task=api.files&path=local-images:/ HTTP/1.1" 500 422 "https://test.rwoc.ca/administrator/index.php?option=com_media&path=local-images:/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36" "192.197.128.19"

Here is my docker-compose.yml, which can be critiqued as needed:

version: '3.7'

services:
  test-joomla-nginx:
    image: nginx:alpine
    restart: always
    volumes:
      - "./html:/var/www/html"
      - "./nginx/conf.d:/etc/nginx/conf.d"
    environment:
      VIRTUAL_HOST: test.rwoc.ca
      LETSENCRYPT_HOST: test.rwoc.ca
      LETSENCRYPT_EMAIL: support@servuit.com
    networks:
      - default
      - db

  test-joomla-php-fpm:
    image: joomla:php8.0-fpm-alpine
    restart: always
    volumes:
      - "./html:/var/www/html"
      - "./php-fpm/conf.d/joomlphpoverides.ini:/usr/local/etc/php/conf.d/joomlphpoverides.ini"
    environment:
      JOOMLA_DB_HOST: mysql
      JOOMLA_DB_USER: joomlausr
      JOOMLA_DB_PASSWORD: "*******************"
      JOOMLA_DB_NAME: joomlatest
    networks:
      - default
      - db

networks:
  default:
    external:
      name: websites
  db:
    external:
      name: dbnet

If @JoMandalor you have this working, then I am not sure what I am doing wrong.

Llewellynvdm commented 2 years ago

Does docker pull joomla:4.0.6-php8.0-fpm-alpine work for you?

Llewellynvdm commented 2 years ago

With last update to Joomla 4.1 we made a change to the images by removing mcrypt and libmcrypt-dev

So that means the image did not change in relation to iconv, so what did change is we moved to Joomla 4.1

What is strange to me is that @JoMandalor said it works. I will check if I can get some time this weekend to investigate this more closely. I personally don't use the fpm-alpine area of the images that often. Our tests did not report any issues with the build.

But okay I will take a closer look, and get back to you.

But please let me know if 4.0.6-php8.0-fpm-alpine works, and if it does not work, please step back until you find an image in the 4.0 range that did work. This will be helpful to find the issue.

Llewellynvdm commented 2 years ago

We are planning a number of improvements, and the images that already run on these improvements that we have in the works can be found here: https://hub.docker.com/r/llewellyn/joomla

So you could also try this image: docker pull llewellyn/joomla:4.1-fpm-alpine

With these images we have already done more work to remove deprecated styles and improved the environment slightly.

Here is the docker file: https://github.com/octoleo/docker-joomla/blob/octoleo/4.1/php8.1/fpm-alpine/Dockerfile

MelbosaRW commented 2 years ago

Does docker pull joomla:4.0.6-php8.0-fpm-alpine work for you?

I was originally on this version with my 3.x to 4.x upgrade, and is where I saw the issue first. I just completed the upgrade yesterday to 4.1 to see if it resolved the issue.

MelbosaRW commented 2 years ago

With last update to Joomla 4.1 we made a change to the images by removing mcrypt and libmcrypt-dev

So that means the image did not change in relation to iconv, so what did change is we moved to Joomla 4.1

What is strange to me is that @JoMandalor said it works. I will check if I can get some time this weekend to investigate this more closely. I personally don't use the fpm-alpine area of the images that often. Our tests did not report any issues with the build.

But okay I will take a closer look, and get back to you.

But please let me know if 4.0.6-php8.0-fpm-alpine works, and if it does not work, please step back until you find an image in the 4.0 range that did work. This will be helpful to find the issue.

I am currently on the road, but can try this step back process this weekend to let you know if I find an image that does work.

MelbosaRW commented 2 years ago

Haven't got this working yet. Did @Llewellynvdm have a chance to look at it?

Llewellynvdm commented 2 years ago

Hi @MelbosaRW you did not have time to find the image version which worked? well once you do please let us know and then I will make time to check what changed and how it effected the images.

MelbosaRW commented 2 years ago

@Llewellynvdm I will try to make time today. The last image I was using was in the 3.x versions when I was on 3.x Joomla so I am hopeful I can find a 4.x that does work. That is the last time we were able to use the image functions in the Media section.

MelbosaRW commented 2 years ago

Confirmed working:

Failed:

Based on age of the images, the changes between 4.0.3 and 4.0.4 seems to have broken this functionality. @Llewellynvdm does this give you the info you need?

Llewellynvdm commented 2 years ago

Yes, this narrows it down... I will check what changed between those images.

@JoMandalor you say you have tested this, care to share your docker-compose.yml?

MelbosaRW commented 2 years ago

@Llewellynvdm just checking in to see if any progress? I know we all work on this in our free time so do appreciate that RL can get in the way.

Llewellynvdm commented 2 years ago

@MelbosaRW the only difference between joomla:4.0.3-php8.0-fpm-alpine and joomla:4.1.0-php8.0-fpm-alpine and you can validate this your self... is the following:

diff --git a/4.0/php8.0/fpm-alpine/Dockerfile b/4.1/php8.0/fpm-alpine/Dockerfile
index 67eaf43..4667e25 100644
--- a/4.0/php8.0/fpm-alpine/Dockerfile
+++ b/4.1/php8.0/fpm-alpine/Dockerfile
@@ -21,8 +21,8 @@ RUN set -ex; \
        autoconf \
        bzip2-dev \
        gmp-dev \
+       icu-dev \
        libjpeg-turbo-dev \
-       libmcrypt-dev \
        libmemcached-dev \
        libpng-dev \
        libzip-dev \
@@ -37,6 +37,7 @@ RUN set -ex; \
        bz2 \
        gd \
        gmp \
+       intl \
        ldap \
        mysqli \
        pdo_mysql \
@@ -46,14 +47,12 @@ RUN set -ex; \
    ; \
    \
 # pecl will claim success even if one install fails, so we need to perform each install separately
-   pecl install APCu-5.1.20; \
-   pecl install mcrypt-1.0.4; \
+   pecl install APCu-5.1.21; \
    pecl install memcached-3.1.5; \
-   pecl install redis-5.3.4; \
+   pecl install redis-5.3.6; \
    \
    docker-php-ext-enable \
        apcu \
-       mcrypt \
        memcached \
        redis \
    ; \
@@ -71,8 +70,8 @@ RUN set -ex; \
 VOLUME /var/www/html

 # Define Joomla version and expected SHA512 signature
-ENV JOOMLA_VERSION 4.0.3
-ENV JOOMLA_SHA512 9c09a621586fdcc30351a915f283581c92080c210294cacc50e0e390d991f32bee9d61dfc7b0af33c81118e46d775152159c0e27dbf73f5531512328d716901f
+ENV JOOMLA_VERSION 4.1.0
+ENV JOOMLA_SHA512 44173170fb1598c465415cba919339c26624322621efed17c95fdffca7b62a1089863615e415f0ec36a6a4c4f5c746b7ec06dddd08929757f737e1a2a15ff714

 # Download package and extract to web volume
 RUN set -ex; \

To see this clone the repository and run:

git diff 5d8f256aa:4.0/php8.0/fpm-alpine/Dockerfile d758ff20:4.1/php8.0/fpm-alpine/Dockerfile

So like I said:

With last update to Joomla 4.1 we made a change to the images by removing mcrypt and libmcrypt-dev So that means the image did not change in relation to iconv, so what did change is we moved to Joomla 4.1

The above diff shows that the only part in the image that changed was the removal of mcrypt and libmcrypt-dev and the addition of icu-dev and intl and yes we updated redis-5.3.6 and APCu-5.1.21

Then going from 4.0.3 to 4.0.5 we see the addition of icu-dev and intl and yes we update of APCu-5.1.21 by J0WI

diff --git a/4.0/php8.0/fpm-alpine/Dockerfile b/4.0/php8.0/fpm-alpine/Dockerfile
index 67eaf43..92c2da3 100644
--- a/4.0/php8.0/fpm-alpine/Dockerfile
+++ b/4.0/php8.0/fpm-alpine/Dockerfile
@@ -21,6 +21,7 @@ RUN set -ex; \
        autoconf \
        bzip2-dev \
        gmp-dev \
+       icu-dev \
        libjpeg-turbo-dev \
        libmcrypt-dev \
        libmemcached-dev \
@@ -37,6 +38,7 @@ RUN set -ex; \
        bz2 \
        gd \
        gmp \
+       intl \
        ldap \
        mysqli \
        pdo_mysql \
@@ -46,7 +48,7 @@ RUN set -ex; \
    ; \
    \
 # pecl will claim success even if one install fails, so we need to perform each install separately
-   pecl install APCu-5.1.20; \
+   pecl install APCu-5.1.21; \
    pecl install mcrypt-1.0.4; \
    pecl install memcached-3.1.5; \
    pecl install redis-5.3.4; \
@@ -71,8 +73,8 @@ RUN set -ex; \
 VOLUME /var/www/html

 # Define Joomla version and expected SHA512 signature
-ENV JOOMLA_VERSION 4.0.3
-ENV JOOMLA_SHA512 9c09a621586fdcc30351a915f283581c92080c210294cacc50e0e390d991f32bee9d61dfc7b0af33c81118e46d775152159c0e27dbf73f5531512328d716901f
+ENV JOOMLA_VERSION 4.0.5
+ENV JOOMLA_SHA512 8c85c444e9183bbe11660202c860b7c8bd5d50ddde6aa5212d183fecb58f04d42a0203f05a996dfc1fd8b037d9130dad89f38f5f4f7889561879df9809253877

 # Download package and extract to web volume
 RUN set -ex; \

Again to see this run:

git diff 54a82e4b7e63:4.0/php8.0/fpm-alpine/Dockerfile 786a6f3236295:4.0/php8.0/fpm-alpine/Dockerfile

So in conclusion whatever errors you're seeing does not seem to be related to the image having changed. This is very strange to me and though I can't fully understand what is the issue, I will continue to be on the lookout.

But without more clear relationship between your errors and the changes to the images its pretty hard to resolve this issue.

I also checked the docker-entrypoint.sh file from 4.0.3 to 4.0.5 and the same for the makedb.php respectively and there is literately not changes to those documents. You can validate that like this:

git diff 54a82e4b7e63:4.0/php8.0/fpm-alpine/docker-entrypoint.sh 786a6f3236295:4.0/php8.0/fpm-alpine/docker-entrypoint.sh
git diff 54a82e4b7e63:4.0/php8.0/fpm-alpine/makedb.php 786a6f3236295:4.0/php8.0/fpm-alpine/makedb.php

I must admit that I hardly ever use these fpm-alpine images, so it could be helpful if we compare notes as to exactly how you deploy these images. Since I am wondering if what is obvious to you may not be that obvious to me. Hope we can get to the bottom of this soon.

Llewellynvdm commented 2 years ago

The information from the following areas may be helpful:

/php-fpm/conf.d/joomlphpoverides.ini
/nginx/conf.d

I know some of that may be private... so making them more generic so we can make sure the configurations your using is not the cause.

MelbosaRW commented 2 years ago

@Llewellynvdm My apologies for the lateness of my reply, RL got in the way. Thanks for your work so far.

For /nginx/conf.d location

default.conf

server {
    server_name rwoc.ca;
    return 302 $scheme://www.rwoc.ca$request_uri;
}

server {
    index index.php index.html;
    root /var/www/html;
    server_name www.rwoc.ca;

    location / {
        try_files $uri $uri/ @joomlaurls; ## adding our processing rules
    }
    location @joomlaurls {
        rewrite ^/(.+)/$ /$1 permanent; ## removing slashes
        try_files $uri $uri/ /index.php?$args; ## setting up SEF urls
    }
    location ~ [^/]\.ph(p\d*|tml)$ {
        try_files /does_not_exists @php; ## processing 404
    }

    location @php {
        try_files $uri =404;
        fastcgi_split_path_info ^((?U).+\.ph(?:p\d*|tml))(/?.+)$;
        fastcgi_pass joomla-php-fpm:9000;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }
}

settings.conf

client_max_body_size  160m;

For /php-fpm/conf.d/joomlphpoverides.ini

memory_limit = 160M
post_max_size = 160M
upload_max_filesize = 160M
upload_tmp_dir = "/var/www/html/tmp"
log_errors = On
error_log = /var/log/nginx/error.log

I can try removing these, but most of them are just around file sizes. I can also try another image set to something you @Llewellynvdm use more to see if it results in a fix? What would you suggest?

MelbosaRW commented 2 years ago

Been a while on this... I haven't tried the latest releases... was this on the list of fixes in a later release or still an issue?

Llewellynvdm commented 2 years ago

So in conclusion whatever errors you're seeing does not seem to be related to the image having changed...

So if you run Joomla 4.2 in the old version way "download the zip, unzip to a folder, link-up to the server, link the database, and serve to the browser. All outside the Joomla official container do you still encounter this issue?

Llewellynvdm commented 1 year ago

I believe there is no issue here: https://github.com/joomla-docker/docker-joomla/pull/156#issuecomment-1514763856