Plugins with [key]http://somesite/somefile.zip don't work

DowlingIT commented 4 years ago

Overview

What's your problem about? Plugins with [key]http://somesite/somefile.zip will not install. I tried your example docker-compose file, with no changes at all (and verified that the zip file you point at actually does exist)
What is your operating system? Ubuntu 18.04.3 LTS
What is your docker version? docker version 18.09.7, build 2d0083d

`docker-compose.yml`

services:
    wordpress:
        image: visiblevc/wordpress

        # required for mounting bindfs
        cap_add:
            - SYS_ADMIN
        devices:
            - /dev/fuse
        # required on certain cloud hosts
        security_opt:
            - apparmor:unconfined

        ports:
            - 8080:80
            - 443:443
        volumes:
            - ./data:/data
            - ./scripts:/docker-entrypoint-initwp.d
        environment:
            DB_NAME: wordpress
            DB_PASS: root
            PLUGINS: >-
                academic-bloggers-toolkit
                co-authors-plus
                [WP-API]https://github.com/WP-API/WP-API/archive/master.zip

    db:
        image: mariadb:10 # or mysql:5.7
        volumes:
            - data:/var/lib/mysql
        environment:
            MYSQL_ROOT_PASSWORD: root

    phpmyadmin:
        image: phpmyadmin/phpmyadmin
        ports:
            - 22222:80

volumes:
    data:

Project structure

Using the example folder in this repository, no changes

`docker-compose up` output

wordpress_1   | Success: Generated 'wp-config.php' file.
wordpress_1   | ==> Checking database
wordpress_1   | ==> Checking themes
wordpress_1   | ==> Checking plugins
wordpress_1   | ==> Finalizing
wordpress_1   | Success: Rewrite structure set.
wordpress_1   | Success: Rewrite rules flushed.
wordpress_1   | ==> Executing user init scripts
wordpress_1   | Warning: The 'my-plugin' plugin could not be found.
wordpress_1   | Error: No plugins activated.

The "my-plugin' notes come from the /scripts/init.sh scrupt you have in your repo. Note that there isn't an attempt to get any other plugins.

I believe this is due to illegal bash characters. In my own project, which has a plugins structure like this:

version: '3.1'
networks:
  dev: {}

volumes:
  data:

services:
  wordpress:
    image: visiblevc/wordpress:latest
    networks:
      - dev

    # required for mounting bindfs
    cap_add:
      - SYS_ADMIN
    devices:
      - /dev/fuse
    # required on certain cloud hosts
    security_opt:
      - apparmor:unconfined
    ports:
      - 8080:80
      - 443:443
    volumes:
      - ./data:/data
      - ./scripts:/docker-entrypoint-initwp.d
      - ./mylabcare-plugin:/app/wp-content/plugins/mylabcare-plugin
    environment:
      DB_NAME: ${MYSQL_DATABASE}
      DB_PASS: ${MYSQL_PASSWORD}
      ADMIN_EMAIL: support@lablynx.com
      SERVER_NAME: ${SITE_ADDRESS} 
      PLUGINS: >-
        elementor
        [WP-API]https://github.com/WP-API/WP-API/archive/master.zip

      THEMES: >-
        super-minimal

    depends_on:
      - db

  db:
    image: mariadb:10 # or mysql:5.7
    networks:
      - dev
    volumes:
      - data:/var/lib/mysql
    environment:
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MYSQL_ROOT_PASSWORD: ${MYSQL_PASSWORD}

  phpmyadmin:
    image: phpmyadmin/phpmyadmin:latest
    networks:
      - dev
    ports:
      - 22222:80
    environment: 
      MYSQL_ROOT_PASSWORD: ${MYSQL_PASSWORD}

my docker-compose up output is:

wordpress_1   | ======================================================================
wordpress_1   |                     Begin WordPress Installation                    
wordpress_1   | ======================================================================
wordpress_1   | ==> Downloading WordPress
wordpress_1   |          Downloading WordPress 5.3.2 (en_US)...
wordpress_1   |          md5 hash verified: 380d41ad22c97bd4fc08b19a4eb97403
wordpress_1   | Success: WordPress downloaded.
wordpress_1   | ==> Waiting for MySQL to initialize...
wordpress_1   | ==> Configuring WordPress
wordpress_1   | Success: Generated 'wp-config.php' file.
wordpress_1   | ==> Checking database
wordpress_1   | ==> Checking themes
wordpress_1   |          Installing Super Minimal (0.1.0)
wordpress_1   |          Downloading installation package from 
wordpress_1   |          https://downloads.wordpress.org/theme/super-minimal.0.1.0.zip
wordpress_1   |          ...
wordpress_1   |          Unpacking the package...
wordpress_1   |          Installing the theme...
wordpress_1   |          Theme installed successfully.
wordpress_1   | Success: Installed 1 of 1 themes.
wordpress_1   | ==> Checking plugins
wordpress_1   |          Installing Elementor Page Builder (2.8.5)
wordpress_1   |          Downloading installation package from 
wordpress_1   |          https://downloads.wordpress.org/plugin/elementor.2.8.5.zip...
wordpress_1   |          Unpacking the package...
wordpress_1   |          Installing the plugin...
wordpress_1   |          Plugin installed successfully.
wordpress_1   | Success: Installed 1 of 1 plugins.
wordpress_1   | ==> Finalizing
wordpress_1   | Success: Rewrite structure set.
wordpress_1   | Success: Rewrite rules flushed.
wordpress_1   | ==> Executing user init scripts
wordpress_1   | ======================================================================
wordpress_1   |                   WordPress Installation Complete!                  
wordpress_1   | ======================================================================

Which gets elementor, but skips the zip file you use in the example. When I leave off the key at the front, like:

PLUGINS: >-
        elementor
        https://github.com/WP-API/WP-API/archive/master.zip

then the docker-compose up becomes:

wordpress_1   | ======================================================================
wordpress_1   |                     Begin WordPress Installation                    
wordpress_1   | ======================================================================
wordpress_1   | ==> Downloading WordPress
wordpress_1   |          Downloading WordPress 5.3.2 (en_US)...
wordpress_1   |          md5 hash verified: 380d41ad22c97bd4fc08b19a4eb97403
wordpress_1   | Success: WordPress downloaded.
wordpress_1   | ==> Waiting for MySQL to initialize...
wordpress_1   | ==> Configuring WordPress
wordpress_1   | Success: Generated 'wp-config.php' file.
wordpress_1   | ==> Checking database
wordpress_1   | ==> Checking themes
wordpress_1   |          Installing Super Minimal (0.1.0)
wordpress_1   |          Downloading installation package from 
wordpress_1   |          https://downloads.wordpress.org/theme/super-minimal.0.1.0.zip
wordpress_1   |          ...
wordpress_1   |          Unpacking the package...
wordpress_1   |          Installing the theme...
wordpress_1   |          Theme installed successfully.
wordpress_1   | Success: Installed 1 of 1 themes.
wordpress_1   | ==> Checking plugins
wordpress_1   |          Installing Elementor Page Builder (2.8.5)
wordpress_1   |          Downloading installation package from 
wordpress_1   |          https://downloads.wordpress.org/plugin/elementor.2.8.5.zip...
wordpress_1   |          Unpacking the package...
wordpress_1   |          Installing the plugin...
wordpress_1   |          Plugin installed successfully.
wordpress_1   | Success: Installed 1 of 1 plugins.
wordpress_1   |          Downloading installation package from 
wordpress_1   |          https://github.com/WP-API/WP-API/archive/master.zip...
wordpress_1   |          Unpacking the package...
wordpress_1   |          Installing the plugin...
wordpress_1   |          Renamed Github-based project from 'WP-API-master' to 
wordpress_1   |          'WP-API'.
wordpress_1   |          Plugin installed successfully.
wordpress_1   | Success: Installed 1 of 1 plugins.
wordpress_1   | Warning: The 
wordpress_1   |          'https://github.com/WP-API/WP-API/archive/master.zip' plugin 
wordpress_1   |          could not be found.
wordpress_1   |   Error: No plugins activated.
wordpress_1   | ==> Finalizing
wordpress_1   | Success: Rewrite structure set.
wordpress_1   | Success: Rewrite rules flushed.
wordpress_1   | ==> Executing user init scripts
wordpress_1   | ======================================================================
wordpress_1   |                   WordPress Installation Complete!                  
wordpress_1   | ======================================================================

At this point, there is at least an attempt to look for it, as given by the "Warning"

I tried this approach after I copied some of the code you have in the run.sh script that gets the plugin dependencies, and altered it so I could see what it was doing:

#!/bin/bash
shopt -s nullglob

# Environment
declare PLUGINS='elementor
    elementor-pro
        [API]https://github.com/WP-API/WP-API/archive/master.zip
    [EP]https://my.elementor.com/index.php?XX
'

declare -A plugin_deps

declare raw_line
declare -a keyvalue

echo "P: $PLUGINS"
for raw_line in $PLUGINS; do
    echo "RL: $raw_line"
    mapfile -t keyvalue < <(
        sed -n '
        s/.*\[\(.*\)\]\([^[:blank:]]*\).*/\1\n\2/p # Matches [key]value form
        t                                          # If previous match succeeds, skip to end
        {p; p;}                                    # Assumes normal form
        ' <<< "$raw_line"
    )
    plugin_deps[${keyvalue[0]}]="${keyvalue[1]}"
done

for key in "${!plugin_deps[@]}"; do
    echo "K: $key V: ${plugin_deps[$key]}"
done

In the above, the output won't even echo the lines that have [xx] in them as raw_line vars If you put a backslash in front (eg. [API]https://github.com/WP-API/WP-API/archive/master.zip ), you'll get some output.

The ? character in a URL that has a query string also causes output issues.

Unfortunately, the same escape tricks that work in the shell script don't translate to the docker-compose env var handling.

dsifford commented 4 years ago

Hi @DowlingIT -- This is strange that you're not having luck with the url form. I use this environment on a daily basis for work and I just used the url form today and had no issues.

Are you certain that you have the latest version of the image?

dsifford commented 4 years ago

~~Also -- are you on windows? It may be CRLF endings that screws this up.~~

Edit: Nope, looks like you're on ubuntu.

DowlingIT commented 4 years ago

I’m using the :latest tag in my own stuff. And did the git clone today to test out the example you had at github. I’m on Ubuntu 18 LTS

Thanks

From: Derek Sifford notifications@github.com Sent: Wednesday, February 12, 2020 6:48 PM To: visiblevc/wordpress-starter wordpress-starter@noreply.github.com Cc: Jeremy Dowling jdowling@dowlingit.com; Mention mention@noreply.github.com Subject: Re: [visiblevc/wordpress-starter] Plugins with [key]http://somesite/somefile.zip don't work (#176)

Also -- are you on windows? It may be CRLF endings that screws this up.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/visiblevc/wordpress-starter/issues/176?email_source=notifications&email_token=AAZQGEZFIDZITAI3IOL6WOLRCSDC5A5CNFSM4KUGH3F2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELS2QRQ#issuecomment-585476166, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAZQGE7KUV6LJOZBZBZZAALRCSDC5ANCNFSM4KUGH3FQ.

dsifford commented 4 years ago

Can you give it a shot with 0.25.3-php7.4 (or whatever php version you need) instead and let me know what happens?

DowlingIT commented 4 years ago

Sure. I did a docker-compose rm -f in between as well, to clear out cached container images

After checking the git repo out to that commit, I ran docker-compose up in the example folder. Got:

wordpress_1 | Downloading WordPress 5.3.2 (en_US)... wordpress_1 | md5 hash verified: 380d41ad22c97bd4fc08b19a4eb97403 wordpress_1 | Success: WordPress downloaded. wordpress_1 | ==> Waiting for MySQL to initialize... db_1 | 2020-02-13 0:16:37 0 [Note] InnoDB: Buffer pool(s) load completed at 200213 0:16:37 wordpress_1 | ==> Configuring WordPress wordpress_1 | Success: Generated 'wp-config.php' file. wordpress_1 | ==> Checking database wordpress_1 | ==> Checking themes wordpress_1 | Installing Twenty Twenty (1.1) wordpress_1 | Downloading installation package from wordpress_1 | https://downloads.wordpress.org/theme/twentytwenty.1.1.zip... wordpress_1 | Unpacking the package... wordpress_1 | Installing the theme... wordpress_1 | Theme installed successfully. wordpress_1 | Success: Installed 1 of 1 themes. wordpress_1 | ==> Checking plugins wordpress_1 | Installing Co-Authors Plus (3.4.2) wordpress_1 | Downloading installation package from wordpress_1 | https://downloads.wordpress.org/plugin/co-authors-plus.3.4.2. wordpress_1 | zip... wordpress_1 | Unpacking the package... wordpress_1 | Installing the plugin... wordpress_1 | Plugin installed successfully. wordpress_1 | Success: Installed 1 of 1 plugins. wordpress_1 | Installing Academic Blogger's Toolkit (5.2.2) wordpress_1 | Downloading installation package from wordpress_1 | https://downloads.wordpress.org/plugin/academic-bloggers-tool wordpress_1 | kit.5.2.2.zip... wordpress_1 | Unpacking the package... wordpress_1 | Installing the plugin... wordpress_1 | Plugin installed successfully. wordpress_1 | Success: Installed 1 of 1 plugins. wordpress_1 | ==> Finalizing wordpress_1 | Success: Rewrite structure set. wordpress_1 | Success: Rewrite rules flushed. wordpress_1 | ==> Executing user init scripts wordpress_1 | Warning: The 'my-plugin' plugin could not be found. wordpress_1 | Error: No plugins activated. wordpress_1 | ====================================================================== wordpress_1 | WordPress Installation Complete! wordpress_1 | ======================================================================

so it skipped that url based install again.

Just to verify, I ran udo docker exec -it 432a072807ca /bin/bash to get into the shell of the running docker container, and listed the plugins folders. No sign of it:

[cid:image001.png@01D5E1D9.C8436330]

Regards

From: Derek Sifford notifications@github.com Sent: Wednesday, February 12, 2020 6:56 PM To: visiblevc/wordpress-starter wordpress-starter@noreply.github.com Cc: Jeremy Dowling jdowling@dowlingit.com; Mention mention@noreply.github.com Subject: Re: [visiblevc/wordpress-starter] Plugins with [key]http://somesite/somefile.zip don't work (#176)

Can you give it a shot with 0.25.3-php7.4 (or whatever php version you need) instead and let me know what happens?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/visiblevc/wordpress-starter/issues/176?email_source=notifications&email_token=AAZQGE56NO2PFZKUS6HE2RDRCSECPA5CNFSM4KUGH3F2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELS3DNY#issuecomment-585478583, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAZQGE3T635BCUW7BMYZKPDRCSECPANCNFSM4KUGH3FQ.

dsifford commented 4 years ago

Ok wow, what a weird bug this was...

Turns out, this is related to nullglob.

Good catch. I'll fix this soon.

visiblevc / wordpress-starter