I'm seeing the following issue on Acquia Pipelines with BLT

Memcache: Extensions not available

The memcache module is installed via composer.

If I run php -i via a bash script I'm not seeing memcache as an installed extension with php.

Any ideas how to get around this?

@justinlevi what version of memcache are you pulling in with composer?

"drupal/memcache": "2.0-alpha7",

with the following patch

            "drupal/memcache": {
                "Undefined method error on Statistics page": "https://www.drupal.org/files/issues/2018-09-05/memcache_undefined-2997537-3.patch"
            },

Per the documentation here: https://docs.acquia.com/acquia-cloud/cd/resource/

Memcache should be installed in the container, but here's what I'm seeing running php -i during the build step:

Note that memcached is not listed

System => Linux f116ff79189b 3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 15 17:36:42 UTC 2018 x86_64
Build Date => Nov 12 2018 09:23:22
Server API => Command Line Interface
Virtual Directory Support => disabled
Configuration File (php.ini) Path => /etc/php/7.1/cli
Loaded Configuration File => /etc/php/7.1/cli/php.ini
Scan this dir for additional .ini files => /etc/php/7.1/cli/conf.d
Additional .ini files parsed => /etc/php/7.1/cli/conf.d/10-mysqlnd.ini,
/etc/php/7.1/cli/conf.d/10-opcache.ini,
/etc/php/7.1/cli/conf.d/10-pdo.ini,
/etc/php/7.1/cli/conf.d/15-xml.ini,
/etc/php/7.1/cli/conf.d/20-bcmath.ini,
/etc/php/7.1/cli/conf.d/20-bz2.ini,
/etc/php/7.1/cli/conf.d/20-calendar.ini,
/etc/php/7.1/cli/conf.d/20-ctype.ini,
/etc/php/7.1/cli/conf.d/20-curl.ini,
/etc/php/7.1/cli/conf.d/20-dom.ini,
/etc/php/7.1/cli/conf.d/20-exif.ini,
/etc/php/7.1/cli/conf.d/20-fileinfo.ini,
/etc/php/7.1/cli/conf.d/20-ftp.ini,
/etc/php/7.1/cli/conf.d/20-gd.ini,
/etc/php/7.1/cli/conf.d/20-gettext.ini,
/etc/php/7.1/cli/conf.d/20-gmp.ini,
/etc/php/7.1/cli/conf.d/20-iconv.ini,
/etc/php/7.1/cli/conf.d/20-imagick.ini,
/etc/php/7.1/cli/conf.d/20-json.ini,
/etc/php/7.1/cli/conf.d/20-mbstring.ini,
/etc/php/7.1/cli/conf.d/20-mcrypt.ini,
/etc/php/7.1/cli/conf.d/20-mysqli.ini,
/etc/php/7.1/cli/conf.d/20-pdo_mysql.ini,
/etc/php/7.1/cli/conf.d/20-pdo_sqlite.ini,
/etc/php/7.1/cli/conf.d/20-phar.ini,
/etc/php/7.1/cli/conf.d/20-posix.ini,
/etc/php/7.1/cli/conf.d/20-readline.ini,
/etc/php/7.1/cli/conf.d/20-shmop.ini,
/etc/php/7.1/cli/conf.d/20-simplexml.ini,
/etc/php/7.1/cli/conf.d/20-sockets.ini,
/etc/php/7.1/cli/conf.d/20-sqlite3.ini,
/etc/php/7.1/cli/conf.d/20-sysvmsg.ini,
/etc/php/7.1/cli/conf.d/20-sysvsem.ini,
/etc/php/7.1/cli/conf.d/20-sysvshm.ini,
/etc/php/7.1/cli/conf.d/20-tokenizer.ini,
/etc/php/7.1/cli/conf.d/20-wddx.ini,
/etc/php/7.1/cli/conf.d/20-xmlreader.ini,
/etc/php/7.1/cli/conf.d/20-xmlrpc.ini,
/etc/php/7.1/cli/conf.d/20-xmlwriter.ini,
/etc/php/7.1/cli/conf.d/20-xsl.ini,
/etc/php/7.1/cli/conf.d/20-yaml.ini,
/etc/php/7.1/cli/conf.d/20-zip.ini

PHP API => 20160303
PHP Extension => 20160303
Zend Extension => 320160303
Zend Extension Build => API320160303,NTS
PHP Extension Build => API20160303,NTS
Debug Build => no
Thread Safety => disabled
Zend Signal Handling => enabled
Zend Memory Manager => enabled
Zend Multibyte Support => provided by mbstring
IPv6 Support => enabled
DTrace Support => available, disabled

I would expect to see the following on the CI Container if the php memcache extension was installed.

drush @my-site-alias.dev eval "phpinfo()" | grep memcache:

/usr/local/php7.1/etc/cli/conf.d/memcached.ini,
memcached
memcached support => enabled
libmemcached version => 1.0.18
memcached.compression_factor => 1.3 => 1.3
memcached.compression_threshold => 2000 => 2000
memcached.compression_type => fastlz => fastlz
memcached.default_binary_protocol => 0 => 0
memcached.default_connect_timeout => 0 => 0
memcached.default_consistent_hash => 0 => 0
memcached.serializer => php => php
memcached.sess_binary_protocol => 1 => 1
memcached.sess_connect_timeout => 0 => 0
memcached.sess_consistent_hash => 1 => 1
memcached.sess_lock_expire => 0 => 0
memcached.sess_lock_max_wait => not set => not set
memcached.sess_lock_retries => 5 => 5
memcached.sess_lock_wait => not set => not set
memcached.sess_lock_wait_max => 2000 => 2000
memcached.sess_lock_wait_min => 1000 => 1000
memcached.sess_locking => 1 => 1
memcached.sess_number_of_replicas => 0 => 0
memcached.sess_persistent => 0 => 0
memcached.sess_prefix => memc.sess. => memc.sess.
memcached.sess_randomize_replica_read => 0 => 0
memcached.sess_remove_failed_servers => 0 => 0
memcached.sess_sasl_password => no value => no value
memcached.sess_sasl_username => no value => no value
memcached.sess_server_failure_limit => 0 => 0
memcached.store_retry_count => 2 => 2
Registered save handlers => files user memcached

I have opened an Acquia support ticket as well and chatted with @lcatlett about this. Hoping to get this sorted out soon though.

Just for reference as well, here's the original memecache.settings.php included from settings.php for the blt 8.9.x

memecache.settings.php

``` addPsr4('Drupal\\memcache\\', 'modules/contrib/memcache/src'); $class_loader->register(); $settings['container_yamls'][] = DRUPAL_ROOT . '/modules/contrib/memcache/memcache.services.yml'; // Bootstrap cache.container with memcache rather than database. $settings['bootstrap_container_definition'] = [ 'parameters' => [], 'services' => [ 'database' => [ 'class' => 'Drupal\Core\Database\Connection', 'factory' => 'Drupal\Core\Database\Database::getConnection', 'arguments' => ['default'], ], 'settings' => [ 'class' => 'Drupal\Core\Site\Settings', 'factory' => 'Drupal\Core\Site\Settings::getInstance', ], 'memcache.settings' => [ 'class' => 'Drupal\memcache\MemcacheSettings', 'arguments' => ['@settings'], ], 'memcache.backend.cache.factory' => [ 'class' => 'Drupal\memcache\MemcacheDriverFactory', 'arguments' => ['@memcache.config'], ], 'memcache.backend.cache.container' => [ 'class' => 'Drupal\memcache\DrupalMemcacheFactory', 'factory' => ['@memcache.backend.cache.factory', 'get'], 'arguments' => ['container'], ], 'lock.container' => [ 'class' => 'Drupal\memcache\Lock\MemcacheLockBackend', 'arguments' => ['container', '@memcache.backend.cache.container'], ], 'cache_tags_provider.container' => [ 'class' => 'Drupal\Core\Cache\DatabaseCacheTagsChecksum', 'arguments' => ['@database'], ], 'cache.container' => [ 'class' => 'Drupal\memcache\MemcacheBackend', 'arguments' => [ 'container', '@memcache.backend.cache.container', '@lock.container', '@memcache.config', '@cache_tags_provider.container', ], ], ], ]; // Override default fastchained backend for static bins. // @see https://www.drupal.org/node/2754947 $settings['cache']['bins']['bootstrap'] = 'cache.backend.memcache'; $settings['cache']['bins']['discovery'] = 'cache.backend.memcache'; $settings['cache']['bins']['config'] = 'cache.backend.memcache'; // Use memcache as the default bin. $settings['cache']['default'] = 'cache.backend.memcache'; // Enable stampede protection. $settings['memcache']['stampede_protection'] = TRUE; // Move locks to memcache. $settings['container_yamls'][] = DRUPAL_ROOT . '/../vendor/acquia/blt/settings/memcache.yml'; } } ```

@lcatlett Just ran into this again with a fresh BLT project using 9.2. Unfortunately the hack I was using on an older BLT setup by creating a memcache.settings.php file in docroot/sites/default/settings/memcache.settings.php with the following conditional no longer seems to work.

Note, I tried loading the following custom memcache.settings.php after blt.settings.php is included at the bottom of the normal setting.php file. Is there a newer way to ignore memcache on pipelines at this time?

I opened a support ticket as well.


<?php
/**
 * @file
 * Contains memcache configuration.
 */
use Composer\Autoload\ClassLoader;

// TEMPORARY HACK UNTIL MEMCACHE IS INSTALLED ON PIPELINES
if(!isset($_ENV['PIPELINE_ENV'])){

  // Check for PHP Memcached libraries.
  $memcache_exists = class_exists('Memcache', FALSE);
  $memcached_exists = class_exists('Memcached', FALSE);
  $memcache_module_is_present = file_exists(DRUPAL_ROOT . '/modules/contrib/memcache/memcache.services.yml');
  if ($memcache_module_is_present && ($memcache_exists || $memcached_exists)) {
    // Use Memcached extension if available.
    if ($memcached_exists) {
      $settings['memcache']['extension'] = 'Memcached';
    }
    if (class_exists(ClassLoader::class)) {
      $class_loader = new ClassLoader();
      $class_loader->addPsr4('Drupal\\memcache\\', 'modules/contrib/memcache/src');
      $class_loader->register();
      $settings['container_yamls'][] = DRUPAL_ROOT . '/modules/contrib/memcache/memcache.services.yml';
      // Bootstrap cache.container with memcache rather than database.
      $settings['bootstrap_container_definition'] = [
        'parameters' => [],
        'services' => [
          'database' => [
            'class' => 'Drupal\Core\Database\Connection',
            'factory' => 'Drupal\Core\Database\Database::getConnection',
            'arguments' => ['default'],
          ],
          'settings' => [
            'class' => 'Drupal\Core\Site\Settings',
            'factory' => 'Drupal\Core\Site\Settings::getInstance',
          ],
          'memcache.settings' => [
            'class' => 'Drupal\memcache\MemcacheSettings',
            'arguments' => ['@settings'],
          ],
          'memcache.backend.cache.factory' => [
            'class' => 'Drupal\memcache\MemcacheDriverFactory',
            'arguments' => ['@memcache.config'],
          ],
          'memcache.backend.cache.container' => [
            'class' => 'Drupal\memcache\DrupalMemcacheFactory',
            'factory' => ['@memcache.backend.cache.factory', 'get'],
            'arguments' => ['container'],
          ],
          'lock.container' => [
            'class' => 'Drupal\memcache\Lock\MemcacheLockBackend',
            'arguments' => ['container', '@memcache.backend.cache.container'],
          ],
          'cache_tags_provider.container' => [
            'class' => 'Drupal\Core\Cache\DatabaseCacheTagsChecksum',
            'arguments' => ['@database'],
          ],
          'cache.container' => [
            'class' => 'Drupal\memcache\MemcacheBackend',
            'arguments' => [
              'container',
              '@memcache.backend.cache.container',
              '@lock.container',
              '@memcache.config',
              '@cache_tags_provider.container',
            ],
          ],
        ],
      ];
      // Override default fastchained backend for static bins.
      // @see https://www.drupal.org/node/2754947
      $settings['cache']['bins']['bootstrap'] = 'cache.backend.memcache';
      $settings['cache']['bins']['discovery'] = 'cache.backend.memcache';
      $settings['cache']['bins']['config'] = 'cache.backend.memcache';
      // Use memcache as the default bin.
      $settings['cache']['default'] = 'cache.backend.memcache';
      // Enable stampede protection.
      $settings['memcache']['stampede_protection'] = TRUE;
      // Move locks to memcache.
      $settings['container_yamls'][] = DRUPAL_ROOT . '/../vendor/acquia/blt/settings/memcache.yml';
    }
  }
}

Just for reference, here's a bit more of the pipelines error:


> internal:drupal:install

[Acquia\Blt\Robo\Tasks\DrushTask] Running /mnt/tmp/local.prod/source/vendor/bin/drush @self site-install webny install_configure_form.update_status_module='array(FALSE,FALSE)' install_configure_form.enable_update_status_module=NULL --sites-subdir=default --site-name='WebNY Core' --site-mail=no-reply@acquia.com --account-name=VDNtlczOBJ --account-mail=no-reply@acquia.com --locale=en --no-interaction -v --ansi in /mnt/tmp/local.prod/source/docroot

 [info] Executing: mysql --defaults-file=/tmp/drush_zmgKfQ --database=drupal --host=127.0.0.1 --port=3306 --silent -A < /tmp/drush_NVPdod > /dev/null

 [info] Sites directory sites/default already exists - proceeding.
 [info] Executing: mysql --defaults-file=/tmp/drush_a2uJ35 --database=drupal --host=127.0.0.1 --port=3306 --silent -A < /tmp/drush_fSRI9s > /dev/null

 [info] Executing: mysql --defaults-file=/tmp/drush_I45hWl --database=drupal --host=127.0.0.1 --port=3306 --silent -A < /tmp/drush_BAvZZI

 [notice] Starting Drupal installation. This takes a while.

In install.core.inc line 2289:

  [Drupal\Core\Installer\Exception\InstallerException]                         
  Memcache: Extensions not available                                           

  Either the <a href="http://php.net/manual/en/book.memcache.php">memcache</a  
  > or <a href="http://php.net/manual/en/book.memcached.php">memcached</a> ex  
  tensions must be installed in order to use memcache integration.             

Exception trace:
 () at /mnt/tmp/local.prod/source/docroot/core/includes/install.core.inc:2289
 install_display_requirements() at /mnt/tmp/local.prod/source/docroot/core/includes/install.core.inc:1095
 install_verify_requirements() at /mnt/tmp/local.prod/source/docroot/core/includes/install.core.inc:709
 install_run_task() at /mnt/tmp/local.prod/source/docroot/core/includes/install.core.inc:584
 install_run_tasks() at /mnt/tmp/local.prod/source/docroot/core/includes/install.core.inc:125
 install_drupal() at /mnt/tmp/local.prod/source/vendor/drush/drush/includes/drush.inc:238
 drush_call_user_func_array() at /mnt/tmp/local.prod/source/vendor/drush/drush/includes/drush.inc:223
 drush_op() at /mnt/tmp/local.prod/source/vendor/drush/drush/src/Commands/core/SiteInstallCommands.php:141
 Drush\Commands\core\SiteInstallCommands->install() at n/a:n/a
 call_user_func_array() at /mnt/tmp/local.prod/source/vendor/consolidation/annotated-command/src/CommandProcessor.php:257
 Consolidation\AnnotatedCommand\CommandProcessor->runCommandCallback() at /mnt/tmp/local.prod/source/vendor/consolidation/annotated-command/src/CommandProcessor.php:212
 Consolidation\AnnotatedCommand\CommandProcessor->validateRunAndAlter() at /mnt/tmp/local.prod/source/vendor/consolidation/annotated-command/src/CommandProcessor.php:178
 Consolidation\AnnotatedCommand\CommandProcessor->process() at /mnt/tmp/local.prod/source/vendor/consolidation/annotated-command/src/AnnotatedCommand.php:302
 Consolidation\AnnotatedCommand\AnnotatedCommand->execute() at /mnt/tmp/local.prod/source/vendor/symfony/console/Command/Command.php:255
 Symfony\Component\Console\Command\Command->run() at /mnt/tmp/local.prod/source/vendor/symfony/console/Application.php:971
 Symfony\Component\Console\Application->doRunCommand() at /mnt/tmp/local.prod/source/vendor/symfony/console/Application.php:248
 Symfony\Component\Console\Application->doRun() at /mnt/tmp/local.prod/source/vendor/symfony/console/Application.php:148
 Symfony\Component\Console\Application->run() at /mnt/tmp/local.prod/source/vendor/drush/drush/src/Runtime/Runtime.php:112
 Drush\Runtime\Runtime->doRun() at /mnt/tmp/local.prod/source/vendor/drush/drush/src/Runtime/Runtime.php:41
 Drush\Runtime\Runtime->run() at /mnt/tmp/local.prod/source/vendor/drush/drush/drush.php:66
 require() at /mnt/tmp/local.prod/source/vendor/drush/drush/drush:4

site:install [--db-url DB-URL] [--db-prefix DB-PREFIX] [--db-su DB-SU] [--db-su-pw DB-SU-PW] [--account-name [ACCOUNT-NAME]] [--account-mail [ACCOUNT-MAIL]] [--site-mail [SITE-MAIL]] [--account-pass ACCOUNT-PASS] [--locale [LOCALE]] [--site-name [SITE-NAME]] [--site-pass SITE-PASS] [--sites-subdir SITES-SUBDIR] [--config-dir CONFIG-DIR] [--existing-config] [-h--help] [-q--quiet] [-vvvvvv--verbose] [-V--version] [--ansi] [--no-ansi] [-n--no-interaction] [-d--debug] [-y--yes] [--no] [--remote-host REMOTE-HOST] [--remote-user REMOTE-USER] [-r--root ROOT] [-l--uri URI] [--simulate] [--pipe] [-D--define DEFINE] [--xh-link XH-LINK] [--notify [NOTIFY]] [--druplicon] [--] <command> [<profile>]...

[Acquia\Blt\Robo\Tasks\DrushTask]  Exit code 1  Time 0.783s
[error]  Failed to install Drupal! 
[error]  Command `internal:drupal:install ` exited with code 1. 
[error]  Command `drupal:install ` exited with code 1. 
3.303s total time elapsed.

pipelines_kill_background_processes
pstree -p $$  grep -E -o '([0-9]+)'  grep -v "^$$\$"  sort -k2 -rn

@justinlevi sorry for the slow responses here. a few thoughts.

I don't think you can actually "use" memcache inside of pipelines. I've seen some commentary recently that it's not there (but is coming soon). I don't have a lot of evidence to back this up, just a thought! This does align with your commentary above about it not being installed in the container (and that is NOT a BLT issue to solve).

I am currently enabling the memcache module on one of my projects during my pipelines builds and not experiencing the issues you are above. HOWEVER I am also not including any custom settings files. I'm worried that if you tried to customize the memcache approach that ships with BLT that could be the root of your problems. Can you remove whatever customizations you've added and see if things stop failing?

Thanks for the response. Yes, I reached that conclusion on a previous project using an older BLT version. Overriding the memcache.settings.php to NOT load via a simple conditional that checked the environment variables seemed to work previously.

Unfortunately I don’t know how to do that in BLT 9.2 given the loading order and file structure is a bit different.

Can you tell me how I would not load memcache as it seems to be happening out of the box. We are seeing this issue with a stock “out of the box” BLT 9.2 setup. Almost no customization at this point

so, that's what's confusing me. blt 9.2.3 has the logic built into it to only work if memcache is present in the container. there shouldn't be any overriding needed.

i'm curious, what did you do previously to override? are you sure that all of those customizations have been reset? it might make sense to run 'blt blt:init:settings' and ensure that there are no customizations in the settings files.

i'm currently running the memcache module on BLT 9.2.3, using pipelines (for continuous delivery) and having no issues deploying. the memcache module IS ON in my core.extensions.yml file, but aside from that, what is in blt is what i've got. no other customizations.

curious, when you are saying pipelines, are you talking about the continuous delivery or the on demand environment portion of the product?

Yeah, I’ve got the standard settings.php that comes with BLT 9.2.x - see below

Here's the stock settings.php file that I'm using:

``` 'databasename', * 'username' => 'sqlusername', * 'password' => 'sqlpassword', * 'host' => 'localhost', * 'port' => '3306', * 'driver' => 'mysql', * 'prefix' => '', * 'collation' => 'utf8mb4_general_ci', * ); * @endcode */ $databases = []; /** * Customizing database settings. * * Many of the values of the $databases array can be customized for your * particular database system. Refer to the sample in the section above as a * starting point. * * The "driver" property indicates what Drupal database driver the * connection should use. This is usually the same as the name of the * database type, such as mysql or sqlite, but not always. The other * properties will vary depending on the driver. For SQLite, you must * specify a database file name in a directory that is writable by the * webserver. For most other drivers, you must specify a * username, password, host, and database name. * * Transaction support is enabled by default for all drivers that support it, * including MySQL. To explicitly disable it, set the 'transactions' key to * FALSE. * Note that some configurations of MySQL, such as the MyISAM engine, don't * support it and will proceed silently even if enabled. If you experience * transaction related crashes with such configuration, set the 'transactions' * key to FALSE. * * For each database, you may optionally specify multiple "target" databases. * A target database allows Drupal to try to send certain queries to a * different database if it can but fall back to the default connection if not. * That is useful for primary/replica replication, as Drupal may try to connect * to a replica server when appropriate and if one is not available will simply * fall back to the single primary server (The terms primary/replica are * traditionally referred to as master/slave in database server documentation). * * The general format for the $databases array is as follows: * @code * $databases['default']['default'] = $info_array; * $databases['default']['replica'][] = $info_array; * $databases['default']['replica'][] = $info_array; * $databases['extra']['default'] = $info_array; * @endcode * * In the above example, $info_array is an array of settings described above. * The first line sets a "default" database that has one primary database * (the second level default). The second and third lines create an array * of potential replica databases. Drupal will select one at random for a given * request as needed. The fourth line creates a new database with a name of * "extra". * * You can optionally set prefixes for some or all database table names * by using the 'prefix' setting. If a prefix is specified, the table * name will be prepended with its value. Be sure to use valid database * characters only, usually alphanumeric and underscore. If no prefixes * are desired, leave it as an empty string ''. * * To have all database names prefixed, set 'prefix' as a string: * @code * 'prefix' => 'main_', * @endcode * * Per-table prefixes are deprecated as of Drupal 8.2, and will be removed in * Drupal 9.0. After that, only a single prefix for all tables will be * supported. * * To provide prefixes for specific tables, set 'prefix' as an array. * The array's keys are the table names and the values are the prefixes. * The 'default' element is mandatory and holds the prefix for any tables * not specified elsewhere in the array. Example: * @code * 'prefix' => array( * 'default' => 'main_', * 'users' => 'shared_', * 'sessions' => 'shared_', * 'role' => 'shared_', * 'authmap' => 'shared_', * ), * @endcode * You can also use a reference to a schema/database as a prefix. This may be * useful if your Drupal installation exists in a schema that is not the default * or you want to access several databases from the same code base at the same * time. * Example: * @code * 'prefix' => array( * 'default' => 'main.', * 'users' => 'shared.', * 'sessions' => 'shared.', * 'role' => 'shared.', * 'authmap' => 'shared.', * ); * @endcode * NOTE: MySQL and SQLite's definition of a schema is a database. * * Advanced users can add or override initial commands to execute when * connecting to the database server, as well as PDO connection settings. For * example, to enable MySQL SELECT queries to exceed the max_join_size system * variable, and to reduce the database connection timeout to 5 seconds: * @code * $databases['default']['default'] = array( * 'init_commands' => array( * 'big_selects' => 'SET SQL_BIG_SELECTS=1', * ), * 'pdo' => array( * PDO::ATTR_TIMEOUT => 5, * ), * ); * @endcode * * WARNING: The above defaults are designed for database portability. Changing * them may cause unexpected behavior, including potential data loss. See * https://www.drupal.org/developing/api/database/configuration for more * information on these defaults and the potential issues. * * More details can be found in the constructor methods for each driver: * - \Drupal\Core\Database\Driver\mysql\Connection::__construct() * - \Drupal\Core\Database\Driver\pgsql\Connection::__construct() * - \Drupal\Core\Database\Driver\sqlite\Connection::__construct() * * Sample Database configuration format for PostgreSQL (pgsql): * @code * $databases['default']['default'] = array( * 'driver' => 'pgsql', * 'database' => 'databasename', * 'username' => 'sqlusername', * 'password' => 'sqlpassword', * 'host' => 'localhost', * 'prefix' => '', * ); * @endcode * * Sample Database configuration format for SQLite (sqlite): * @code * $databases['default']['default'] = array( * 'driver' => 'sqlite', * 'database' => '/path/to/databasefilename', * ); * @endcode */ /** * Location of the site configuration files. * * The $config_directories array specifies the location of file system * directories used for configuration data. On install, the "sync" directory is * created. This is used for configuration imports. The "active" directory is * not created by default since the default storage for active configuration is * the database rather than the file system. (This can be changed. See "Active * configuration settings" below). * * The default location for the "sync" directory is inside a randomly-named * directory in the public files path. The setting below allows you to override * the "sync" location. * * If you use files for the "active" configuration, you can tell the * Configuration system where this directory is located by adding an entry with * array key CONFIG_ACTIVE_DIRECTORY. * * Example: * @code * $config_directories = array( * CONFIG_SYNC_DIRECTORY => '/directory/outside/webroot', * ); * @endcode */ $config_directories = []; /** * Settings: * * $settings contains environment-specific configuration, such as the files * directory and reverse proxy address, and temporary configuration, such as * security overrides. * * @see \Drupal\Core\Site\Settings::get() */ /** * Salt for one-time login links, cancel links, form tokens, etc. * * This variable will be set to a random value by the installer. All one-time * login links will be invalidated if the value is changed. Note that if your * site is deployed on a cluster of web servers, you must ensure that this * variable has the same value on each server. * * For enhanced security, you may set this variable to the contents of a file * outside your document root; you should also ensure that this file is not * stored with backups of your database. * * Example: * @code * $settings['hash_salt'] = file_get_contents('/home/example/salt.txt'); * @endcode */ $settings['hash_salt'] = ''; /** * Deployment identifier. * * Drupal's dependency injection container will be automatically invalidated and * rebuilt when the Drupal core version changes. When updating contributed or * custom code that changes the container, changing this identifier will also * allow the container to be invalidated as soon as code is deployed. */ # $settings['deployment_identifier'] = \Drupal::VERSION; /** * Access control for update.php script. * * If you are updating your Drupal installation using the update.php script but * are not logged in using either an account with the "Administer software * updates" permission or the site maintenance account (the account that was * created during installation), you will need to modify the access check * statement below. Change the FALSE to a TRUE to disable the access check. * After finishing the upgrade, be sure to open this file again and change the * TRUE back to a FALSE! */ $settings['update_free_access'] = FALSE; /** * External access proxy settings: * * If your site must access the Internet via a web proxy then you can enter the * proxy settings here. Set the full URL of the proxy, including the port, in * variables: * - $settings['http_client_config']['proxy']['http']: The proxy URL for HTTP * requests. * - $settings['http_client_config']['proxy']['https']: The proxy URL for HTTPS * requests. * You can pass in the user name and password for basic authentication in the * URLs in these settings. * * You can also define an array of host names that can be accessed directly, * bypassing the proxy, in $settings['http_client_config']['proxy']['no']. */ # $settings['http_client_config']['proxy']['http'] = 'http://proxy_user:proxy_pass@example.com:8080'; # $settings['http_client_config']['proxy']['https'] = 'http://proxy_user:proxy_pass@example.com:8080'; # $settings['http_client_config']['proxy']['no'] = ['127.0.0.1', 'localhost']; /** * Reverse Proxy Configuration: * * Reverse proxy servers are often used to enhance the performance * of heavily visited sites and may also provide other site caching, * security, or encryption benefits. In an environment where Drupal * is behind a reverse proxy, the real IP address of the client should * be determined such that the correct client IP address is available * to Drupal's logging, statistics, and access management systems. In * the most simple scenario, the proxy server will add an * X-Forwarded-For header to the request that contains the client IP * address. However, HTTP headers are vulnerable to spoofing, where a * malicious client could bypass restrictions by setting the * X-Forwarded-For header directly. Therefore, Drupal's proxy * configuration requires the IP addresses of all remote proxies to be * specified in $settings['reverse_proxy_addresses'] to work correctly. * * Enable this setting to get Drupal to determine the client IP from * the X-Forwarded-For header (or $settings['reverse_proxy_header'] if set). * If you are unsure about this setting, do not have a reverse proxy, * or Drupal operates in a shared hosting environment, this setting * should remain commented out. * * In order for this setting to be used you must specify every possible * reverse proxy IP address in $settings['reverse_proxy_addresses']. * If a complete list of reverse proxies is not available in your * environment (for example, if you use a CDN) you may set the * $_SERVER['REMOTE_ADDR'] variable directly in settings.php. * Be aware, however, that it is likely that this would allow IP * address spoofing unless more advanced precautions are taken. */ # $settings['reverse_proxy'] = TRUE; /** * Specify every reverse proxy IP address in your environment. * This setting is required if $settings['reverse_proxy'] is TRUE. */ # $settings['reverse_proxy_addresses'] = ['a.b.c.d', ...]; /** * Set this value if your proxy server sends the client IP in a header * other than X-Forwarded-For. */ # $settings['reverse_proxy_header'] = 'X_CLUSTER_CLIENT_IP'; /** * Set this value if your proxy server sends the client protocol in a header * other than X-Forwarded-Proto. */ # $settings['reverse_proxy_proto_header'] = 'X_FORWARDED_PROTO'; /** * Set this value if your proxy server sends the client protocol in a header * other than X-Forwarded-Host. */ # $settings['reverse_proxy_host_header'] = 'X_FORWARDED_HOST'; /** * Set this value if your proxy server sends the client protocol in a header * other than X-Forwarded-Port. */ # $settings['reverse_proxy_port_header'] = 'X_FORWARDED_PORT'; /** * Set this value if your proxy server sends the client protocol in a header * other than Forwarded. */ # $settings['reverse_proxy_forwarded_header'] = 'FORWARDED'; /** * Page caching: * * By default, Drupal sends a "Vary: Cookie" HTTP header for anonymous page * views. This tells a HTTP proxy that it may return a page from its local * cache without contacting the web server, if the user sends the same Cookie * header as the user who originally requested the cached page. Without "Vary: * Cookie", authenticated users would also be served the anonymous page from * the cache. If the site has mostly anonymous users except a few known * editors/administrators, the Vary header can be omitted. This allows for * better caching in HTTP proxies (including reverse proxies), i.e. even if * clients send different cookies, they still get content served from the cache. * However, authenticated users should access the site directly (i.e. not use an * HTTP proxy, and bypass the reverse proxy if one is used) in order to avoid * getting cached pages from the proxy. */ # $settings['omit_vary_cookie'] = TRUE; /** * Cache TTL for client error (4xx) responses. * * Items cached per-URL tend to result in a large number of cache items, and * this can be problematic on 404 pages which by their nature are unbounded. A * fixed TTL can be set for these items, defaulting to one hour, so that cache * backends which do not support LRU can purge older entries. To disable caching * of client error responses set the value to 0. Currently applies only to * page_cache module. */ # $settings['cache_ttl_4xx'] = 3600; /** * Expiration of cached forms. * * Drupal's Form API stores details of forms in a cache and these entries are * kept for at least 6 hours by default. Expired entries are cleared by cron. * * @see \Drupal\Core\Form\FormCache::setCache() */ # $settings['form_cache_expiration'] = 21600; /** * Class Loader. * * If the APC extension is detected, the Symfony APC class loader is used for * performance reasons. Detection can be prevented by setting * class_loader_auto_detect to false, as in the example below. */ # $settings['class_loader_auto_detect'] = FALSE; /* * If the APC extension is not detected, either because APC is missing or * because auto-detection has been disabled, auto-loading falls back to * Composer's ClassLoader, which is good for development as it does not break * when code is moved in the file system. You can also decorate the base class * loader with another cached solution than the Symfony APC class loader, as * all production sites should have a cached class loader of some sort enabled. * * To do so, you may decorate and replace the local $class_loader variable. For * example, to use Symfony's APC class loader without automatic detection, * uncomment the code below. */ /* if ($settings['hash_salt']) { $prefix = 'drupal.' . hash('sha256', 'drupal.' . $settings['hash_salt']); $apc_loader = new \Symfony\Component\ClassLoader\ApcClassLoader($prefix, $class_loader); unset($prefix); $class_loader->unregister(); $apc_loader->register(); $class_loader = $apc_loader; } */ /** * Authorized file system operations: * * The Update Manager module included with Drupal provides a mechanism for * site administrators to securely install missing updates for the site * directly through the web user interface. On securely-configured servers, * the Update manager will require the administrator to provide SSH or FTP * credentials before allowing the installation to proceed; this allows the * site to update the new files as the user who owns all the Drupal files, * instead of as the user the webserver is running as. On servers where the * webserver user is itself the owner of the Drupal files, the administrator * will not be prompted for SSH or FTP credentials (note that these server * setups are common on shared hosting, but are inherently insecure). * * Some sites might wish to disable the above functionality, and only update * the code directly via SSH or FTP themselves. This setting completely * disables all functionality related to these authorized file operations. * * @see https://www.drupal.org/node/244924 * * Remove the leading hash signs to disable. */ # $settings['allow_authorize_operations'] = FALSE; /** * Default mode for directories and files written by Drupal. * * Value should be in PHP Octal Notation, with leading zero. */ # $settings['file_chmod_directory'] = 0775; # $settings['file_chmod_file'] = 0664; /** * Public file base URL: * * An alternative base URL to be used for serving public files. This must * include any leading directory path. * * A different value from the domain used by Drupal to be used for accessing * public files. This can be used for a simple CDN integration, or to improve * security by serving user-uploaded files from a different domain or subdomain * pointing to the same server. Do not include a trailing slash. */ # $settings['file_public_base_url'] = 'http://downloads.example.com/files'; /** * Public file path: * * A local file system path where public files will be stored. This directory * must exist and be writable by Drupal. This directory must be relative to * the Drupal installation directory and be accessible over the web. */ # $settings['file_public_path'] = 'sites/default/files'; /** * Private file path: * * A local file system path where private files will be stored. This directory * must be absolute, outside of the Drupal installation directory and not * accessible over the web. * * Note: Caches need to be cleared when this value is changed to make the * private:// stream wrapper available to the system. * * See https://www.drupal.org/documentation/modules/file for more information * about securing private files. */ # $settings['file_private_path'] = ''; /** * Session write interval: * * Set the minimum interval between each session write to database. * For performance reasons it defaults to 180. */ # $settings['session_write_interval'] = 180; /** * String overrides: * * To override specific strings on your site with or without enabling the Locale * module, add an entry to this list. This functionality allows you to change * a small number of your site's default English language interface strings. * * Remove the leading hash signs to enable. * * The "en" part of the variable name, is dynamic and can be any langcode of * any added language. (eg locale_custom_strings_de for german). */ # $settings['locale_custom_strings_en'][''] = [ # 'forum' => 'Discussion board', # '@count min' => '@count minutes', # ]; /** * A custom theme for the offline page: * * This applies when the site is explicitly set to maintenance mode through the * administration page or when the database is inactive due to an error. * The template file should also be copied into the theme. It is located inside * 'core/modules/system/templates/maintenance-page.html.twig'. * * Note: This setting does not apply to installation and update pages. */ # $settings['maintenance_theme'] = 'bartik'; /** * PHP settings: * * To see what PHP settings are possible, including whether they can be set at * runtime (by using ini_set()), read the PHP documentation: * http://php.net/manual/ini.list.php * See \Drupal\Core\DrupalKernel::bootEnvironment() for required runtime * settings and the .htaccess file for non-runtime settings. * Settings defined there should not be duplicated here so as to avoid conflict * issues. */ /** * If you encounter a situation where users post a large amount of text, and * the result is stripped out upon viewing but can still be edited, Drupal's * output filter may not have sufficient memory to process it. If you * experience this issue, you may wish to uncomment the following two lines * and increase the limits of these variables. For more information, see * http://php.net/manual/pcre.configuration.php. */ # ini_set('pcre.backtrack_limit', 200000); # ini_set('pcre.recursion_limit', 200000); /** * Active configuration settings. * * By default, the active configuration is stored in the database in the * {config} table. To use a different storage mechanism for the active * configuration, do the following prior to installing: * - Create an "active" directory and declare its path in $config_directories * as explained under the 'Location of the site configuration files' section * above in this file. To enhance security, you can declare a path that is * outside your document root. * - Override the 'bootstrap_config_storage' setting here. It must be set to a * callable that returns an object that implements * \Drupal\Core\Config\StorageInterface. * - Override the service definition 'config.storage.active'. Put this * override in a services.yml file in the same directory as settings.php * (definitions in this file will override service definition defaults). */ # $settings['bootstrap_config_storage'] = ['Drupal\Core\Config\BootstrapConfigStorageFactory', 'getFileStorage']; /** * Configuration overrides. * * To globally override specific configuration values for this site, * set them here. You usually don't need to use this feature. This is * useful in a configuration file for a vhost or directory, rather than * the default settings.php. * * Note that any values you provide in these variable overrides will not be * viewable from the Drupal administration interface. The administration * interface displays the values stored in configuration so that you can stage * changes to other environments that don't have the overrides. * * There are particular configuration values that are risky to override. For * example, overriding the list of installed modules in 'core.extension' is not * supported as module install or uninstall has not occurred. Other examples * include field storage configuration, because it has effects on database * structure, and 'core.menu.static_menu_link_overrides' since this is cached in * a way that is not config override aware. Also, note that changing * configuration values in settings.php will not fire any of the configuration * change events. */ # $config['system.file']['path']['temporary'] = '/tmp'; # $config['system.site']['name'] = 'My Drupal site'; # $config['system.theme']['default'] = 'stark'; # $config['user.settings']['anonymous'] = 'Visitor'; /** * Fast 404 pages: * * Drupal can generate fully themed 404 pages. However, some of these responses * are for images or other resource files that are not displayed to the user. * This can waste bandwidth, and also generate server load. * * The options below return a simple, fast 404 page for URLs matching a * specific pattern: * - $config['system.performance']['fast_404']['exclude_paths']: A regular * expression to match paths to exclude, such as images generated by image * styles, or dynamically-resized images. The default pattern provided below * also excludes the private file system. If you need to add more paths, you * can add '|path' to the expression. * - $config['system.performance']['fast_404']['paths']: A regular expression to * match paths that should return a simple 404 page, rather than the fully * themed 404 page. If you don't have any aliases ending in htm or html you * can add '|s?html?' to the expression. * - $config['system.performance']['fast_404']['html']: The html to return for * simple 404 pages. * * Remove the leading hash signs if you would like to alter this functionality. */ # $config['system.performance']['fast_404']['exclude_paths'] = '/\/(?:styles)|(?:system\/files)\//'; # $config['system.performance']['fast_404']['paths'] = '/\.(?:txt|png|gif|jpe?g|css|js|ico|swf|flv|cgi|bat|pl|dll|exe|asp)$/i'; # $config['system.performance']['fast_404']['html'] = '404 Not Found

Not Found

The requested URL "@path" was not found on this server.

'; /** * Load services definition file. */ $settings['container_yamls'][] = $app_root . '/' . $site_path . '/services.yml'; /** * Override the default service container class. * * This is useful for example to trace the service container for performance * tracking purposes, for testing a service container with an error condition or * to test a service container that throws an exception. */ # $settings['container_base_class'] = '\Drupal\Core\DependencyInjection\Container'; /** * Override the default yaml parser class. * * Provide a fully qualified class name here if you would like to provide an * alternate implementation YAML parser. The class must implement the * \Drupal\Component\Serialization\SerializationInterface interface. */ # $settings['yaml_parser_class'] = NULL; /** * Trusted host configuration. * * Drupal core can use the Symfony trusted host mechanism to prevent HTTP Host * header spoofing. * * To enable the trusted host mechanism, you enable your allowable hosts * in $settings['trusted_host_patterns']. This should be an array of regular * expression patterns, without delimiters, representing the hosts you would * like to allow. * * For example: * @code * $settings['trusted_host_patterns'] = array( * '^www\.example\.com$', * ); * @endcode * will allow the site to only run from www.example.com. * * If you are running multisite, or if you are running your site from * different domain names (eg, you don't redirect http://www.example.com to * http://example.com), you should specify all of the host patterns that are * allowed by your site. * * For example: * @code * $settings['trusted_host_patterns'] = array( * '^example\.com$', * '^.+\.example\.com$', * '^example\.org$', * '^.+\.example\.org$', * ); * @endcode * will allow the site to run off of all variants of example.com and * example.org, with all subdomains included. */ /** * The default list of directories that will be ignored by Drupal's file API. * * By default ignore node_modules and bower_components folders to avoid issues * with common frontend tools and recursive scanning of directories looking for * extensions. * * @see file_scan_directory() * @see \Drupal\Core\Extension\ExtensionDiscovery::scanDirectory() */ $settings['file_scan_ignore_directories'] = [ 'node_modules', 'bower_components', ]; /** * The default number of entities to update in a batch process. * * This is used by update and post-update functions that need to go through and * change all the entities on a site, so it is useful to increase this number * if your hosting configuration (i.e. RAM allocation, CPU speed) allows for a * larger number of entities to be processed in a single batch run. */ $settings['entity_update_batch_size'] = 50; /** * Load local development override configuration, if available. * * Use settings.local.php to override variables on secondary (staging, * development, etc) installations of this site. Typically used to disable * caching, JavaScript/CSS compression, re-routing of outgoing emails, and * other things that should not happen on development and testing sites. * * Keep this code block at the end of this file to take full effect. */ # # if (file_exists($app_root . '/' . $site_path . '/settings.local.php')) { # include $app_root . '/' . $site_path . '/settings.local.php'; # } require DRUPAL_ROOT . "/../vendor/acquia/blt/settings/blt.settings.php"; ```

Memcache still doesn't seem to work though??

The conditional I was using on a BLT 8.9x site is above using the PIPELINE_ENV variable to conditionally load the memcache.settings.php file. In 8.9.x, or at least in the old site I was working on, all of the settings files were included directly from the settings.php file, now they are all included from the vendor BLT.settings.php file.

This allowed me to override if memcache would get loaded via the conditional:

// TEMPORARY HACK UNTIL MEMCACHE IS INSTALLED ON PIPELINES
if(!isset($_ENV['PIPELINE_ENV'])){
...
}

I’ve found the conditional that checks if the environment is pipelines in the vendor code it just doesn't seem to be working.

https://github.com/acquia/blt/blob/d737f52c6374492ccc237886cb6322edbfbb3526/settings/cache.settings.php#L38

I’m wondering if the environment variables have recently changed in the container?

I’m just trying to get past the Drupal install step in the standard acquia-pipelines.yml file.

I’ll try and do a printenv in the container later to see what I get.This is almost a stock BLT generated project so I’m pretty sure there is an issue somewhere.

Would be great if the pipelines team could get memcache installed though. I can do this on Travis no problem

Update, went through the following script and pipelines didn't throw the memcache error. This tells me that the issue is somewhere in the installation profile I'm loading.

$ composer create-project --no-interaction acquia/blt-project my-project
$ cd my-project
$ blt blt:init:settings
$ blt recipes:ci:pipelines:init

// Note, I already have pipelines wired up to the github repo
$ git remote add origin git@github.com:ny/labor.ny.gov.git
$ git checkout -b blt-out-of-the-box-test
$ git push --set-upstream origin blt-out-of-the-box-test

Here's the screen capture video of the entire process if anyone is interested: http://justinleviwinter.s3.amazonaws.com/blt-new-project-pipelines-working.mp4

screen shot 2019-02-17 at 11 42 25 am

The only difference that I can see between this "out-of-the-box" codebase and the one that was throwing the error is that the one with the memcache error is requiring the memcache module in the profile composer.json file via:

"drupal/memcache": "2.0-alpha7"

I'm going to try and duplicate the issue by requiring that module in the project root composer.json file.

Update: Nope... requiring the module has no impact. Pipelines still passes.

Next up - I'll try and do a visual comparison of the code base and slowly add pieces back to the one that is passing.

The issue was the our custom profile was requiring memcache as a dependency. I saw that lightning doesn't do this so I went ahead and removed the line from our [profile].info.yml file

removing:

dependencies:
...
  - memcache
...

After updating the site's composer.lock file, seems that the memcache error went away.

acquia / blt

BLT Failing on Pipelines - Missing memcache php extension #3343

Not Found

Memcache still doesn't seem to work though??