Payment settings disappears on deploy routine [braintree, paypal]

[peec]

Magento version: 2.0.7 PHP version: 7.0.3

Every time i deploy payment configuration from sales tabs is flushed. And customers can NOT use payment methods.

The deployment routine is basically this:

1. Run setup:install so all instances ( i have multiple support ) get the same db details etc. Is this the problem ? .. Should be possible to run setup:install multiple times according to docs .
2. Reindex, clear cache, setup upgrade etc.

Here is the exact script i run on every deploy to keep my install consistent.

echo "Running magento setup:install"
/src/bin/magento setup:install \
      --db-host="${RDS_HOSTNAME}" \
      --db-name="${RDS_DB_NAME}" \
      --db-user="${RDS_USERNAME}" \
      --db-password="${RDS_PASSWORD}" \
      --backend-frontname="${MAGENTO_BACKEND_FRONTNAME}" \
      --language="${MAGENTO_LANGUAGE}" \
      --timezone="${MAGENTO_TIMEZONE}" \
      --currency="${MAGENTO_CURRENCY}" \
      --admin-lastname="${MAGENTO_ADMIN_LASTNAME}"\
      --admin-firstname="${MAGENTO_ADMIN_FIRSTNAME}" \
      --admin-email="${MAGENTO_ADMIN_EMAIL}" \
      --admin-user="${MAGENTO_ADMIN_USERNAME}" \
      --admin-password="${MAGENTO_ADMIN_PASSWORD}" \
      --use-rewrites="${MAGENTO_USE_REWRITES}"

echo "Setting asset minification defaults"
mysql -u "${RDS_USERNAME}" -p"${RDS_PASSWORD}" -h "${RDS_HOSTNAME}" -P "${RDS_PORT}" "${RDS_DB_NAME}" -e "REPLACE INTO core_config_data (value, path) VALUES \
                    ('${WEBSITE_SECURE_URL}', 'web/secure/base_url'), ('${WEBSITE_UNSECURE_URL}', 'web/unsecure/base_url'), \
                    ('', 'web/secure/base_static_url'), ('', 'web/unsecure/base_static_url'), \
                    ('0',  'dev/js/merge_files'), ('0', 'dev/js/minify_files'), ('0', 'dev/js/enable_js_bundling'), \
                    ('1',  'dev/css/merge_files'), ('0', 'dev/css/minify_files'), \
                    ('0', 'dev/template/minify_html');"

echo "Running full reindex"
/src/bin/magento indexer:reindex

echo "Clearing cache"
/src/bin/magento cache:clean
/src/bin/magento cache:flush

echo "Setup upgrade"
/src/bin/magento setup:upgrade

echo "DI Compliation"
/src/bin/magento setup:di:compile 

echo "Deploying static files"
chown -R www-data:www-data /src
chmod -R 0777 /src
/src/bin/magento setup:static-content:deploy "${MAGENTO_LANGUAGE} en_US"

[peec]

After further investigation it seems that the encrypted fields that disappears after deploying.

Affected Braintree values that gets removed:

• Public Key
• Private key

Affected PayPal values that gets removed ( Website Payments Standard and Express Checkout):

• API Username
• API Password
• API Signature

All other non encrypted information stays.

[joni-jones]

Hi, @peec. We can't reproduce described issue. Please, attach apache and magento logs. Also, if you use additional extensions, try to disable them and repeat deployment.

[m0zziter]

I am seeing the exact same symptoms and we're also using Braintree and a similar-ish deploy script. Typically what I see is the Public Key and Private Key are blank (as @peec mentioned) but there's still a green checkmark next to it in the list of payment methods.

Although we are now running PHP 7.0.3 and Magento 2.0.7 I experienced this issue on PHP 5.6 and all other versions of Magento 2.

[peec]

Maybe relevant information, but we only version control composer.json and composer.lock for each Magento 2 site. We use memcache for backend cache and varnish. Media is stored in database. So basically the filesystem does not store anything.

So basically, all files are deleted per deploy.

On a deploy we:

• Push up composer.json with composer.lock
• Run composer install with related magento bin/setup routine to configure database.
• Maybe somehow Magento 2 thinks it's a fresh install?

[joni-jones]

All payments credentials are stored in the database. Also in my previous comment, I'm asked about logs and additional extensions.

[m0zziter]

I was able to reproduce this on a fresh installation of Magento deployed with zero extensions or themes installed.

Some notes about the environment: First, it's docker based. A docker-compose file spins up nginx, redis and php containers. Then Magento is installed via composer and configured to use an external MySQL database (in our case, this lives in RDS). Similar to @peec we only include the composer.json and composer.lock files when deploying the application.

Now onto how we replicated the issue.

First we deployed a new Magento instance to a blank database. After everything fired up we configured Braintree and were able to submit test orders as expected. Not much else to say at this point.

Next, we stopped the docker containers and then started them back up. This means the containers were started with all the application files present (vs the first time when it was just composer.json/composer.lock). At this point we expect to be able go to a product page, add it to our cart and submit an order using Braintree as the payment method. And indeed that is what happened.

Lastly, we stopped the docker containers and then deleted all application files except for the composer.json and composer.lock files then relaunched the docker containers. As with the previous step we expect to go right to a product page, add the product to our cart and submit an order using Braintree. Once we go to the Review/Payments step of checkout, we do not see Braintree listed. But instead an error in the browser's console that reads cc-form.js:159 Uncaught TypeError: Cannot read property 'addErrorMessage' of undefined. So we go into the Payment Methods section in the Magento admin section and sure enough the Public Key and Private Key are missing and Enable this Solution is set to No.

If needed here's a docker-compose file that's very similar to ours: https://github.com/mageinferno/magento2-docker-compose

[joni-jones]

@m0zziter did you use 2.1 or 2.0.5 (from linked compose file)?

[m0zziter]

2.0.7 but I have seen this behavior on all 2.0.x versions of Magento.

[rplant09485]

I had some success, with the following: For some reason the config value for path=payment/payflow_advanced/user Which should be encrypted but has the unencrypted word PayPal in it (Set I don't know where, not in the db) and it gets a very dubious decrypted value. On my system this path did not exist in the core_config_data table. So I set the paypal api name to PayPal, then using phpmysql copied this (encrypted) entry to a new entry with a path of payment/payflow_advanced/user Then reset the paypal api settings to what they should be, and it worked (2.0.7) ymmv

[joni-jones]

@rplant09485, the partner field (PayPal value), should be not encrypted because it's not sensitive data.

[borishuseincehajic]

HI,

Having similar issue with the braintree payment extension but the problem is that the payment is not appearing at all on the checkout page. Just saying no payment methods available.

I tried merge, unmerge js, tried moving braintree.js and utils to the footer (some said it cannot target container because it didn't rendered yet) with no success. Currencies are the same in sandbox and shop, I checked Public and Private keys multiple times as well as Merchant account ID.

Only error that shows in console is:

Uncaught Error: Unable to find an input with data-braintree-name="number" in your form. Please add one.

Using magento 1.9.20

[joni-jones]

Hi, @borishuseincehajic, this Github repo only for Magento 2. The Magento 2.1 uses new implementation of Braintree, which has full reworked frontend and backend code.

[borishuseincehajic]

Ahh got it. Sorry about that. Thnx for the info.

[m0zziter]

@joni-jones I was playing around with 2.1 today (we're still using 2.0.7 in production) and experienced the same issue. Were you able to replicate? Do you need additional information?

Honestly this issue is a huge drag and is preventing us from using multiple webnodes so we'd like to get it resolved. Let me know if there's anything I can do to help that process along I'd be happy to oblige.

[joni-jones]

Hi @m0zziter, previously, I have tried to reproduce this issue. I had upgraded Magento from 2.0.4 to 2.0.7 and 2.1, the configuration doesn't disappear. I hadn't enough time to use you docker-compose and suppose will have time only after current sprint will be finished.

[peec]

FYI, still reproduced in 2.1. Even re-installed a couple of times.

The encrypted fields are removed ( Braintree private and secret keys ) on deploy . Something seems to be wrong with the encryption of fields it seems. It doesn't persist on redeploy.

[peec]

I think i solved the case now, had to read some code to figure this out. This is related to app/etc/env.php and the Encrypter relying on the filesystem.

As one only version control composer.lock and composer.json, env.php will be generated on each deploy and each physical server node being allocated to magento.

app/etc/env.php

<?php
return array (
 //.....
  'crypt' => 
  array (
    'key' => 'zzzzzzzzzzzzzzzz',
  ),
 //.....
);

The crypt/key has to stay the same on ALL nodes, and must never change. Changing it in a production system will make all the fields that are of type Magento\Config\Model\Config\Backend\Encrypted be deleted. Since the decryption allgorithm doesn't work when the crypt key is not equal to what the string was originally encrypted with.

See

• https://github.com/magento/magento2/blob/develop/app/code/Magento/Braintree/etc/config.xml#L38
• https://github.com/magento/magento2/blob/develop/lib/internal/Magento/Framework/Encryption/Encryptor.php#L278
• https://github.com/magento/magento2/blob/develop/lib/internal/Magento/Framework/Encryption/Crypt.php#L83

How to fix:

• Clearly tell in documentation that crypt/key must not be changed once going to production. Encryption algorithms will then blank all the originally filled fields.
• When using deployment mechanisms that only contains composer.lock and composer.json. Remember to always use the same crypt.key inside env.php.

[m0zziter]

@peec That was definitely the cause of it! I tweaked my deployment script to specify an encryption key and sure enough the Braintree settings stuck around after deployment completed.

Thanks so much for your leg work on this one!

[alena-marchenko]

Hi @peec

We've created MAGETWO-57756 internal ticket for updating the documentation. Thank you.

[magento-engcom-team]

Hi @peec starting from 2.2.0 release you can share config.php file between instances and this file may contain all configurations you need to share. Configuration mechanism and options were significantly re-worked in 2.2.0 and several useful features have been added.