Closed flovntp closed 1 year ago
Thanks so much for this!
The problem is that in a late PR that went into v1.20.0, ddev get
does a start before proceeding, which means there's no easy way for you to to this as an all-in-one.
I've been exploring this issue elsewhere and ddev may need a pre-start section where this kind of check takes place. It's also quite likely that the start-on-get behavior can be rolled back.
In the meantime, I'd love to have you proceed -
ddev config
)ddev delete -Oy
Thanks so much for testing it! And thanks for making this issue, which was discovered in discord the other day as well.
@flovntp thanks for helping with this issue, and thanks for giving me access to your project on Platform.sh.
I need you to give me access to your gitlab project as well, gitlab.com/flovntp/pilatus, or I can't check out the code.
Thanks!
@rfay do you have a gitlab.com account ? i send an invite with your PSH address (randy.fay@platform.sh), let me know if you need to use another email address :wink:
@rfay I still have an error :
Network ddev-vntp_default Created
Container ddev-vntp-dba Started
Container ddev-vntp-db Started
Container ddev-vntp-db Healthy
Container ddev-vntp-web Started
Container ddev-router Started
Instrumentation is opted in, but SegmentKey is not available. This usually means you have a locally-built ddev binary or one from a PR build. It's not an error. Please report it if you're using an official release build.
Using existing PLATFORMSH_CLI_TOKEN.
Using existing PLATFORM_PROJECT from project config.yaml.
Using existing PLATFORM_ENVIRONMENT from project config.yaml.
local vntp-mariadb
mariadb_10.4
Current db is mariadb:10.4 (mariadb_10.4)
New db is mariadb:10.4
database versions match: mariadb:10.4=mariadb:10.4
could not process pre-install action '#ddev-nodisplay
# set -x
platform_routes=$(cat <<-ENDROUTES
{
"${DDEV_PRIMARY_URL}": {
"primary": true,
"id": null,
"production_url": "${DDEV_PRIMARY_URL}",
"attributes": {},
"upstream": "drupal",
"type": "upstream",
"original_url": "https://{default}/" }
}
ENDROUTES
)
cat <<-EOF >.ddev/config.platformsh.yaml
# #ddev-generated
# Generated configuration based on platform.sh project configuration
{{ $dbheader := index (split ":" .platformapp.relationships.database) "_0" }}
{{ $dbtype := replace "postgresql" "postgres" (get (get .services $dbheader) "type") }}
php_version: {{ trimPrefix "php:" .platformapp.type }}
database:
type: {{ regexReplaceAll "oracle-" (regexReplaceAll ":.*$" $dbtype "") "" }}
version: {{ regexReplaceAll "^.*:" $dbtype "" }}
docroot: {{ dig "web" "locations" "/" "root" "notfound" .platformapp }}
web_environment:
# TODO: Review which of these matters and which can be dummied up
- "PLATFORM_MOUNTS={{ range $key, $value := .platformapp.mounts }}{{ $key }} {{ end }}"
- "PLATFORM_APP_DIR=/var/www/html"
- PLATFORM_PROJECT_ENTROPY=$(echo $RANDOM | shasum -a 256 | awk '{print $1}')
# Consider commit hash for PLATFORM_TREE_ID
- "PLATFORM_TREE_ID=2dc356f2fea13ef683f9adc5fc5bd28e05ad992a"
- "PLATFORM_DIR=/var/www/html"
- "PLATFORM_ROUTES=$( if base64 --version >/dev/null 2>&1; then echo -n ${platform_routes} | base64 -w0; else echo -n ${platform_routes} | base64; fi)"
- PLATFORM_VARIABLES=e30=
- PATH=$PATH:/var/www/html/.global/bin
hooks:
post-start:
{{ if eq .platformapp.build.flavor "composer" }}
- composer: install
{{ end }}
# TODO: Separate the hooks into separate exec lines
{{ if .platformapp.hooks.build }}
# platformsh build hooks
- exec: '{{ trimAll "\n" .platformapp.hooks.build | replace "\n\n" "\n" | splitList "\n" | join ` && ` }}'
{{ end }}
# TODO: Separate the hooks into separate exec lines
{{ if .platformapp.hooks.deploy }}
# platformsh deploy hooks
- exec: '{{ trimAll "\n" .platformapp.hooks.deploy | replace "\n\n" "\n" | splitList "\n" | join ` && ` }}'
{{ end }}
{{ if .platformapp.hooks.post_deploy }}
# platformsh post_deploy hooks
- exec: '{{ trimAll "\n" .platformapp.hooks.post_deploy | replace "\n\n" "\n" | splitList "\n" | join ` && ` }}'
{{ end }}
EOF
': Unable to run action set -eu -o pipefail
#ddev-nodisplay
# set -x
platform_routes=$(cat <<-ENDROUTES
{
"${DDEV_PRIMARY_URL}": {
"primary": true,
"id": null,
"production_url": "${DDEV_PRIMARY_URL}",
"attributes": {},
"upstream": "drupal",
"type": "upstream",
"original_url": "https://{default}/" }
}
ENDROUTES
)
cat <<-EOF >.ddev/config.platformsh.yaml
# #ddev-generated
# Generated configuration based on platform.sh project configuration
php_version: 7.4
database:
type: mariadb
version: 10.4
docroot: public
web_environment:
# TODO: Review which of these matters and which can be dummied up
- "PLATFORM_MOUNTS=config/graphql/types/ezplatform public/var var/cache var/encore var/log var/sessions "
- "PLATFORM_APP_DIR=/var/www/html"
- PLATFORM_PROJECT_ENTROPY=$(echo $RANDOM | shasum -a 256 | awk '{print $1}')
# Consider commit hash for PLATFORM_TREE_ID
- "PLATFORM_TREE_ID=2dc356f2fea13ef683f9adc5fc5bd28e05ad992a"
- "PLATFORM_DIR=/var/www/html"
- "PLATFORM_ROUTES=$( if base64 --version >/dev/null 2>&1; then echo -n ${platform_routes} | base64 -w0; else echo -n ${platform_routes} | base64; fi)"
- PLATFORM_VARIABLES=e30=
- PATH=$PATH:/var/www/html/.global/bin
hooks:
post-start:
# TODO: Separate the hooks into separate exec lines
# platformsh build hooks
- exec: 'set -e && # Install newer version of Node.js, as current default (v6LTS) does not play well with WebpackEncore && unset NPM_CONFIG_PREFIX && curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.8/install.sh | dash && export NVM_DIR="$HOME/.nvm" && [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" && nvm current && nvm install 12 && if [ -z "$COMPOSER_AUTH" ]; then && echo "TIP: If you need to authenticate against Github/Gitlab/updates.ez.no, use COMPOSER_AUTH env variable" && echo "See: https://docs.platform.sh/guides/general/composer-auth.html#set-the-envcomposer_auth-project-variable" && fi && composer install --no-dev --prefer-dist --no-progress --no-interaction --optimize-autoloader'
# TODO: Separate the hooks into separate exec lines
# platformsh deploy hooks
- exec: 'set -e && unset NPM_CONFIG_PREFIX && command -v nvm && nvm use 12 && # Mainly relevant for eZ Platform demo usage, for own setup adapt this or remove and rely on migrations. && if [ ! -f public/var/.platform.installed ]; then && # To workaround issues with p.sh Varnish we clear container cache & temporary set Symfony Proxy && rm -Rf var/cache/$APP_ENV/*.* && SKIP_HTTPCACHE_PURGE="1" php -d memory_limit=-1 `which composer` && touch public/var/.platform.installed && fi && # Now that mounts are available, clear cache on mount. && # Note: Skip on PE Cluster setup using e.g. "if [$PLATFORM_BRANCH" != 'production']; then" & get p.sh to enable this on internal per node "pre_start" hook && sh bin/platformsh_prestart_cacheclear.sh && # If you also need to clear Redis cache on every deploy, you can either use this command or redis-cli && # Normally this should only be needed if cached data structures changes (upgrades), or you change data via sql (e.g. restore backup) && ##php bin/console cache:pool:clear cache.redis && # Example of additional deploy hooks if you use doctrine and/or kaliop migration bundle && ##php bin/console doctrine:migrations:migrate --no-interaction --allow-no-migration && php bin/console kaliop:migration:migrate --no-interaction --no-debug && # When using Solr, there are two cases where you'll need to rebuild indexes: && # - When Solr / search configuration changes && # - On database import/restore && # So in development it might be convenient to rebuild indexes, slowing down deploy time && ##php bin/console ezplatform:reindex --processes=auto && # When using Varnish/Fastly, HttpCache is not cleared when clearing Symfony Cache folder above, you need to && # clear cache yourself when for instance templates or config affecting responses change, for instance with: && ##bin/console fos:httpcache:invalidate:tag ez-all && # Depending on your VCL, buy default this would trigger a soft purge (expiry) and allow grace period, however && # even so if your change only affected certain subset of content, ideally you should only clear specific tags: && ##bin/console fos:httpcache:invalidate:tag l44 c33 ct2 ...'
EOF
: exit status 1, output=bash: line 17: NVM_DIR: unbound variable
Thanks so much @flovntp - I was able to use my gitlab account to get that, just had to add my platform.sh account to it. Now I'll be able to try out your situation.
The work I've done for your situation is in a PR, https://github.com/platformsh/ddev-platformsh/pull/25, but it does require the current HEAD of ddev. But now that I can try out your project we'll be able to make some progress.
Thanks!
Well, there's progress, but a number of problems, and this is a great example to work on because it has much more exotic things like trying to install nvm itself, etc.
I first tried to install the default branch, "main", but main doesn't have platform stuff. Then I tried the "master" branch, and made more progress with that.
rfay@rfay-mbp-2021:~/workspace/vntp$ ddev get ../ddev-platformsh/
Using existing PLATFORMSH_CLI_TOKEN.
Using existing PLATFORM_PROJECT from project config.yaml.
Using existing PLATFORM_ENVIRONMENT from project config.yaml.
# #ddev-generated
Installed file /Users/rfay/workspace/vntp/.ddev/web-build/Dockerfile.platformsh
Installed file /Users/rfay/workspace/vntp/.ddev/homeadditions/.bashrc.d/platformsh-environment.sh
Installed file /Users/rfay/workspace/vntp/.ddev/providers/platform.yaml
Installed file /Users/rfay/.ddev/commands/web/platform
Downloaded add-on ../ddev-platformsh/, use `ddev restart` to enable.
rfay@rfay-mbp-2021:~/workspace/vntp$ ddev restart
Restarting project vntp...
Container ddev-vntp-dba Removed
Container ddev-vntp-web Removed
Container ddev-vntp-db Removed
Network ddev-vntp_default Removed
Network ddev-vntp_default Created
Container ddev-vntp-dba Started
Container ddev-vntp-web Started
Container ddev-vntp-db Started
Container ddev-router Running
nvm is not compatible with the npm config "prefix" option: currently set to "/mnt/ddev-global-cache/nvm_dir/vntp-web/versions/node/v10.15.3"
Run `npm config delete prefix` or `nvm use --delete-prefix v10.15.3 --silent` to unset it.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 12540 100 12540 0 0 62388 0 --:--:-- --:--:-- --:--:-- 62388
=> nvm is already installed in /home/rfay/.nvm, trying to update using git
=> => Compressing and cleaning up git repository
=> Appending nvm source string to /home/rfay/.profile
=> bash_completion source string already in /home/rfay/.profile
nvm is not compatible with the npm config "prefix" option: currently set to "/mnt/ddev-global-cache/nvm_dir/vntp-web/versions/node/v10.15.3"
Run `npm config delete prefix` or `nvm use --delete-prefix v10.15.3 --silent` to unset it.
npm WARN config global `--global`, `--local` are deprecated. Use `--location=global` instead.
=> You currently have modules installed globally with `npm`. These will no
=> longer be linked to the active version of Node when you install a new node
=> with `nvm`; and they may (depending on how you construct your `$PATH`)
=> override the binaries of modules installed with `nvm`:
/usr/lib
├── corepack@0.10.0
├── gulp-cli@2.3.0
└── yarn@1.22.19
=> If you wish to uninstall them at a later point (or re-install them under your
=> `nvm` Nodes), you can remove them from the system Node as follows:
$ nvm use system
$ npm uninstall -g a_module
=> Close and reopen your terminal to start using nvm or run the following to use it now:
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
nvm is not compatible with the npm config "prefix" option: currently set to "/mnt/ddev-global-cache/nvm_dir/vntp-web/versions/node/v10.15.3"
Run `npm config delete prefix` or `nvm use --delete-prefix v10.15.3 --silent` to unset it.
Task failed: Exec command 'set -e
# Install newer version of Node.js, as current default (v6LTS) does not play well with WebpackEncore
unset NPM_CONFIG_PREFIX
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.8/install.sh | dash
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
nvm current
nvm install 10.15.3
if [ -z "$COMPOSER_AUTH" ]; then
echo "TIP: If you need to authenticate against Github/Gitlab/updates.ez.no, use COMPOSER_AUTH env variable"
echo "See: https://docs.platform.sh/guides/general/composer-auth.html#set-the-envcomposer_auth-project-variable"
fi
composer install --no-dev --prefer-dist --no-progress --no-interaction --optimize-autoloader
' in container/service 'web': exit status 11
A task failure does not mean that ddev failed, but your hook configuration has a command that failed.
nvm is not compatible with the npm config "prefix" option: currently set to "/mnt/ddev-global-cache/nvm_dir/vntp-web/versions/node/v10.15.3"
Run `npm config delete prefix` or `nvm use --delete-prefix v10.15.3 --silent` to unset it.
nvm
nvm is not compatible with the npm config "prefix" option: currently set to "/mnt/ddev-global-cache/nvm_dir/vntp-web/versions/node/v10.15.3"
Run `npm config delete prefix` or `nvm use --delete-prefix v10.15.3` to unset it.
Task failed: Exec command 'set -e
unset NPM_CONFIG_PREFIX
command -v nvm && nvm use 10.15.3
# Mainly relevant for eZ Platform demo usage, for own setup adapt this or remove and rely on migrations.
if [ ! -f public/var/.platform.installed ]; then
# To workaround issues with p.sh Varnish we clear container cache & temporary set Symfony Proxy
rm -Rf var/cache/$APP_ENV/*.*
SKIP_HTTPCACHE_PURGE="1" php -d memory_limit=-1 `which composer` ezplatform-install
touch public/var/.platform.installed
fi
# Now that mounts are available, clear cache on mount.
# Note: Skip on PE Cluster setup using e.g. "if [$PLATFORM_BRANCH" != 'production']; then" & get p.sh to enable this on internal per node "pre_start" hook
sh bin/platformsh_prestart_cacheclear.sh
# If you also need to clear Redis cache on every deploy, you can either use this command or redis-cli
# Normally this should only be needed if cached data structures changes (upgrades), or you change data via sql (e.g. restore backup)
##php bin/console cache:pool:clear cache.redis
# Example of additional deploy hooks if you use doctrine and/or kaliop migration bundle
##php bin/console doctrine:migrations:migrate --no-interaction --allow-no-migration
##php bin/console kaliop:migration:migrate --no-interaction --no-debug
# When using Solr, there are two cases where you'll need to rebuild indexes:
# - When Solr / search configuration changes
# - On database import/restore
# So in development it might be convenient to rebuild indexes, slowing down deploy time
##php bin/console ezplatform:reindex --processes=auto
# When using Varnish/Fastly, HttpCache is not cleared when clearing Symfony Cache folder above, you need to
# clear cache yourself when for instance templates or config affecting responses change, for instance with:
##bin/console fos:httpcache:invalidate:tag ez-all
# Depending on your VCL, buy default this would trigger a soft purge (expiry) and allow grace period, however
# even so if your change only affected certain subset of content, ideally you should only clear specific tags:
##bin/console fos:httpcache:invalidate:tag l44 c33 ct2 ...
' in container/service 'web': exit status 11
A task failure does not mean that ddev failed, but your hook configuration has a command that failed.
Restarted vntp
Your project can be reached at https://vntp.ddev.site https://127.0.0.1:61937
The biggest problems we have are in both sets of hooks:
nvm is not compatible with the npm config "prefix" option: currently set to "/mnt/ddev-global-cache/nvm_dir/vntp-web/versions/node/v10.15.3" Run
npm config delete prefix
ornvm use --delete-prefix v10.15.3 --silent
to unset it.
This seems to be caused by https://stackoverflow.com/a/58559982/215713 - and it may only be with super-old node versions, because it hasn't been reported elsewhere.
DDEV's in-container .nvm is in fact a link, as in the Stack Overflow link.
ls -l ~/.nvm
lrwxrwxrwx 1 rfay dialout 39 Aug 20 21:24 /home/rfay/.nvm -> /mnt/ddev-global-cache/nvm_dir/vntp-web
Since node 10.15.3 is several years out of support, it's not clear to me whether this has to do with node 10 or what.
Your build hooks are:
# Install newer version of Node.js, as current default (v6LTS) does not play well with WebpackEncore
unset NPM_CONFIG_PREFIX
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.8/install.sh | dash
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
nvm current
nvm install 10.15.3
if [ -z "$COMPOSER_AUTH" ]; then
echo "TIP: If you need to authenticate against Github/Gitlab/updates.ez.no, use COMPOSER_AUTH env variable"
echo "See: https://docs.platform.sh/guides/general/composer-auth.html#set-the-envcomposer_auth-project-variable"
fi
composer install --no-dev --prefer-dist --no-progress --no-interaction --optimize-autoloader
So ... DDEV already has nvm installed, and this tries to install over the top of it. We can obviously improve it to check for nvm's existence before overwriting everything, but we'd hope that we wouldn't have to do that.
And of course, then it uses node 10 for a "newer" nodejs, but node 10 has been end-of-life since 2021-04-30.
And I think if we ever get to if [ -z "$COMPOSER_AUTH" ];
I imagine it will fail as needing to be if [ -z "${COMPOSER_AUTH:-}" ];
, not sure.
So this is a great, great example of how hard it is to take something complex written for Platform and just place it into another context.
It's easy enough to solve for this exact case with modest changes, checking for nvm existing for example. But IMO it's not going to be possible to solve every problem that we will encounter.
If you want to spend a few minutes with this I'd love to do it together.
Also, ddev pull platform
with environment main gets
[ClientException] Client error response [url] https://api.platform.sh/api/projects/lg4gpxi5g22he/environments/mai n/deployments/current [status code] 400 [reason phrase] Bad Request [message] The server could not comply with the request since it is either malformed or otherwise incorrect. [detail] {"app lications":"No
.platform.app.yaml
file found anywhere in the repository.","routes":"Error loa ding file: No.platform\/routes.yaml
file found in the repository."} [title] Bad Request
I spent some more time with this, and unfortunately it can't be made to work because it depends on python 2, which we won't be supporting.
While chasing it, I added to config.yaml
fail_on_hook_fail: true
hooks:
post-start:
- exec: npm config delete prefix && npm config set prefix ~/.nvm/versions/node/v12.22.12
to sort out nvm problems.
And in your build hooks in .platform.app.yaml I changed to:
hooks:
# Build hook, done before connected to services, disk is still writable here
build: |
set -e +u
because DDEV by default executes with set -e -u
, not allowing unset variables, and COMPOSER_AUTH is unset in there.
Closing, this one won't go with python 2, but thanks so much for trying it out, and it helped me learn a lot about Platform.sh
Hi,
I've got an issue while pulling my Ibexa project (Symfony) from Platform.sh.
This is steps to reproduce (maybe i did something wrong)....
platform get lg4gpxi5g22he
cd vntp
brew unlink ddev && brew install --HEAD --fetch-head drud/ddev/ddev
(because i had already ddev installed and discover that i needed to install the new version)ddev config
ddev get platformsh/ddev-platformsh
And this is the error i get
So i executed
ddev delete && ddev get platformsh/ddev-platformsh
but same error occurs.Am i doing something wrong ?
@rfay I give you access to my PSH project (VNTP, dev branch), let me know if the invite expired, i'm in vacation but i'm able to connect and invite you again.