lorisleiva / laravel-docker

🐳 Generic docker image for Laravel Applications
MIT License
927 stars 313 forks source link

Composer Install Laravel 7.2 on Gitlab Pipeline Failed #67

Closed yoelpc4 closed 3 years ago

yoelpc4 commented 3 years ago

Hi @lorisleiva, thanks for writing this helpful article https://lorisleiva.com/using-gitlabs-pipeline-with-laravel/. Here is my following setup:

{
    "name": "wika-overseas-marketing/api-crm-wika-overseas",
    "type": "project",
    "description": "API CRM PT. Wijaya Karya (Persero) Tbk. Departemen Pemasaran Bangunan Gedung & Luar Negeri.",
    "keywords": [
        "laravel",
        "api",
        "crm"
    ],
    "license": "MIT",
    "require": {
        "php": ">=7.2",
        "ext-imagick": "*",
        "ext-json": "*",
        "barryvdh/laravel-snappy": "^0.4.8",
        "cloudcreativity/laravel-json-api": "^1.7.0",
        "fideloper/proxy": "^4.4.1",
        "intervention/image": "^2.5.1",
        "laravel/framework": "^6.20.7",
        "laravel/horizon": "^3.7.2",
        "laravel/slack-notification-channel": "^2.2.0",
        "laravel/tinker": "^1.0.10",
        "laravelcollective/errors": "^1.1.0",
        "lorisleiva/laravel-deployer": "^0.3.5",
        "maatwebsite/excel": "^3.1.26",
        "pusher/pusher-php-server": "^4.1.5",
        "sentry/sentry-laravel": "^1.9.0",
        "spatie/laravel-activitylog": "^3.16.1",
        "spatie/laravel-model-states": "^1.9.1",
        "spatie/laravel-permission": "^3.17.0",
        "tymon/jwt-auth": "^1.0.2",
        "yoelpc4/laravel-cloudinary": "^2.0.0",
        "yoelpc4/laravel-exchange-rate": "^3.0.0"
    },
    "require-dev": {
        "barryvdh/laravel-ide-helper": "^2.8.2",
        "beyondcode/laravel-dump-server": "^1.3.0",
        "cloudcreativity/json-api-testing": "^2.1.0",
        "facade/ignition": "^1.16.4",
        "fakerphp/faker": "^1.9.1",
        "laravel/ui": "^1.2.0",
        "mockery/mockery": "^1.3.3",
        "nunomaduro/collision": "^v3.0.1",
        "phpunit/phpunit": "^8.5.13|^9.3.11"
    },
    "config": {
        "optimize-autoloader": true,
        "preferred-install": "dist",
        "sort-packages": true
    },
    "extra": {
        "laravel": {
            "dont-discover": []
        }
    },
    "autoload": {
        "psr-4": {
            "App\\": "app/"
        },
        "classmap": [
            "database/seeds",
            "database/factories"
        ],
        "files": [
            "helpers.php"
        ]
    },
    "autoload-dev": {
        "psr-4": {
            "Tests\\": "tests/"
        }
    },
    "minimum-stability": "dev",
    "prefer-stable": true,
    "scripts": {
        "post-autoload-dump": [
            "Illuminate\\Foundation\\ComposerScripts::postAutoloadDump",
            "@php artisan package:discover --ansi"
        ],
        "post-root-package-install": [
            "@php -r \"file_exists('.env') || copy('.env.example', '.env');\""
        ],
        "post-create-project-cmd": [
            "@php artisan key:generate --ansi"
        ],
        "post-install-cmd": [
            "php artisan clear-compiled",
            "chmod -R 775 public/"
        ],
        "post-update-cmd": [
            "Illuminate\\Foundation\\ComposerScripts::postUpdate",
            "@php artisan clear-compiled",
            "@php artisan ide-helper:generate",
            "@php artisan ide-helper:meta",
            "@php artisan horizon:install"
        ]
    }
}
.set_git_credentials: &set_git_credentials |
  git config --global user.email "$GITLAB_USER_EMAIL"
  git config --global user.name "$GITLAB_USER_NAME"

.init_ssh: &init_ssh |
  eval $(ssh-agent -s)
  echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
  mkdir -p ~/.ssh
  chmod 700 ~/.ssh
  touch ~/.ssh/known_hosts
  echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
  chmod 644 ~/.ssh/known_hosts

.change_file_permissions: &change_file_permissions |
  find . -type f ! -path "./vendor/*" ! -path "./node_modules/*" -exec chmod 664 {} \;
  find . -type d ! -path "./vendor/*" ! -path "./node_modules/*" -exec chmod 775 {} \;

image: lorisleiva/laravel-docker:7.2

stages:
  - build
  - deploy

composer:
  stage: build

  only:
    - master
    - tags

  except:
    - develop

  cache:
    key: ${CI_COMMIT_REF_SLUG}-composer
    paths:
      - vendor/

  script:
    - composer install -o --prefer-dist --no-ansi --no-interaction --no-progress --no-scripts
    - cp .env.example .env
    - php artisan key:generate

  artifacts:
    expire_in: 1 month
    paths:
      - vendor/
      - .env

yarn:
  stage: build

  only:
    - master
    - tags

  except:
    - develop

  cache:
    key: ${CI_COMMIT_REF_SLUG}-yarn
    paths:
      - node_modules/

  script:
    - yarn install
    - yarn run prod

  artifacts:
    expire_in: 1 month
    paths:
      - node_modules/
      - public/css/
      - public/fonts/
      - public/js/

staging:
  stage: deploy

  environment:
    name: staging
    url: https://HEROKU_APP_NAME.herokuapp.com

  only:
    - master

  except:
    - tags

  script:
    - apk --update --no-cache add ruby ruby-dev ruby-irb ruby-rake ruby-io-console ruby-bigdecimal ruby-json ruby-bundler ruby-rdoc
    - gem install dpl
    - dpl --provider=heroku --app=$HEROKU_APP_NAME --api-key=$HEROKU_API_KEY
    - yarn global add heroku
    - heroku run php artisan migrate:fresh --drop-views --force --app $HEROKU_APP_NAME
    - heroku run php artisan db:seed --class=MasterDataSeeder --force --app $HEROKU_APP_NAME

production:
  stage: deploy

  environment:
    name: production
    url: https://$DIGITAL_OCEAN_APP_NAME

  only:
    - tags

  except:
    - branches

  script:
    - *set_git_credentials
    - *init_ssh
    - *change_file_permissions
    - php artisan deploy $DIGITAL_OCEAN_APP_NAME -v

Between January-November 2020, I faced no critical issue while implementing this package to execute CI/CD along with my laravel 7.2 project on the gitlab shared runner.

Currently I'm encountering a critical issue on my pipeline while the runner executing command composer install -o --prefer-dist --no-ansi --no-interaction --no-progress --no-scripts.

This is the build log from gitlab shared runner

Running with gitlab-runner 13.7.0-rc1 (98e2e32d)
  on docker-auto-scale 0277ea0f
Preparing the "docker+machine" executor
00:35
Using Docker executor with image lorisleiva/laravel-docker:7.2 ...
Pulling docker image lorisleiva/laravel-docker:7.2 ...
Using docker image sha256:e289e99c3a485a8df8370088d3f5657ae88a92d45e431e920018b547a4c1f5c4 for lorisleiva/laravel-docker:7.2 with digest lorisleiva/laravel-docker@sha256:589a65741f745ea844479c12ad4232dd1dae10748a9a366d1b278aaa9d1caba5 ...
Preparing environment
00:01
Running on runner-0277ea0f-project-15543402-concurrent-0 via runner-0277ea0f-srm-1608416499-5ddcf709...
Getting source from Git repository
00:04
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/api-crm-wika-overseas/api-crm-wika-overseas/.git/
Created fresh repository.
Checking out 36233de2 as master...
Skipping Git submodules setup
Restoring cache
00:04
Checking cache for master-composer-6...
Downloading cache.zip from https://storage.googleapis.com/gitlab-com-runners-cache/project/15543402/master-composer-6 
Successfully extracted cache
Executing "step_script" stage of the job script
00:07
$ composer install -o --prefer-dist --no-ansi --no-interaction --no-progress --no-scripts
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Package operations: 0 installs, 49 updates, 0 removals
  - Downloading symfony/css-selector (v5.2.1)
  - Downloading symfony/var-dumper (v4.4.18)
  - Downloading symfony/routing (v4.4.18)
  - Downloading symfony/process (v4.4.18)
  - Downloading symfony/mime (v5.2.1)
  - Downloading symfony/http-foundation (v4.4.18)
  - Downloading symfony/event-dispatcher (v4.4.18)
  - Downloading symfony/debug (v4.4.18)
  - Downloading symfony/error-handler (v4.4.18)
  - Downloading symfony/http-kernel (v4.4.18)
  - Downloading symfony/finder (v4.4.18)
  - Downloading symfony/console (v4.4.18)
  - Downloading swiftmailer/swiftmailer (v6.2.4)
  - Downloading symfony/translation (v4.4.18)
  - Downloading nesbot/carbon (2.43.0)
  - Downloading monolog/monolog (2.2.0)
  - Downloading laravel/framework (v6.20.7)
  - Downloading symfony/filesystem (v5.2.1)
  - Downloading composer/spdx-licenses (1.5.5)
  - Downloading composer/composer (2.0.8)
  - Downloading barryvdh/laravel-ide-helper (v2.8.2)
  - Downloading sebastian/type (1.1.4)
  - Downloading sebastian/resource-operations (2.0.2)
  - Downloading sebastian/recursion-context (3.0.1)
  - Downloading sebastian/object-reflector (1.1.2)
  - Downloading sebastian/object-enumerator (3.0.4)
  - Downloading sebastian/global-state (3.0.1)
  - Downloading sebastian/exporter (3.1.3)
  - Downloading sebastian/environment (4.2.4)
  - Downloading sebastian/diff (3.0.3)
  - Downloading sebastian/comparator (3.0.3)
  - Downloading phpunit/php-timer (2.1.3)
  - Downloading phpunit/php-file-iterator (2.0.3)
  - Downloading sebastian/code-unit-reverse-lookup (1.0.2)
  - Downloading phpunit/php-token-stream (3.1.2)
  - Downloading phpunit/php-code-coverage (7.0.14)
  - Downloading phpspec/prophecy (1.12.2)
  - Downloading phar-io/version (3.0.4)
  - Downloading phar-io/manifest (2.0.1)
  - Downloading phpunit/phpunit (8.5.13)
  - Downloading fakerphp/faker (v1.13.0)
  - Downloading nikic/php-parser (v4.10.3)
  - Downloading symfony/yaml (v5.2.1)
  - Downloading paragonie/sodium_compat (v1.14.0)
  - Downloading pusher/pusher-php-server (v4.1.5)
  - Downloading symfony/options-resolver (v5.2.1)
  - Downloading symfony/http-client (v5.2.1)
  - Downloading sentry/sentry (2.5.1)
  - Downloading spatie/laravel-model-states (1.9.1)
  - Upgrading symfony/css-selector (v5.1.8 => v5.2.1): Extracting archive
  - Upgrading symfony/var-dumper (v4.4.16 => v4.4.18): Extracting archive
  - Upgrading symfony/routing (v4.4.16 => v4.4.18): Extracting archive
  - Upgrading symfony/process (v4.4.16 => v4.4.18): Extracting archive
  - Upgrading symfony/mime (v5.1.8 => v5.2.1): Extracting archive
  - Upgrading symfony/http-foundation (v4.4.16 => v4.4.18): Extracting archive
  - Upgrading symfony/event-dispatcher (v4.4.16 => v4.4.18): Extracting archive
  - Upgrading symfony/debug (v4.4.16 => v4.4.18): Extracting archive
  - Upgrading symfony/error-handler (v4.4.16 => v4.4.18): Extracting archive
  - Upgrading symfony/http-kernel (v4.4.16 => v4.4.18): Extracting archive
  - Upgrading symfony/finder (v4.4.16 => v4.4.18): Extracting archive
  - Upgrading symfony/console (v4.4.16 => v4.4.18): Extracting archive
  - Upgrading swiftmailer/swiftmailer (v6.2.3 => v6.2.4): Extracting archive
  - Upgrading symfony/translation (v4.4.16 => v4.4.18): Extracting archive
  - Upgrading nesbot/carbon (2.42.0 => 2.43.0): Extracting archive
  - Upgrading monolog/monolog (2.1.1 => 2.2.0): Extracting archive
  - Upgrading laravel/framework (v6.20.5 => v6.20.7): Extracting archive
  - Upgrading symfony/filesystem (v5.1.8 => v5.2.1): Extracting archive
  - Upgrading composer/spdx-licenses (1.5.4 => 1.5.5): Extracting archive
  - Upgrading composer/composer (2.0.7 => 2.0.8): Extracting archive
  - Upgrading barryvdh/laravel-ide-helper (v2.8.1 => v2.8.2): Extracting archive
  - Upgrading sebastian/type (1.1.3 => 1.1.4): Extracting archive
  - Upgrading sebastian/resource-operations (2.0.1 => 2.0.2): Extracting archive
  - Upgrading sebastian/recursion-context (3.0.0 => 3.0.1): Extracting archive
  - Upgrading sebastian/object-reflector (1.1.1 => 1.1.2): Extracting archive
  - Upgrading sebastian/object-enumerator (3.0.3 => 3.0.4): Extracting archive
  - Upgrading sebastian/global-state (3.0.0 => 3.0.1): Extracting archive
  - Upgrading sebastian/exporter (3.1.2 => 3.1.3): Extracting archive
  - Upgrading sebastian/environment (4.2.3 => 4.2.4): Extracting archive
  - Upgrading sebastian/diff (3.0.2 => 3.0.3): Extracting archive
  - Upgrading sebastian/comparator (3.0.2 => 3.0.3): Extracting archive
  - Upgrading phpunit/php-timer (2.1.2 => 2.1.3): Extracting archive
  - Upgrading phpunit/php-file-iterator (2.0.2 => 2.0.3): Extracting archive
  - Upgrading sebastian/code-unit-reverse-lookup (1.0.1 => 1.0.2): Extracting archive
  - Upgrading phpunit/php-token-stream (3.1.1 => 3.1.2): Extracting archive
  - Upgrading phpunit/php-code-coverage (7.0.12 => 7.0.14): Extracting archive
  - Upgrading phpspec/prophecy (1.12.1 => 1.12.2): Extracting archive
  - Upgrading phar-io/version (2.0.1 => 3.0.4): Extracting archive
  - Upgrading phar-io/manifest (1.0.3 => 2.0.1): Extracting archive
  - Upgrading phpunit/phpunit (8.5.11 => 8.5.13): Extracting archive
  - Upgrading fakerphp/faker (v1.12.0 => v1.13.0): Extracting archive
  - Upgrading nikic/php-parser (v4.10.2 => v4.10.3): Extracting archive
  - Upgrading symfony/yaml (v5.1.8 => v5.2.1): Extracting archive
  - Upgrading paragonie/sodium_compat (v1.13.0 => v1.14.0): Extracting archive
  - Upgrading pusher/pusher-php-server (v4.1.4 => v4.1.5): Extracting archive
  - Upgrading symfony/options-resolver (v5.1.8 => v5.2.1): Extracting archive
  - Upgrading symfony/http-client (v5.1.8 => v5.2.1): Extracting archive
  - Upgrading sentry/sentry (2.5.0 => 2.5.1): Extracting archive
  - Upgrading spatie/laravel-model-states (1.9.0 => 1.9.1): Extracting archive
    Update of symfony/css-selector failed
    Update of symfony/var-dumper failed
    Update of symfony/routing failed
    Update of symfony/process failed
    Update of symfony/mime failed
    Update of symfony/http-foundation failed
    Update of symfony/event-dispatcher failed
    Update of symfony/debug failed
    Update of symfony/error-handler failed
    Update of symfony/http-kernel failed
    Update of symfony/finder failed
    Update of symfony/console failed
    Update of symfony/translation failed
    Update of symfony/filesystem failed
    Update of composer/spdx-licenses failed
    Update of swiftmailer/swiftmailer failed
    Update of monolog/monolog failed
    Update of barryvdh/laravel-ide-helper failed
    Update of sebastian/type failed
    Update of sebastian/resource-operations failed
    Update of sebastian/recursion-context failed
    Update of sebastian/object-reflector failed
    Update of composer/composer failed
    Update of sebastian/object-enumerator failed
    Update of sebastian/global-state failed
    Update of sebastian/exporter failed
    Update of sebastian/environment failed
    Update of sebastian/diff failed
    Update of sebastian/comparator failed
    Update of phpunit/php-timer failed
    Update of nesbot/carbon failed
    Update of laravel/framework failed
    Update of phpunit/php-file-iterator failed
    Update of sebastian/code-unit-reverse-lookup failed
    Update of phpunit/php-token-stream failed
    Update of phpunit/php-code-coverage failed
    Update of phpspec/prophecy failed
    Update of phar-io/version failed
    Update of phar-io/manifest failed
    Update of phpunit/phpunit failed
    Update of symfony/yaml failed
    Update of pusher/pusher-php-server failed
    Update of symfony/options-resolver failed
    Update of symfony/http-client failed
    Update of sentry/sentry failed
    Update of spatie/laravel-model-states failed
    Update of nikic/php-parser failed
    Update of paragonie/sodium_compat failed
    Update of fakerphp/faker failed
In ClassLoader.php line 444:

  include(/builds/api-crm-wika-overseas/api-crm-wika-overseas/vendor/composer  
  /../composer/composer/src/Composer/Console/GithubActionError.php): failed t  
  o open stream: No such file or directory                                     

install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-suggest] [--no-dev] [--no-autoloader] [--no-scripts] [--no-progress] [--no-install] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--apcu-autoloader-prefix APCU-AUTOLOADER-PREFIX] [--ignore-platform-req IGNORE-PLATFORM-REQ] [--ignore-platform-reqs] [--] [<packages>...]
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 1

As you can see this is the error thrown from composer

In ClassLoader.php line 444:

  include(/builds/api-crm-wika-overseas/api-crm-wika-overseas/vendor/composer  
  /../composer/composer/src/Composer/Console/GithubActionError.php): failed t  
  o open stream: No such file or directory

Do you have any idea how to solves this issue?

admin1for2 commented 3 years ago

I have the exact same problem as you : composer updates failing with error finding the file GithubActionError.php. Did you manage to find a solution ?

Thanks

yoelpc4 commented 3 years ago

@admin1for2 Nope, current solution is upgrading to 7.3 or 7.4

lorisleiva commented 3 years ago

Sorry guys, this repo no longer maintains versions below 7.3.

Majicbob commented 3 years ago

I ran into this same issue with a forked version of 7.4 After some debugging I wasn't able to determine the root cause but it appeared to be on GitLab's end. Clearing the runner caches on GitLab fixed it for me.

jnbn commented 2 years ago

We're having the same error with 8.0 image and still couldn't find any solutions

https://stackoverflow.com/questions/70608657/error-on-composer-before-deploying-on-bitbucket-pipeline

fgilio commented 2 years ago

I'm having this exact same issue when using the PHP 8 image, will share any info I find


EDIT: Oooook. We found an acceptable workaround. TLDR: Run rm -rf ./vendor before your composer install before deploying.

Now to our analysis 👇🏼

We run all our tests and deploys in GitLab CI (thanks to @lorisleiva 🤗 ). And we have 3 jobs in 3 stages:

  1. preparation stage runs the "composer" job, which runs composer install --prefer-dist --no-ansi --no-interaction --no-progress --no-scripts
  2. testing stage runs the "phpunit" job
  3. deploy stage runs our Vapor deploy script, which runs COMPOSER_MIRROR_PATH_REPOS=1 composer install --prefer-dist --no-ansi --no-interaction --no-progress --optimize-autoloader --no-dev

So, in the composer job we install our dev dependencies because we need them to test the app. The resulting ./vendor directory gets cached by GitLab's CI system and it's automatically made available for the subsequent stages. That's great, cause it means we install dependencies once and then we can re use that in the testing and deploying stages. So we "deploy the same thing that we tested"... But that's actually false, cause for production we don't want to install the development dependencies. That's why we use the --no-dev flag in composer in the deploy stage. Keep in mind, we do need those development dependencies to run phpunit. And when we run it we see a message like:

Installing dependencies from lock file
Verifying lock file contents can be installed on current platform.
Package operations: 0 installs, 0 updates, 73 removals

That makes sense, we already have access to the cached ./vendor directory from the composer job and now we only need to remove the development dependencies. That's when things fall apart. I've no idea if this is a bug in composer itself, in a dependency, in our codebase, etc... but composer errors out with the ...GithubActionError.php error when trying to remove the development dependencies. If we remove the --no-dev it works perfectly, but That's A NoNo.

Long story short, our solution is to embrace the fact that composer.lock exists and that this job runs in CI (where the download speed is insanely fast). So we nuke the ./vendor directory running rm -rf ./vendor right before the deployable composer install ... --no-dev.

In the end, I think this is perfectly acceptable. I'm sure there's a way to tell GitLab to avoid downloading the cached ./vendor directory, or an overall better way to do this. But we've spent way to much time today trying to understand and fix this... so, it's going to stay like this. And, no, it doesn't seen to be related to lorisleiva/laravel-docker:x.x docker images.

I hope this is helpful or at least interesting :) Please do let me know if anyone finds a better approach.

lorisleiva commented 2 years ago

Hi Franco 👋

Thanks for the detailed report and for sharing it with us!

Do let me know if there's anything this repository can do to improve the developer experience regarding this issue.

hakanersu commented 2 years ago

I had same issue and i found a different solution that i post here https://stackoverflow.com/a/71529389/7736155

include(/builds/myapp/myapp-api/vendor/composer/../composer/composer/sr  
  c/Composer/Console/GithubActionError.php): Failed to open stream: No such f  
  ile or directory 

TL;TD

composer command actually not working as expected i download new composer.phar file and everything works fine now.