search

FriendsOfShopware / shopware-cli

CLI for Shopware Account and Shopware 6
MIT License
81 stars 31 forks source link

0.3.5-php-7.4 - Exit-Code 137 (out of memory?) in Bitbucket Pipeline #238

Closed roman-1983 closed 11 months ago

roman-1983 commented 11 months ago

PHP Version

7.4

Shopware Version

6.4.20.2

Actual behaviour

I am running a bitbucket pipeline which runs the validation with tag shopware-cli:latest-php-7.4 in a separate step. The step crashes with Exit-Code 137. When i am running 0.3.4-php-7.4 it does not crash.

    - step: &validate-plugin
        name: Validate Plugin
        services:
          - docker
        size: 2x
        script:
          - for f in artifacts/*.source; do source $f; done
          - docker run --rm -v $(shell pwd):$(shell pwd) -w$(shell pwd) -eNODE_OPTIONS=--openssl-legacy-provider ghcr.io/friendsofshopware/shopware-cli:latest-php-7.4 extension validate $(shell pwd) --verbose

Log:

Unable to find image '[ghcr.io/friendsofshopware/shopware-cli:latest-php-7.4](http://ghcr.io/friendsofshopware/shopware-cli:latest-php-7.4)' locally
latest-php-7.4: Pulling from friendsofshopware/shopware-cli
...
59d4e2bca3da: Pull complete
Digest: sha256:9c2dbcabac4a0f6a4afb2c82426084c2503a593d78f2a794fb85568656115b14
Status: Downloaded newer image for ghcr.io/friendsofshopware/shopware-cli:latest-php-7.4
make: *** [validate] Error 137

Docker log:

time="2023-10-10T21:07:24.295631738Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
time="2023-10-10T21:07:24.295948199Z" level=warning msg="Binding to IP address without --tlsverify is insecure and gives root access on this machine to everyone who has access to your network." host="tcp://0.0.0.0:2375"
time="2023-10-10T21:07:24.295967590Z" level=warning msg="Binding to an IP address, even on localhost, can also give access to scripts run in a browser. Be safe out there!" host="tcp://0.0.0.0:2375"
time="2023-10-10T21:07:25.296056300Z" level=warning msg="Binding to an IP address without --tlsverify is deprecated. Startup is intentionally being slowed down to show this message" host="tcp://0.0.0.0:2375"
time="2023-10-10T21:07:25.296099608Z" level=warning msg="Please consider generating tls certificates with client validation to prevent exposing unauthenticated root access to your network" host="tcp://0.0.0.0:2375"
time="2023-10-10T21:07:25.296115462Z" level=warning msg="You can override this by explicitly specifying '--tls=false' or '--tlsverify=false'" host="tcp://0.0.0.0:2375"
time="2023-10-10T21:07:25.296126360Z" level=warning msg="Support for listening on TCP without authentication or explicit intent to run without authentication will be removed in the next release" host="tcp://0.0.0.0:2375"
time="2023-10-10T21:07:40Z" level=warning msg="containerd config version `1` has been deprecated and will be removed in containerd v2.0, please switch to version `2`, see https://github.com/containerd/containerd/blob/main/docs/PLUGINS.md#version-header"
time="2023-10-10T21:07:40.376979617Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
time="2023-10-10T21:07:40.377555611Z" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
time="2023-10-10T21:07:40.380222366Z" level=warning msg="failed to load plugin io.containerd.internal.v1.opt" error="mkdir /opt/containerd: read-only file system"
time="2023-10-10T21:07:40.380362972Z" level=error msg="failed to initialize a tracing processor \"otlp\"" error="no OpenTelemetry endpoint: skip plugin"
time="2023-10-10T21:07:40.482933626Z" level=warning msg="Your kernel does not support CPU realtime scheduler"
time="2023-10-10T21:07:40.483023224Z" level=warning msg="Your kernel does not support cgroup blkio weight"
time="2023-10-10T21:07:40.483065812Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
time="2023-10-10T21:07:40Z" level=info msg="Pipelines plugin request authorization." allowed=false method=HEAD plugin=pipelines uri=/_ping
time="2023-10-10T21:07:40.810845967Z" level=error msg="AuthZRequest for HEAD /_ping returned error: authorization denied by plugin pipelines: "
time="2023-10-10T21:07:40Z" level=info msg="Pipelines plugin request authorization." allowed=true method=GET plugin=pipelines uri=/_ping
time="2023-10-10T21:07:40Z" level=info msg="Container create request." ArgsEscaped=false AttachStderr=true AttachStdin=false AttachStdout=true ExposedPorts="map[]" Healthcheck="<nil>" Labels="map[]" MacAddress= NetworkDisabled=false OnBuild="[]" OpenStdin=false StdinOnce=false StopSignal= StopTimeout="<nil>" Tty=false plugin=pipelines
time="2023-10-10T21:07:40Z" level=info msg="Container create request." AutoRemove=true BlkioDeviceReadBps="[]" BlkioDeviceReadIOps="[]" BlkioDeviceWriteBps="[]" BlkioDeviceWriteIOps="[]" BlkioWeight=0 BlkioWeightDevice="[]" CPUCount=0 CPUPercent=0 CPUPeriod=0 CPUQuota=0 CPURealtimePeriod=0 CPURealtimeRuntime=0 CPUShares=0 CapAdd="[]" CapDrop="[]" Cgroup= CgroupParent= ConsoleSize="[0 0]" ContainerIDFile= CpusetCpus= CpusetMems= DNS="[]" DNSOptions="[]" DNSSearch="[]" DeviceCgroupRules="[]" Devices="[]" ExtraHosts="[]" GroupAdd="[]" IOMaximumBandwidth=0 IOMaximumIOps=0 Init="<nil>" IpcMode= Isolations= KernelMemory=0 Links="[]" LogConfig="{ map[]}" MaskedPaths="[]" Memory=0 MemoryReservation=0 MemorySwap=0 MemorySwappiness=0xc00021a608 Mounts="[]" NanoCPUs=0 NetworkMode=default OomKillDisable=0xc00021a612 OomScoreAdj=0 PidMode= PidsLimit=0xc00021a618 PortBindings="map[]" Privileged=false PublishAllPorts=false ReadOnlyPaths="[]" RestartPolicy="{no 0}" Runtime= SecurityOpt="[]" ShmSize=0 StorageOpt="map[]" Sysctls="map[]" Ulimits="[]" UsernsMode= VolumeDriver= VolumesFrom="[]" plugin=pipelines
time="2023-10-10T21:07:40Z" level=info msg="Container create request." EndpointsConfig="map[]" plugin=pipelines
time="2023-10-10T21:07:40Z" level=info msg="Pipelines plugin request authorization." allowed=true method=POST plugin=pipelines uri=/v1.41/containers/create
time="2023-10-10T21:07:40.817176429Z" level=error msg="Handler for POST /v1.41/containers/create returned error: No such image: ghcr.io/friendsofshopware/shopware-cli:latest-php-7.4"
time="2023-10-10T21:07:40Z" level=info msg="Pipelines plugin request authorization." allowed=true method=POST plugin=pipelines uri="/v1.41/images/create?fromImage=ghcr.io%2Ffriendsofshopware%2Fshopware-cli&tag=latest-php-7.4"
time="2023-10-10T21:07:51Z" level=info msg="Container create request." ArgsEscaped=false AttachStderr=true AttachStdin=false AttachStdout=true ExposedPorts="map[]" Healthcheck="<nil>" Labels="map[]" MacAddress= NetworkDisabled=false OnBuild="[]" OpenStdin=false StdinOnce=false StopSignal= StopTimeout="<nil>" Tty=false plugin=pipelines
time="2023-10-10T21:07:51Z" level=info msg="Container create request." AutoRemove=true BlkioDeviceReadBps="[]" BlkioDeviceReadIOps="[]" BlkioDeviceWriteBps="[]" BlkioDeviceWriteIOps="[]" BlkioWeight=0 BlkioWeightDevice="[]" CPUCount=0 CPUPercent=0 CPUPeriod=0 CPUQuota=0 CPURealtimePeriod=0 CPURealtimeRuntime=0 CPUShares=0 CapAdd="[]" CapDrop="[]" Cgroup= CgroupParent= ConsoleSize="[0 0]" ContainerIDFile= CpusetCpus= CpusetMems= DNS="[]" DNSOptions="[]" DNSSearch="[]" DeviceCgroupRules="[]" Devices="[]" ExtraHosts="[]" GroupAdd="[]" IOMaximumBandwidth=0 IOMaximumIOps=0 Init="<nil>" IpcMode= Isolations= KernelMemory=0 Links="[]" LogConfig="{ map[]}" MaskedPaths="[]" Memory=0 MemoryReservation=0 MemorySwap=0 MemorySwappiness=0xc00021a9e8 Mounts="[]" NanoCPUs=0 NetworkMode=default OomKillDisable=0xc00021a9f2 OomScoreAdj=0 PidMode= PidsLimit=0xc00021a9f8 PortBindings="map[]" Privileged=false PublishAllPorts=false ReadOnlyPaths="[]" RestartPolicy="{no 0}" Runtime= SecurityOpt="[]" ShmSize=0 StorageOpt="map[]" Sysctls="map[]" Ulimits="[]" UsernsMode= VolumeDriver= VolumesFrom="[]" plugin=pipelines
time="2023-10-10T21:07:51Z" level=info msg="Container create request." EndpointsConfig="map[]" plugin=pipelines
time="2023-10-10T21:07:51Z" level=info msg="Pipelines plugin request authorization." allowed=true method=POST plugin=pipelines uri=/v1.41/containers/create
time="2023-10-10T21:07:51Z" level=info msg="Pipelines plugin request authorization." allowed=true method=POST plugin=pipelines uri="/v1.41/containers/d0e480d8dd14c67603550a1fe2a55868d431fdc23f8806537b6ea87dc010c37f/attach?stderr=1&stdout=1&stream=1"
time="2023-10-10T21:07:51Z" level=info msg="Pipelines plugin request authorization." allowed=true method=POST plugin=pipelines uri="/v1.41/containers/d0e480d8dd14c67603550a1fe2a55868d431fdc23f8806537b6ea87dc010c37f/wait?condition=removed"
time="2023-10-10T21:07:51Z" level=info msg="Pipelines plugin request authorization." allowed=true method=POST plugin=pipelines uri=/v1.41/containers/d0e480d8dd14c67603550a1fe2a55868d431fdc23f8806537b6ea87dc010c37f/start
time="2023-10-10T21:07:51.682713632Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
time="2023-10-10T21:07:51.682769812Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
time="2023-10-10T21:07:51.682784725Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
time="2023-10-10T21:07:51.683140302Z" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/d0e480d8dd14c67603550a1fe2a55868d431fdc23f8806537b6ea87dc010c37f pid=395 runtime=io.containerd.runc.v2
time="2023-10-10T21:07:56.137593026Z" level=warning msg="cleaning up after shim disconnected" id=d0e480d8dd14c67603550a1fe2a55868d431fdc23f8806537b6ea87dc010c37f namespace=moby
time="2023-10-10T21:07:56.146643116Z" level=warning msg="cleanup warnings time=\"2023-10-10T21:07:56Z\" level=info msg=\"starting signal loop\" namespace=moby pid=790 runtime=io.containerd.runc.v2\n"

Expected behaviour

It should not crash.

Steps to Reproduce?

It's not that easy as i cant reproduce it locally. It just crashes on Bitbucket Pipeline. Also i cant log the memory usage on my local machine :/

shyim commented 11 months ago

You can set as env GOMAXPROCS=2 and try out it works? 🤔

shyim commented 11 months ago

How many PHP files does that plugin have?

roman-1983 commented 11 months ago

I was just trying around and by raising the memory i got it working with this configuration:

definitions:
  services:
    docker:
      memory: 3048

The plugin has 120 files.

In version 0.3.4 it worked with the default memory-settings in Bitbucket.

roman-1983 commented 11 months ago

docker run --rm -v $(shell pwd):$(shell pwd) -w$(shell pwd) -eNODE_OPTIONS=--openssl-legacy-provider -e GOMAXPROCS=2 ghcr.io/friendsofshopware/shopware-cli:${SHOPWARE_CLI_TAG} extension validate $(shell pwd) --verbose

I have set the addition env-variable but it didn't help.

roman-1983 commented 11 months ago

@shyim I could solve it by raising the memory for my pipeline. I don't know what exactly changed between 0.3.4 and 0.3.5 but that change is the reason that the memory-limit of bitbucket isn't enough anymore. 😅 Maybe there is any chance to limit the memory-consumption of shopware-cli?

shyim commented 11 months ago

Should be fixed with https://github.com/FriendsOfShopware/shopware-cli/actions/runs/6479259781