search

hubmapconsortium / sprm

GNU General Public License v3.0
4 stars 0 forks source link

SPRM excessive memory allocation on large stitched images #9

Closed jswelling closed 3 years ago

jswelling commented 3 years ago

Reference "/hive/hubmap-test/data/consortium/University of Florida TMC/a21f0f5baa02d38f6e63e92946366629" and /hive/hubmap-test/scratch/trig__2021-06-11T04:17:50.304061+00:00 . Executed on l002 June 11 2021; error log follows.

^[[1;30mINFO^[[0m [job sprm] /hive/hubmap-test/scratch/trig__2021-06-11T04:17:50.304061+00:00/cwl-out-tmp/scma78f0$ docker \
    run \
    -i \
    --mount=type=bind,source=/hive/hubmap-test/scratch/trig__2021-06-11T04:17:50.304061+00:00/cwl-out-tmp/scma78f0,target=/noaaQJ \
    --mount=type=bind,source=/hive/hubmap-test/scratch/trig__2021-06-11T04:17:50.304061+00:00/cwl-tmp/5bt5117_,target=/tmp \
    --mount=type=bind,source=/hive/hubmap-test/scratch/trig__2021-06-11T04:17:50.304061+00:00/cwl_out/stitched/expressions,target=/var/lib/cwl/stg84be57e7-81d3-46d7-af05-8b22ae9b3a68/expressions,reado\
nly \
    --mount=type=bind,source=/hive/hubmap-test/scratch/trig__2021-06-11T04:17:50.304061+00:00/cwl_out/stitched/mask,target=/var/lib/cwl/stg02308545-b005-4ca8-9415-61df59d9cc13/mask,readonly \
    --workdir=/noaaQJ \
    --read-only=true \
    --user=68728:23629 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/noaaQJ \
    --cidfile=/hive/hubmap-test/scratch/trig__2021-06-11T04:17:50.304061+00:00/cwl-tmp/y7up35a7/20210611013843-402242.cid \
    hubmap/sprm:1.0.3.2 \
    sprm \
    /var/lib/cwl/stg84be57e7-81d3-46d7-af05-8b22ae9b3a68/expressions \
    /var/lib/cwl/stg02308545-b005-4ca8-9415-61df59d9cc13/mask \
    --enable-manhole
Manhole[1:1623389928.4650]: Not patching os.fork and os.forkpty. Activation is done by signal Signals.SIGUSR1
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
BLAS : Program i^[[1;30mINFO^[[0m [job sprm] Max memory used: 89165MiB
^[[1;30mWARNING^[[0m ^[[33m[job sprm] completed permanentFail^[[0m
^[[1;30mWARNING^[[0m ^[[33m[step sprm] completed permanentFail^[[0m
^[[1;30mINFO^[[0m [workflow ] completed permanentFail
jswelling commented 3 years ago

Redundant with #10