manifest.manifest.layers is not iterable

openedhardware commented 3 years ago

balena version: 12.44.14 OS: Ubuntu 20.04

Building Docker preloader image. [===                     ] 12%
Step 1/7 : FROM docker:20.10.6-dind
Building Docker preloader image. [======                  ] 25%
Step 2/7 : RUN apk update && apk add --no-cache py3-pip parted btrfs-progs util-linux sfdisk file coreutils sgdisk
 ---> Running in d0cf31551d63
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/x86_64/APKINDEX.tar.gz
v3.13.5-113-gfce5d4a13b [https://dl-cdn.alpinelinux.org/alpine/v3.13/main]
v3.13.5-112-gb735a38e0a [https://dl-cdn.alpinelinux.org/alpine/v3.13/community]
OK: 13888 distinct packages available
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/x86_64/APKINDEX.tar.gz
(1/60) Installing libacl (2.2.53-r0)
(2/60) Installing libattr (2.4.48-r0)
(3/60) Installing skalibs (2.10.0.0-r0)
(4/60) Installing s6-ipcserver (2.10.0.0-r0)
(5/60) Installing utmps (0.1.0.0-r0)
Executing utmps-0.1.0.0-r0.pre-install
(6/60) Installing coreutils (8.32-r2)
(7/60) Installing libmagic (5.39-r0)
(8/60) Installing file (5.39-r0)
(9/60) Installing device-mapper-libs (2.02.187-r1)
(10/60) Installing readline (8.1.0-r0)
(11/60) Installing parted (3.3-r1)
(12/60) Installing libbz2 (1.0.8-r1)
(13/60) Installing expat (2.2.10-r1)
(14/60) Installing libffi (3.3-r2)
(15/60) Installing gdbm (1.19-r0)
(16/60) Installing sqlite-libs (3.34.1-r0)
(17/60) Installing python3 (3.8.10-r0)
(18/60) Installing py3-appdirs (1.4.4-r1)
(19/60) Installing py3-ordered-set (4.0.2-r0)
(20/60) Installing py3-parsing (2.4.7-r1)
(21/60) Installing py3-six (1.15.0-r0)
(22/60) Installing py3-packaging (20.9-r0)
(23/60) Installing py3-setuptools (51.3.3-r0)
(24/60) Installing py3-chardet (4.0.0-r0)
(25/60) Installing py3-idna (3.1-r0)
(26/60) Installing py3-urllib3 (1.26.2-r1)
(27/60) Installing py3-requests (2.25.1-r1)
(28/60) Installing py3-msgpack (1.0.2-r0)
(29/60) Installing py3-lockfile (0.12.2-r3)
(30/60) Installing py3-cachecontrol (0.12.6-r0)
(31/60) Installing py3-colorama (0.4.4-r0)
(32/60) Installing py3-contextlib2 (0.6.0-r0)
(33/60) Installing py3-distlib (0.3.1-r1)
(34/60) Installing py3-distro (1.5.0-r1)
(35/60) Installing py3-webencodings (0.5.1-r3)
(36/60) Installing py3-html5lib (1.1-r0)
(37/60) Installing py3-pytoml (0.1.21-r0)
(38/60) Installing py3-pep517 (0.9.1-r0)
(39/60) Installing py3-progress (1.5-r0)
(40/60) Installing py3-retrying (1.3.3-r0)
(41/60) Installing py3-toml (0.10.2-r0)
(42/60) Installing py3-pip (20.3.4-r0)
(43/60) Installing libfdisk (2.36.1-r1)
(44/60) Installing libsmartcols (2.36.1-r1)
(45/60) Installing sfdisk (2.36.1-r1)
(46/60) Installing libgcc (10.2.1_pre1-r3)
(47/60) Installing popt (1.18-r0)
(48/60) Installing libstdc++ (10.2.1_pre1-r3)
(49/60) Installing sgdisk (1.0.7-r0)
(50/60) Installing blkid (2.36.1-r1)
(51/60) Installing libcap-ng (0.8.2-r0)
(52/60) Installing setpriv (2.36.1-r1)
(53/60) Installing findmnt (2.36.1-r1)
(54/60) Installing mcookie (2.36.1-r1)
(55/60) Installing hexdump (2.36.1-r1)
(56/60) Installing lsblk (2.36.1-r1)
(57/60) Installing cfdisk (2.36.1-r1)
(58/60) Installing partx (2.36.1-r1)
(59/60) Installing libeconf (0.3.8-r0)
(60/60) Installing util-linux (2.36.1-r1)
Executing busybox-1.32.1-r6.trigger
OK: 112 MiB in 110 packages
Removing intermediate container d0cf31551d63
Building Docker preloader image. [=========               ] 37%
Step 3/7 : COPY ./requirements.txt /tmp/
Building Docker preloader image. [============            ] 50%
Step 4/7 : RUN pip3 install -r /tmp/requirements.txt
 ---> Running in 318e1ae19c13
Collecting sh==1.12.14
  Downloading sh-1.12.14-py2.py3-none-any.whl (38 kB)
Collecting retry==0.9.2
  Downloading retry-0.9.2-py2.py3-none-any.whl (8.0 kB)
Collecting py<2.0.0,>=1.4.26
  Downloading py-1.10.0-py2.py3-none-any.whl (97 kB)
Collecting decorator>=3.4.2
  Downloading decorator-5.0.7-py3-none-any.whl (8.8 kB)
Installing collected packages: py, decorator, sh, retry
Successfully installed decorator-5.0.7 py-1.10.0 retry-0.9.2 sh-1.12.14
Removing intermediate container 318e1ae19c13
Building Docker preloader image. [===============         ] 62%
Step 5/7 : COPY ./src /usr/src/app
Building Docker preloader image. [==================      ] 75%
Step 6/7 : WORKDIR /usr/src/app
 ---> Running in 6b381b6cf722
Removing intermediate container 6b381b6cf722
Building Docker preloader image. [=====================   ] 87%
Step 7/7 : CMD ["python3", "/usr/src/app/preload.py"]
 ---> Running in c41d0af1564f
Removing intermediate container c41d0af1564f
 ---> 094fb6c1d19b
Successfully built 094fb6c1d19b
Building Docker preloader image. [========================] 100%
| Checking that the image is a writable file
| Finding a free tcp port and getting balena settings
| Checking if the image is an edison zip archive
/ Creating preloader container
\ Starting preloader container
- Fetching application yokosoapp
- Reading image informationWaiting for Docker to start...
| Reading image informationDocker started
\ Reading image information
- Fetching application 14
- Estimating required additional space
/ Cleaning up temporary files
manifest.manifest.layers is not iterable

TypeError: manifest.manifest.layers is not iterable
    at Preloader._getLayersSizes (/snapshot/versioned-source/node_modules/balena-preload/build/preload.js:489:51)
    at Preloader._getApplicationSize (/snapshot/versioned-source/node_modules/balena-preload/build/preload.js:507:40)
    at runNextTicks (internal/process/task_queues.js:62:5)
    at processImmediate (internal/timers.js:429:9)
    at process.topLevelDomainCallback (domain.js:137:15)
    at async Preloader._getSize (/snapshot/versioned-source/node_modules/balena-preload/build/preload.js:525:16)
    at async Preloader._getRequiredAdditionalSpace (/snapshot/versioned-source/node_modules/balena-preload/build/preload.js:531:34)
From previous event:
    at processImmediate (internal/timers.js:456:21)
    at process.topLevelDomainCallback (domain.js:137:15)
From previous event:
    at Preloader.preload (/snapshot/versioned-source/node_modules/balena-preload/build/preload.js:780:14)
    at PreloadCmd.prepareAndPreload (/snapshot/versioned-source/build/commands/preload.js:305:25)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async PreloadCmd.run (/snapshot/versioned-source/build/commands/preload.js:118:13)
    at async PreloadCmd._run (/snapshot/versioned-source/node_modules/@oclif/command/lib/command.js:43:20)
    at async Config.runCommand (/snapshot/versioned-source/node_modules/@oclif/config/lib/config.js:175:24)
    at async CustomMain.run (/snapshot/versioned-source/node_modules/@oclif/command/lib/main.js:27:9)
    at async CustomMain._run (/snapshot/versioned-source/node_modules/@oclif/command/lib/command.js:43:20)
    at async Promise.all (index 1)
    at async oclifRun (/snapshot/versioned-source/build/app.js:75:5)
    at async Object.run (/snapshot/versioned-source/build/app.js:88:9)

For further help or support, visit:
https://www.balena.io/docs/reference/balena-cli/#support-faq-and-troubleshooting

This suddenly happened and I am not sure what is causing this issue...

jellyfish-bot commented 3 years ago

[mpous] This issue has attached support thread https://jel.ly.fish/9699fc64-dbe3-41a5-b578-0209e0c75a0d

openedhardware commented 3 years ago

Seems this function doesn't return manifests correctly?

@pdcastro

openedhardware commented 3 years ago

Seems openBalena issue? I have tried 3.2, 3.2.1, 3.3, and master branch. But no luck.

pdcastro commented 3 years ago

@openedhardware, thanks for reporting it. Do you have steps to reproduce this error? In my previous attempt: https://github.com/balena-io/balena-cli/issues/2249#issuecomment-821744251 , I was not able to reproduce it. If you can, please share, for example:

Docker version and how it was installed:
- apt-get install docker.io ?
- Install using the repository ?
openBalena version already shared: 3.2 and 3.2.1 and 3.3
OS version already shared: Ubuntu 20.04
balena CLI version already shared: 12.44.14 (*)
Full balena CLI command line (not just the output)
balenaOS version (so we can test with a similar image file)
If possible, in case the error is dependent on the app, a minimal app that reproduces it. This could be a minimal Dockerfile, or a link to some git repo.

(*) Although I don't have evidence that it will solve it, balena CLI v12.44.16 made changes to that area of the preload implementation, so it would helpful if you could test that version as well, to rule out whether it makes a difference.

@klutchell: would like to investigate another preload issue? :-) cc: @dfunckt as well, in case you have some idea of what might be happening.

openedhardware commented 3 years ago

Bingo!

v12.44.16 is now working!

Thanks a lot, @pdcastro !!!

balena-io-modules / balena-preload

manifest.manifest.layers is not iterable #243