search

andsens / bootstrap-vz

Bootstrap Debian images for virtualized environments
http://bootstrap-vz.readthedocs.io/
Other
263 stars 143 forks source link

Elasticsearch: Some index files failed to download. #476

Closed kevin-olbrich closed 6 years ago

kevin-olbrich commented 6 years ago

Hi!

Executing: chroot /target/ee083aa0/root apt-get update
Get:1 http://packages.icinga.com/debian icinga-stretch InRelease [6,969 B]
Get:2 http://security.debian.org stretch/updates InRelease [94.3 kB]
Ign:3 http://ftp.de.debian.org/debian stretch InRelease
Get:4 http://ftp.de.debian.org/debian stretch-updates InRelease [91.0 kB]
Get:5 http://packages.icinga.com/debian icinga-stretch/main Sources [9,008 B]
Get:6 http://packages.icinga.com/debian icinga-stretch/main amd64 Packages [28.4 kB]
Get:7 http://ftp.de.debian.org/debian stretch Release [118 kB]
Get:8 http://ftp.de.debian.org/debian stretch Release.gpg [2,434 B]
Get:9 http://security.debian.org stretch/updates/main amd64 Packages [351 kB]
Get:10 http://security.debian.org stretch/updates/main Translation-en [163 kB]
Get:11 http://security.debian.org stretch/updates/contrib amd64 Packages [1,776 B]
Get:12 http://security.debian.org stretch/updates/contrib Translation-en [1,759 B]
Get:13 http://security.debian.org stretch/updates/non-free amd64 Packages [1,268 B]
Get:14 http://security.debian.org stretch/updates/non-free Translation-en [481 B]
Get:15 http://ftp.de.debian.org/debian stretch-updates/main amd64 Packages [11.0 kB]
Get:16 http://ftp.de.debian.org/debian stretch-updates/main Translation-en [6,782 B]
Ign:17 https://artifacts.elastic.co/packages/6.x/apt stable InRelease
Get:18 http://ftp.de.debian.org/debian stretch/main amd64 Packages [7,122 kB]
Ign:19 https://artifacts.elastic.co/packages/6.x/apt stable Release
Ign:20 https://artifacts.elastic.co/packages/6.x/apt stable/main all Packages
Get:21 http://ftp.de.debian.org/debian stretch/main Translation-en [5,394 kB]
Get:22 http://ftp.de.debian.org/debian stretch/contrib amd64 Packages [50.9 kB]
Get:23 http://ftp.de.debian.org/debian stretch/contrib Translation-en [45.9 kB]
Get:24 http://ftp.de.debian.org/debian stretch/non-free amd64 Packages [78.0 kB]
Get:25 http://ftp.de.debian.org/debian stretch/non-free Translation-en [80.1 kB]
Ign:26 https://artifacts.elastic.co/packages/6.x/apt stable/main amd64 Packages
Ign:27 https://artifacts.elastic.co/packages/6.x/apt stable/main Translation-en
Ign:28 https://artifacts.elastic.co/packages/6.x/apt stable/main Translation-en_US
Ign:20 https://artifacts.elastic.co/packages/6.x/apt stable/main all Packages
Ign:26 https://artifacts.elastic.co/packages/6.x/apt stable/main amd64 Packages
Ign:27 https://artifacts.elastic.co/packages/6.x/apt stable/main Translation-en
Ign:28 https://artifacts.elastic.co/packages/6.x/apt stable/main Translation-en_US
Ign:20 https://artifacts.elastic.co/packages/6.x/apt stable/main all Packages
Ign:26 https://artifacts.elastic.co/packages/6.x/apt stable/main amd64 Packages
Ign:27 https://artifacts.elastic.co/packages/6.x/apt stable/main Translation-en
Ign:28 https://artifacts.elastic.co/packages/6.x/apt stable/main Translation-en_US
Ign:20 https://artifacts.elastic.co/packages/6.x/apt stable/main all Packages
Ign:26 https://artifacts.elastic.co/packages/6.x/apt stable/main amd64 Packages
Ign:27 https://artifacts.elastic.co/packages/6.x/apt stable/main Translation-en
Ign:28 https://artifacts.elastic.co/packages/6.x/apt stable/main Translation-en_US
Ign:20 https://artifacts.elastic.co/packages/6.x/apt stable/main all Packages
Ign:26 https://artifacts.elastic.co/packages/6.x/apt stable/main amd64 Packages
Ign:27 https://artifacts.elastic.co/packages/6.x/apt stable/main Translation-en
Ign:28 https://artifacts.elastic.co/packages/6.x/apt stable/main Translation-en_US
Ign:20 https://artifacts.elastic.co/packages/6.x/apt stable/main all Packages
Err:26 https://artifacts.elastic.co/packages/6.x/apt stable/main amd64 Packages
Ign:27 https://artifacts.elastic.co/packages/6.x/apt stable/main Translation-en
Ign:28 https://artifacts.elastic.co/packages/6.x/apt stable/main Translation-en_US
Fetched 13.7 MB in 3s (4,283 kB/s)
Reading package lists...
W: The repository 'https://artifacts.elastic.co/packages/6.x/apt stable Release' does not have a Release file.
E: Failed to fetch https://artifacts.elastic.co/packages/6.x/apt/dists/stable/main/binary-amd64/Packages
E: Some index files failed to download. They have been ignored, or old ones used instead.
Command 'chroot /target/ee083aa0/root apt-get update' returned non-zero exit status 100

This is an issue because Elastic repos use a dynamic generation method for repo files (not file based).

Adding the same repo to an already installed system works flawless.

Any ideas?

Kind regards, Kevin

kevin-olbrich commented 6 years ago

https://artifacts.elastic.co/packages/6.x/apt/dists/stable/main/binary-amd64/Packages This file exists. Wget from the building server is also working correctly.

kevin-olbrich commented 6 years ago 
  sources:
    # mariadb:
      # - deb http://mirror2.hs-esslingen.de/mariadb/repo/10.2/debian stretch main
      # - deb-src http://mirror2.hs-esslingen.de/mariadb/repo/10.2/debian stretch main
    icinga:
      - deb http://packages.icinga.com/debian icinga-stretch main
      - deb-src http://packages.icinga.com/debian icinga-stretch main
    elastic:
      - deb https://artifacts.elastic.co/packages/6.x/apt stable main
andsens commented 6 years ago

This is an issue because Elastic repos use a dynamic generation method for repo files (not file based). Adding the same repo to an already installed system works flawless.

I don't get that part, could you elaborate? What is different during bootstrapping that causes this error?

kevin-olbrich commented 6 years ago

I don't get that part, could you elaborate? What is different during bootstrapping that causes this error?

The Elastic Repo is generated on the fly, it's not static as most repos are (files exposed with index). This is just an idea, not sure if this is the root of the problem. I just never had issues with regular repos.

I am not sure why this issue exists. The same repo works fine on installed systems but failed during bootstrap.

kevin-olbrich commented 6 years ago

I got the same problem with a repo from microsoft:

W: The repository 'https://packages.microsoft.com/debian/9/prod stretch Release' does not have a Release file.
E: Failed to fetch https://packages.microsoft.com/debian/9/prod/dists/stretch/main/binary-amd64/Packages
E: Some index files failed to download. They have been ignored, or old ones used instead.
Command 'chroot /target/abf44eb5/root apt-get update' returned non-zero exit status 100

The build system does not use or define any proxy. Other build jobs which download files to the local machine also have no problem in receiving files over https.

Is there any known issue in using https repos? "apt-transport-https" is listed under "include_packages".

Full log using an alternative repo url from microsoft:

Installing trusted keys
Writing aptitude sources to disk
Configuring apt to always clean everything out when it's done
Updating the package cache
Executing: chroot /target/2a72a9d9/root apt-get update
Get:1 http://packages.icinga.com/debian icinga-stretch InRelease [6,969 B]
Ign:2 http://ftp.de.debian.org/debian stretch InRelease
Get:3 http://security.debian.org stretch/updates InRelease [94.3 kB]
Get:4 http://ftp.de.debian.org/debian stretch-updates InRelease [91.0 kB]
Ign:5 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch InRelease
Ign:6 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch Release
Ign:7 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main all Packages

Ign:8 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main amd64 Packages
Ign:9 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main Translation-en_US
Ign:10 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main Translation-en
Get:11 http://ftp.de.debian.org/debian stretch Release [118 kB]
Ign:7 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main all Packages
Ign:8 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main amd64 Packages
Ign:9 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main Translation-en_US
Get:12 http://packages.icinga.com/debian icinga-stretch/main Sources [9,008 B]
Ign:10 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main Translation-en
Get:13 http://ftp.de.debian.org/debian stretch Release.gpg [2,434 B]
Ign:7 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main all Packages
Get:14 http://packages.icinga.com/debian icinga-stretch/main amd64 Packages [28.4 kB]
Ign:8 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main amd64 Packages
Ign:9 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main Translation-en_US
Ign:10 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main Translation-en
Ign:7 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main all Packages
Ign:8 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main amd64 Packages
Ign:9 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main Translation-en_US
Get:15 http://security.debian.org stretch/updates/main amd64 Packages [360 kB]
Ign:10 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main Translation-en
Ign:7 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main all Packages
Ign:8 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main amd64 Packages
Ign:9 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main Translation-en_US
Ign:10 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main Translation-en
Ign:7 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main all Packages
Err:8 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main amd64 Packages

Get:16 http://security.debian.org stretch/updates/main Translation-en [169 kB]
Ign:9 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main Translation-en_US
Ign:10 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch/main Translation-en
Get:17 http://security.debian.org stretch/updates/contrib amd64 Packages [1,776 B]
Get:18 http://security.debian.org stretch/updates/contrib Translation-en [1,759 B]
Get:19 http://security.debian.org stretch/updates/non-free amd64 Packages [1,268 B]
Get:20 http://security.debian.org stretch/updates/non-free Translation-en [481 B]
Get:21 http://ftp.de.debian.org/debian stretch-updates/main amd64 Packages [11.0 kB]
Get:22 http://ftp.de.debian.org/debian stretch-updates/main Translation-en [6,782 B]
Get:23 http://ftp.de.debian.org/debian stretch/main amd64 Packages [7,122 kB]
Get:24 http://ftp.de.debian.org/debian stretch/main Translation-en [5,394 kB]
Get:25 http://ftp.de.debian.org/debian stretch/contrib amd64 Packages [50.9 kB]
Get:26 http://ftp.de.debian.org/debian stretch/contrib Translation-en [45.9 kB]
Get:27 http://ftp.de.debian.org/debian stretch/non-free amd64 Packages [78.0 kB]
Get:28 http://ftp.de.debian.org/debian stretch/non-free Translation-en [80.1 kB]

Fetched 13.7 MB in 3s (4,189 kB/s)
Reading package lists...
W: The repository 'https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch Release' does not have a Release file.
E: Failed to fetch https://packages.microsoft.com/repos/microsoft-debian-stretch-prod/dists/stretch/main/binary-amd64/Packages
E: Some index files failed to download. They have been ignored, or old ones used instead.
Command 'chroot /target/2a72a9d9/root apt-get update' returned non-zero exit status 100
Traceback (most recent call last):
  File "/var/lib/jenkins/jobs/example-appserver_build_trunk/workspace/bvz-git/bootstrapvz/base/main.py", line 111, in run
    tasklist.run(info=bootstrap_info, dry_run=dry_run)
  File "/var/lib/jenkins/jobs/example-appserver_build_trunk/workspace/bvz-git/bootstrapvz/base/tasklist.py", line 44, in run
    task.run(info)
  File "/var/lib/jenkins/jobs/example-appserver_build_trunk/workspace/bvz-git/bootstrapvz/common/tasks/apt.py", line 200, in run
    'apt-get', 'update'])
  File "/var/lib/jenkins/jobs/example-appserver_build_trunk/workspace/bvz-git/bootstrapvz/common/tools.py", line 14, in log_check_call
    raise e
CalledProcessError: Command 'chroot /target/2a72a9d9/root apt-get update' returned non-zero exit status 100
Rolling back

Tasklist:
    bootstrapvz.plugins.minimize_size.tasks.mounts.RemoveFolderMounts
    bootstrapvz.common.tasks.filesystem.UnmountRoot
    bootstrapvz.common.tasks.partitioning.UnmapPartitions
    bootstrapvz.common.tasks.volume.Detach
    bootstrapvz.common.tasks.filesystem.DeleteMountDir
    bootstrapvz.common.tasks.volume.Delete
    bootstrapvz.common.tasks.workspace.DeleteWorkspace

This works perfectly: root@slde0005:~# curl -s https://packages.microsoft.com/repos/microsoft-debian-stretch-prod/dists/stretch/main/binary-amd64/Packages

The packages are also installed on the build machine, which is also Debian stretch.

kevin-olbrich commented 6 years ago

Found the issue!

bootstrapper:
  workspace: /target
  mirror: http://ftp.de.debian.org/debian/
  include_packages:
   - apt-transport-https
   - ca-certificates

The package "ca-certificates" must be installed for "apt-transport-https" to work properly. Apt could be more verbose in this case (like CA validation failed).

Should I add these details to the docs?

andsens commented 6 years ago

Wow. Yeah, agreed on the unhelpful error message. PR merged and closing issue. Good job figuring it out :-)