search

devrandom / gitian-builder

Build packages in a secure deterministic fashion inside a VM
https://gitian.org/
Other
401 stars 226 forks source link

failed to run on-target -u root apt-get update > var/install.log 2>&1 (RuntimeError) #180

Open cisociall opened 6 years ago

cisociall commented 6 years ago

Hello.I try to build dash wallet with gitian-builder. I follow all steps from https://github.com/dashpay/dash/blob/master/doc/gitian-building.md . But i have errors. What did i do wrong???

Use Oracle VirtualBox, Debian 8.5.0, Dash Core, LXC..

------------------------------------ Error Message ----------------------------- Starting target Checking if target is up Preparing build environment Updating apt-get repository (log in var/install.log) bin/gbuild:21:in "system!": failed to run on-target -u root apt-get update > var/install.log 2>&1 (RuntimeError)

from ./bin/gbuild:105:in `build_one_configuration'
from ./bin/gbuild:300:in `block (2 levels) in <main>'
from ./bin/gbuild:295:in `each'
from ./bin/gbuild:295:in `block in <main>'
from ./bin/gbuild:293:in `each'
from ./bin/gbuild:293:in `<main>'

------------------------------------ var/install.log -----------------------------

Ign http://10.0.3.2:3142 trusty InRelease Get:1 http://10.0.3.2:3142 trusty-security InRelease [65.9 kB] Get:2 http://10.0.3.2:3142 trusty-updates InRelease [65.9 kB] Get:3 http://10.0.3.2:3142 trusty Release.gpg [933 B] Get:4 http://10.0.3.2:3142 trusty Release [58.5 kB] Get:5 http://10.0.3.2:3142 trusty-security/main amd64 Packages [735 kB] Get:6 http://10.0.3.2:3142 trusty-security/universe amd64 Packages [229 kB] Get:7 http://10.0.3.2:3142 trusty-security/main Translation-en [401 kB] Get:8 http://10.0.3.2:3142 trusty-security/universe Translation-en [125 kB] Get:9 http://10.0.3.2:3142 trusty-updates/main amd64 Packages [1079 kB] Get:10 http://10.0.3.2:3142 trusty-updates/universe amd64 Packages [455 kB] Get:11 http://10.0.3.2:3142 trusty-updates/main Translation-en [533 kB] Get:12 http://10.0.3.2:3142 trusty-updates/universe Translation-en [244 kB] Get:13 http://10.0.3.2:3142 trusty/main amd64 Packages [1350 kB] Get:14 http://10.0.3.2:3142 trusty/universe amd64 Packages [5859 kB] Get:15 http://10.0.3.2:3142 trusty/main Translation-en [762 kB] Get:16 http://10.0.3.2:3142 trusty/universe Translation-en [4089 kB] Fetched 16.1 MB in 26s (607 kB/s) W: Failed to fetch bzip2:/var/lib/apt/lists/partial/10.0.3.2:3142_archive.ubuntu.com_ubuntu_dists_trusty-updates_universe_binary-amd64_Packages Hash Sum mismatch

E: Some index files failed to download. They have been ignored, or old ones used instead.

devrandom commented 6 years ago

This is likely a problem with the apt cache on the host machine where you are trying to build.

josephnicholas commented 6 years ago

@devrandom same issue here. Running: Ubuntu 16.04

There is an intermittent behaviour of building, sometimes it does, sometimes it doesn't, but most of the time it doesn't. I am setting up a CI/CD for a project, and running a script to launch the gitian builder.

init.lxc: failed to mount /dev/shm : No such file or directory
sudo: unable to resolve host gitian
stdin: is not a tty
initctl: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused
init.lxc: failed to mount /dev/shm : No such file or directory
init.lxc: failed to mount /dev/shm : No such file or directory
init.lxc: failed to mount /dev/shm : No such file or directory
init.lxc: failed to mount /dev/shm : No such file or directory
./bin/gbuild:21:in `system!': failed to run on-target -u root apt-get update > var/install.log 2>&1 (RuntimeError)
        from ./bin/gbuild:105:in `build_one_configuration'
        from ./bin/gbuild:308:in `block (2 levels) in <main>'
        from ./bin/gbuild:303:in `each'
        from ./bin/gbuild:303:in `block in <main>'
        from ./bin/gbuild:301:in `each'
        from ./bin/gbuild:301:in `<main>'
./bin/gsign:70:in `<main>': result/zcoin-linux-0.13.6-res.yml does not exist (RuntimeError)

gitan-builder/var/install.log

init.lxc: failed to mount /dev/shm : No such file or directory
Ign http://10.0.2.2:3142 trusty InRelease
Ign http://10.0.2.2:3142 trusty-security InRelease
Ign http://10.0.2.2:3142 trusty-updates InRelease
Err http://10.0.2.2:3142 trusty Release.gpg
  Unable to connect to 10.0.2.2:3142:
Err http://10.0.2.2:3142 trusty-security Release.gpg
  Unable to connect to 10.0.2.2:3142:
Err http://10.0.2.2:3142 trusty-updates Release.gpg
  Unable to connect to 10.0.2.2:3142:
Ign http://10.0.2.2:3142 trusty Release
Ign http://10.0.2.2:3142 trusty-security Release
Ign http://10.0.2.2:3142 trusty-updates Release
Err http://10.0.2.2:3142 trusty/main amd64 Packages
  Unable to connect to 10.0.2.2:3142:
Err http://10.0.2.2:3142 trusty/universe amd64 Packages
  Unable to connect to 10.0.2.2:3142:
Err http://10.0.2.2:3142 trusty/main Translation-en
  Unable to connect to 10.0.2.2:3142:
Err http://10.0.2.2:3142 trusty/universe Translation-en
  Unable to connect to 10.0.2.2:3142:
Err http://10.0.2.2:3142 trusty-security/main amd64 Packages
  Unable to connect to 10.0.2.2:3142:
Err http://10.0.2.2:3142 trusty-security/universe amd64 Packages
  Unable to connect to 10.0.2.2:3142:
Err http://10.0.2.2:3142 trusty-security/main Translation-en
  Unable to connect to 10.0.2.2:3142:
Err http://10.0.2.2:3142 trusty-security/universe Translation-en
  Unable to connect to 10.0.2.2:3142:
Err http://10.0.2.2:3142 trusty-updates/main amd64 Packages
  Unable to connect to 10.0.2.2:3142:
Err http://10.0.2.2:3142 trusty-updates/universe amd64 Packages
  Unable to connect to 10.0.2.2:3142:
Err http://10.0.2.2:3142 trusty-updates/main Translation-en
  Unable to connect to 10.0.2.2:3142:
Err http://10.0.2.2:3142 trusty-updates/universe Translation-en
  Unable to connect to 10.0.2.2:3142:
W: Failed to fetch http://10.0.2.2:3142/archive.ubuntu.com/ubuntu/dists/trusty/Release.gpg  Unable to connect to 10.0.2.2:3142:

W: Failed to fetch http://10.0.2.2:3142/security.ubuntu.com/ubuntu/dists/trusty-security/Release.gpg  Unable to connect to 10.0.2.2:3142:

W: Failed to fetch http://10.0.2.2:3142/archive.ubuntu.com/ubuntu/dists/trusty-updates/Release.gpg  Unable to connect to 10.0.2.2:3142:

W: Failed to fetch http://10.0.2.2:3142/archive.ubuntu.com/ubuntu/dists/trusty/main/binary-amd64/Packages  Unable to connect to 10.0.2.2:3142:

W: Failed to fetch http://10.0.2.2:3142/archive.ubuntu.com/ubuntu/dists/trusty/universe/binary-amd64/Packages  Unable to connect to 10.0.2.2:3142:
W: Failed to fetch http://10.0.2.2:3142/archive.ubuntu.com/ubuntu/dists/trusty/main/i18n/Translation-en  Unable to connect to 10.0.2.2:3
142:
W: Failed to fetch http://10.0.2.2:3142/archive.ubuntu.com/ubuntu/dists/trusty/universe/i18n/Translation-en  Unable to connect to 10.0.2
.2:3142:
W: Failed to fetch http://10.0.2.2:3142/security.ubuntu.com/ubuntu/dists/trusty-security/main/binary-amd64/Packages  Unable to connect t
o 10.0.2.2:3142:
W: Failed to fetch http://10.0.2.2:3142/security.ubuntu.com/ubuntu/dists/trusty-security/universe/binary-amd64/Packages  Unable to conne
ct to 10.0.2.2:3142:
W: Failed to fetch http://10.0.2.2:3142/security.ubuntu.com/ubuntu/dists/trusty-security/main/i18n/Translation-en  Unable to connect to 
10.0.2.2:3142:
W: Failed to fetch http://10.0.2.2:3142/security.ubuntu.com/ubuntu/dists/trusty-security/universe/i18n/Translation-en  Unable to connect
 to 10.0.2.2:3142:
W: Failed to fetch http://10.0.2.2:3142/archive.ubuntu.com/ubuntu/dists/trusty-updates/main/binary-amd64/Packages  Unable to connect to 
10.0.2.2:3142:
W: Failed to fetch http://10.0.2.2:3142/archive.ubuntu.com/ubuntu/dists/trusty-updates/universe/binary-amd64/Packages  Unable to connect
 to 10.0.2.2:3142:
W: Failed to fetch http://10.0.2.2:3142/archive.ubuntu.com/ubuntu/dists/trusty-updates/main/i18n/Translation-en  Unable to connect to 10
.0.2.2:3142:
W: Failed to fetch http://10.0.2.2:3142/archive.ubuntu.com/ubuntu/dists/trusty-updates/universe/i18n/Translation-en  Unable to connect t
o 10.0.2.2:3142:
E: Some index files failed to download. They have been ignored, or old ones used instead.

Thanks in advance.

marshalljp commented 6 years ago

It's timing out trying to connect

Err http://10.0.2.2:3142 trusty Release.gpg Unable to connect to 10.0.2.2:3142:

Check and see if your gitian builder machine has IPv6 enabled. Seems like a stupid issue, but as soon as I disabled IPv6 on my builder machine, it worked fine and much faster.

partyka1 commented 6 years ago

For me first flushing and then deleting chains from iptables on virtualbox VM helped:

$ iptables -F
$ iptables -X
Ciip1996 commented 5 years ago

still have this issue. Any ideas? Already tried what @partyka1 said

also I tried disabling ipv6

@devrandom any idea?

kimmy3697 commented 2 years ago

I'm trying to build gitian-builder follow this guys tutorial video. https://www.youtube.com/watch?v=_ti-d5t1WX8&t=1044s&ab_channel=Aqua I have same problems to fetching errors 404 not found. I think all because of those packages are moved to from 'archive' to 'old-releases' so I have to change that address indicate to old-releases but I don't have no idea where should be changed. I changed the file /bin/make-base-vm at line 114 originally from MIRROR=$MIRROR_BASE/archive.ubuntu.com/ubuntu at line 114 to MIRROR=$MIRROR_BASE/old-releases.ubuntu.com/ubuntu at line 114 but this can make base-precise-amd64 but not affects to gbuild things.

ibithub commented 2 years ago

I'm trying to build gitian-builder follow this guys tutorial video. https://www.youtube.com/watch?v=_ti-d5t1WX8&t=1044s&ab_channel=Aqua I have same problems to fetching errors 404 not found. I think all because of those packages are moved to from 'archive' to 'old-releases' so I have to change that address indicate to old-releases but I don't have no idea where should be changed. I changed the file /bin/make-base-vm at line 114 originally from MIRROR=$MIRROR_BASE/archive.ubuntu.com/ubuntu at line 114 to MIRROR=$MIRROR_BASE/old-releases.ubuntu.com/ubuntu at line 114 but this can make base-precise-amd64 but not affects to gbuild things.

In gitian-builder/target-bin/bootstrap-fixup.in

Change lines 9-11: Change archive.ubuntu.com/security.ubuntu.com to old-releases.ubuntu.com

if [ $DISTRIB_NAME = "Ubuntu" ]; then echo "deb http://HOSTIP:3142/archive.ubuntu.com/ubuntu $DISTRIB_CODENAME main universe" > $1/etc/apt/sources.list echo "deb http://HOSTIP:3142/security.ubuntu.com/ubuntu $DISTRIB_CODENAME-security main universe" >> $1/etc/apt/sources.list echo "deb http://HOSTIP:3142/archive.ubuntu.com/ubuntu $DISTRIB_CODENAME-updates main universe" >> $1/etc/apt/sources.list

Kailkox commented 2 years ago

Hi, im from aqua too, I done everything that @ibithub and @kimmy3697 wrote, I'm still stuck, some move was to good side, but didn't ended problem, I get something like that: ./bin/gbuild:23:in `system!': failed to run on-target -u root bash < target-bin/upgrade-system.sh >> var/install.log 2>&1 (RuntimeError) In install.log is: Processing triggers for libc-bin ... ldconfig deferred processing now taking place debian error stdin: is not tty Please help, thx for any tip

dotada commented 2 years ago

Have you found a solution? I have the same problem.

Kailkox commented 2 years ago

Have you found a solution? I have the same problem.

im trying friend

bitoreum commented 2 years ago

I came from Aqua youtube as well having the same issues like many people here:

debian@debian:~/gitian-builder$ ./bin/gbuild ../bugacoin/contrib/gitian-descriptors/boost-win32.yml --- Building for precise amd64 --- Stopping target if it is up Making a new image copy sudo: unable to resolve host gitian stdin: is not a tty Reading package lists... Done Building dependency tree... Done The following packages will be REMOVED: rsyslog ubuntu-minimal 0 upgraded, 0 newly installed, 2 to remove and 0 not upgraded. After this operation, 1289 kB disk space will be freed. (Reading database ... 16075 files and directories currently installed.) Removing ubuntu-minimal ... Removing rsyslog ... initctl: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused Purging configuration files for rsyslog ... dpkg: warning: while removing rsyslog, directory '/etc/apparmor.d/local' not empty so not removed. Processing triggers for ureadahead ... Adding 'local diversion of /sbin/initctl to /sbin/initctl.distrib' Adding 'local diversion of /usr/bin/ischroot to /usr/bin/ischroot.distrib' Adding 'local diversion of /usr/sbin/policy-rc.d to /usr/sbin/policy-rc.d.distrib' Starting target Checking if target is up Preparing build environment Updating apt-get repository (log in var/install.log) ./bin/gbuild:23:in system!': failed to run on-target -u root apt-get update >> var/install.log 2>&1 (RuntimeError) from ./bin/gbuild:117:inbuild_one_configuration' from ./bin/gbuild:341:in block (2 levels) in <main>' from ./bin/gbuild:336:ineach' from ./bin/gbuild:336:in block in <main>' from ./bin/gbuild:334:ineach' from ./bin/gbuild:334:in `

'

I can confirm that after modifying the 3 lines in target-bin/bootstrap-fixup.in to old-releases.ubuntu.com it started working, currently going forward. I will update if everything goes ok.

debian@debian:~/gitian-builder$ ./bin/gbuild ../bugacoin/contrib/gitian-descriptors/boost-win32.yml --- Building for precise amd64 --- Stopping target if it is up Making a new image copy sudo: unable to resolve host gitian stdin: is not a tty Reading package lists... Done Building dependency tree... Done The following packages will be REMOVED: rsyslog ubuntu-minimal 0 upgraded, 0 newly installed, 2 to remove and 0 not upgraded. After this operation, 1289 kB disk space will be freed. (Reading database ... 16075 files and directories currently installed.) Removing ubuntu-minimal ... Removing rsyslog ... initctl: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused Purging configuration files for rsyslog ... dpkg: warning: while removing rsyslog, directory '/etc/apparmor.d/local' not empty so not removed. Processing triggers for ureadahead ... Adding 'local diversion of /sbin/initctl to /sbin/initctl.distrib' Adding 'local diversion of /usr/bin/ischroot to /usr/bin/ischroot.distrib' Adding 'local diversion of /usr/sbin/policy-rc.d to /usr/sbin/policy-rc.d.distrib' Starting target Checking if target is up Preparing build environment Updating apt-get repository (log in var/install.log) Installing additional packages (log in var/install.log) Upgrading system, may take a while (log in var/install.log) Creating package manifest stdin: is not a tty Creating build script (var/build-script) Running build script (log in var/build.log)

bitoreum commented 2 years ago

if you get then the author date error format:

change line in gitian-builder/bin/gbuild

author_date = 'cd inputs/#{dir} && TZ=UTC git log --date=format-local:%F %T --format="%ad" -1'.strip

by this

author_date = 'cd inputs/#{dir} && TZ=UTC git log --date=local --format="%ad" -1'.strip

dotada commented 2 years ago

after modifying the 3 lines I get failed to run on-target -u root -e DEBIAN_FRONTEND=noninteractive apt-get --no-install-recommends -y install mingw-w64 g++-mingw-w64 faketime zip >> var/install.log 2>&1 (RuntimeError)

install.log

Err http://10.0.3.2 precise InRelease

Err http://10.0.3.2 precise-security InRelease

Err http://10.0.3.2 precise-security InRelease

Err http://10.0.3.2 precise-updates InRelease

Err http://10.0.3.2 precise Release.gpg
  Unable to connect to 10.0.3.2:3142:
Err http://10.0.3.2 precise-security Release.gpg
  Unable to connect to 10.0.3.2:3142:
Err http://10.0.3.2 precise-updates Release.gpg
  Unable to connect to 10.0.3.2:3142:
Reading package lists...
W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise/InRelease

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise-security/InRelease

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise-updates/InRelease

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise/Release.gpg  Unable to connect to 10.0.3.2:3142:

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise-security/Release.gpg  Unable to connect to 10.0.3.2:3142:

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise-updates/Release.gpg  Unable to connect to 10.0.3.2:3142:

W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package mingw-w64
E: Unable to locate package g++-mingw-w64
E: Couldn't find any package by regex 'g++-mingw-w64'
E: Unable to locate package faketime
E: Unable to locate package zip
bitoreum commented 2 years ago

if you get then the author date error format:

change line in gitian-builder/bin/gbuild

author_date = 'cd inputs/#{dir} && TZ=UTC git log --date=format-local:%F %T --format="%ad" -1'.strip

by this

author_date = 'cd inputs/#{dir} && TZ=UTC git log --date=local --format="%ad" -1'.strip

read my second reply on this:

if you get then the author date error format:

change line in gitian-builder/bin/gbuild

author_date = 'cd inputs/#{dir} && TZ=UTC git log --date=format-local:%F %T --format="%ad" -1'.strip

by this

author_date = 'cd inputs/#{dir} && TZ=UTC git log --date=local --format="%ad" -1'.strip

Kailkox commented 2 years ago

after modifying the 3 lines I get failed to run on-target -u root -e DEBIAN_FRONTEND=noninteractive apt-get --no-install-recommends -y install mingw-w64 g++-mingw-w64 faketime zip >> var/install.log 2>&1 (RuntimeError)

install.log

Err http://10.0.3.2 precise InRelease

Err http://10.0.3.2 precise-security InRelease

Err http://10.0.3.2 precise-security InRelease

Err http://10.0.3.2 precise-updates InRelease

Err http://10.0.3.2 precise Release.gpg
  Unable to connect to 10.0.3.2:3142:
Err http://10.0.3.2 precise-security Release.gpg
  Unable to connect to 10.0.3.2:3142:
Err http://10.0.3.2 precise-updates Release.gpg
  Unable to connect to 10.0.3.2:3142:
Reading package lists...
W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise/InRelease

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise-security/InRelease

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise-updates/InRelease

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise/Release.gpg  Unable to connect to 10.0.3.2:3142:

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise-security/Release.gpg  Unable to connect to 10.0.3.2:3142:

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise-updates/Release.gpg  Unable to connect to 10.0.3.2:3142:

W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package mingw-w64
E: Unable to locate package g++-mingw-w64
E: Couldn't find any package by regex 'g++-mingw-w64'
E: Unable to locate package faketime
E: Unable to locate package zip

i'm in same problem, something worked and again is on it

ibithub commented 2 years ago

after modifying the 3 lines I get failed to run on-target -u root -e DEBIAN_FRONTEND=noninteractive apt-get --no-install-recommends -y install mingw-w64 g++-mingw-w64 faketime zip >> var/install.log 2>&1 (RuntimeError) install.log

Err http://10.0.3.2 precise InRelease

Err http://10.0.3.2 precise-security InRelease

Err http://10.0.3.2 precise-security InRelease

Err http://10.0.3.2 precise-updates InRelease

Err http://10.0.3.2 precise Release.gpg
  Unable to connect to 10.0.3.2:3142:
Err http://10.0.3.2 precise-security Release.gpg
  Unable to connect to 10.0.3.2:3142:
Err http://10.0.3.2 precise-updates Release.gpg
  Unable to connect to 10.0.3.2:3142:
Reading package lists...
W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise/InRelease

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise-security/InRelease

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise-updates/InRelease

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise/Release.gpg  Unable to connect to 10.0.3.2:3142:

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise-security/Release.gpg  Unable to connect to 10.0.3.2:3142:

W: Failed to fetch http://10.0.3.2:3142/old-releases.ubuntu.com/ubuntu/dists/precise-updates/Release.gpg  Unable to connect to 10.0.3.2:3142:

W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package mingw-w64
E: Unable to locate package g++-mingw-w64
E: Couldn't find any package by regex 'g++-mingw-w64'
E: Unable to locate package faketime
E: Unable to locate package zip

i'm in same problem, something worked and again is on it

Haven't tested this yet, but I may have downloaded the precise before changing those 3 lines. Will run thru it again to see if I can help find a solution. Just gotta make sure you have all the packages downloaded successfully.

Kailkox commented 1 year ago

Hi, im from aqua too, I done everything that @ibithub and @kimmy3697 wrote, I'm still stuck, some move was to good side, but didn't ended problem, I get something like that: ./bin/gbuild:23:in `system!': failed to run on-target -u root bash < target-bin/upgrade-system.sh >> var/install.log 2>&1 (RuntimeError) In install.log is: Processing triggers for libc-bin ... ldconfig deferred processing now taking place debian error stdin: is not tty Please help, thx for any tip

I repaired it few months ago by deleting Upgrade System functions and everything that made error about not existing function, idk if I should but except for that it works.