Build issue - photon OS

[sudhesam]

I followed the build instructions from https://blogs.vmware.com/cloudnative/building-photon-os-from-source/

I am running 'sudo make all' and I see the following error. Any help appreciated.

---

2017-04-24 09:56:40,109 - Serializable Spec objects - INFO - -------------------------------------------------------------------------- 2017-04-24 09:56:40,109 - Serializable Spec objects - INFO - Starting Log 2017-04-24 09:56:40,109 - Serializable Spec objects - INFO - -------------------------------------------------------------------------- 2017-04-24 09:56:41,020 - /home/vmware/GIT_REPOS/photon/stage/LOGS/Main - INFO - List of packages to build: 2017-04-24 09:56:41,020 - /home/vmware/GIT_REPOS/photon/stage/LOGS/Main - INFO - ['cython3', 'pth', 'iputils', 'liblogging', 'fcgi', 'gc', 'rsyslog', 'librsync', 'gdbm', 'gpgme', 'python-jinja2', 'cython', 'grub2', 'trousers', 'openscap', 'python-setuptools', 'unzip', 'mpc', 'gnome-common', 'mkinitcpio', 'gnutls', 'npth', 'python-packaging', 'micro-config-drive', 'libyaml', 'python-pip', 'mercurial', 'libnl', 'falco', 'tboot', 'perl-Module-Build', 'zlib', 'dosfstools', 'rng-tools', 'go', 'libwebp', 'iperf', 'python-automat', 'libunistring', 'elfutils', 'inotify-tools', 'libgpg-error', 'boost', 'gawk', 'httpd-mod_jk', 'xinetd', 'pygobject', 'device-mapper-multipath', 'c-ares', 'perl-File-HomeDir', 'rubygem-rbvmomi', 'gmock', 'jansson', 'apache-maven', 'rpm-ostree', 'ltrace', 'mesos', 'apr', 'tmux', 'vim', 'deltarpm', 'python-lxml', 'httpd', 'python-webob', 'zip', 'libnetfilter_conntrack', 'syslog-ng', 'perl', 'tzdata', 'm4', 'bash', 'libgsystem', 'autogen', 'fatrace', 'perl-Types-Serialiser', 'python-simplejson', 'apache-tomcat', 'python-ntplib', 'pycurl', 'mc', 'openjdk', 'libtasn1', 'python-appdirs', 'python-Pygments', 'krb5', 'ethtool', 'sqlite', 'tar', 'sudo', 'logrotate', 'vmware-dns', 'socat', 'gcc', 'vmware-ic-config', 'libdnet', 'tdnf', 'python-pycparser', 'docbook-xsl', 'ebtables', 'python-greenlet', 'protobuf', 'tcsh', 'gperf', 'openvswitch', 'rubygem-terminal-table', 'clang', 'netcat', 'perl-libintl', 'grub2-efi', 'urlgrabber', 'pciutils', 'libnss-ato', 'iana-etc', 'libseccomp', 'python-daemon', 'kbd', 'kexec-tools', 'jna', 'perl-common-sense', 'rubygem-trollop', 'perl-Object-Accessor', 'systemd', 'coreutils', 'perl-JSON-XS', 'erlang', 'pixman', 'flex', 'crash', 'blktrace', 'python-pam', 'libatomic_ops', 'lzo', 'cifs-utils', 'rubygem-highline', 'pango', 'python-prettytable', 'cloud-utils', 'python-pbr', 'nspr', 'dbus', 'python-prompt_toolkit', 'psmisc', 'python-service_identity', 'python-pyOpenSSL', 'python-attrs', 'cairo', 'perl-Config-IniFiles', 'PyYAML', 'distrib-compat', 'perl-DBIx-Simple', 'librelp', 'unbound', 'efibootmgr', 'sshpass', 'dhcp', 'python-incremental', 'libconfig', 'sendmail', 'ntp', 'vmware-ca', 'procmail', 'git', 'libvirt', 'yum-metadata-parser', 'haveged', 'eventlog', 'openssh', 'chrpath', 'openssl', 'libarchive', 'openldap', 'yum', 'cloud-init', 'ndctl', 'lvm2', 'tcp_wrappers', 'parted', 'harfbuzz', 'xz', 'lshw', 'docker-volume-vsphere', 'python2', 'sshfs', 'python-sphinx', 'docbook-xml', 'mpfr', 'python-werkzeug', 'NetworkManager', 'zeromq', 'vmware-afd', 'vmware-lightwave-clients', 'python-configparser', 'pcstat', 'libunwind', 'hdparm', 'python-docutils', 'libevent', 'python-setuptools_scm', 'libtar', 'guile', 'google-startup-scripts', 'xmlto', 'perl-List-MoreUtils', 'python-mako', 'gnupg', 'zsh', 'perl-File-Which', 'xfsprogs', 'man-db', 'ceph', 'python-ipaddr', 'vmware-event', 'libnetfilter_cttimeout', 'libxslt', 'vmware-sts', 'libgudev', 'paramiko', 'net-snmp', 'strongswan', 'which', 'libcap', 'python-gevent', 'linux-esx', 'perl-Exporter-Tiny', 'perl-IO-Socket-SSL', 'ecdsa', 'libdb', 'efivar', 'initramfs', 'glog', 'gzip', 'nicstat', 'cronie', 'photon-release', 'nano', 'strace', 'sg3_utils', 'nss-altfiles', 'nettle', 'rabbitmq-server', 'libusb', 'rubygem-backports', 'linux-secure', 'runit', 'libpipeline', 'google-daemon', 'libtirpc', 'findutils', 'popt', 'gobject-introspection', 'gtest', 'fontconfig', 'perl-DBD-SQLite', 'libnetfilter_queue', 'iptables', 'createrepo', 'cpio', 'photon-repos', 'lz4', 'ant-contrib', 'ostree', 'util-linux', 'freetype2', 'libfastjson', 'gdb', 'python-requests', 'patch', 'python-idna', 'XML-Parser', 'bison', 'net-tools', 'lttng-ust', 'rpcbind', 'json-c', 'linux-api-headers', 'zookeeper', 'aufs-util', 'sed', 'dkms', 'glib-networking', 'kmod', 'dstat', 'libestr', 'ipvsadm', 'lua', 'perl-YAML-Tiny', 'finger', 'python-coverage', 'libpng', 'libgd', 'python-iniparse', 'irqbalance', 'fakeroot-ng', 'systemtap', 'cyrus-sasl', 'python-urllib3', 'python-lockfile', 'python-cffi', 'libgcrypt', 'gtk-doc', 'libsolv', 'itstool', 'python-pyvmomi', 'valgrind', 'sysstat', 'lsof', 'python-jsonpatch', 'binutils', 'build-essential', 'python-configobj', 'perl-DBI', 'iotop', 'jaxws-ri', 'linux', 'rpm', 'libssh2', 'python-zmq', 'newt', 'bridge-utils', 'cracklib', 'procps-ng', 'ncurses', 'python-numpy', 'nfs-utils', 'geoip-api-c', 'ruby', 'python-wcwidth', 'pam_tacplus', 'rubygem-nokogiri', 'libaio', 'fleet', 'libselinux', 'python-pyvim', 'libassuan', 'nodejs', 'libxml2', 'python-jsonpointer', 'expat', 'ktap', 'thin-provisioning-tools', 'ansible', 'python-ipaddress', 'userspace-rcu', 'chkconfig', 'texinfo', 'Linux-PAM', 'leveldb', 'make', 'glibmm', 'libcap-ng', 'dbus-glib', 'sqlite2', 'libsoup', 'unixODBC', 'cgroup-utils', 'perl-WWW-Curl', 'cdrkit', 'fuse', 'perl-Net-SSLeay', 'xml-security-c', 'librepo', 'audit', 'rrdtool', 'compat-gdbm', 'libsepol', 'filesystem', 'iproute2', 'docker', 'keyutils', 'netkit-telnet', 'less', 'vsftpd', 'tcl', 'python-constantly', 'rubygem-zip', 'sysdig', 'mingetty', 'libpcap', 'lapack', 'subversion', 'snappy', 'python-Twisted', 'gmp', 'ipset', 'bindutils', 'swig', 'apr-util', 'nginx', 'ca-certificates', 'python-pytz', 'shadow', 'libnetfilter_cthelper', 'python3', 'flannel', 'python-markupsafe', 'python-netaddr', 'python-dnspython', 'ndsend', 'dnsmasq', 'protobuf-c', 'man-pages', 'pkg-config', 'python-antlrpythonruntime', 'XML-XPath', 'vmware-directory', 'json-glib', 'libmspack', 'gperftools', 'python-pyasn1', 'python-cryptography', 'rsync', 'GConf2', 'glibc', 'redis', 'btrfs-progs', 'automake', 'msr-tools', 'cni', 'rubygem-mini_portile2', 'rubygem-unicode-display_width', 'bc', 'tree', 'xerces-c', 'rpm-ostree-toolbox', 'python-pyasn1-modules', 'dmidecode', 'vmware-lightwave-server', 'libsigc++', 'apache-ant', 'wget', 'python-pyparsing', 'haproxy', 'atftp', 'postgresql', 'perl-Module-Install', 'libjpeg-turbo', 'acl', 'iptraf', 'dialog', 'cve-check-tool', 'ceph-deploy', 'lldpad', 'libtiff', 'libmnl', 'nss', 'pcre', 'WALinuxAgent', 'python-sqlalchemy', 'PyPAM', 'json_spirit', 'lldb', 'etcd', 'rubygem-libxml-ruby', 'ctags', 'groff', 'tcpdump', 'glib', 'traceroute', 'python-enum', 'rubygem-builder', 'grep', 'libnfnetlink', 'dracut', 'gptfdisk', 'attr', 'initscripts', 'python-m2r', 'perl-CGI', 'tiptop', 'autoconf', 'ipxe', 'kaigen-gothic-cjk', 'libndp', 'pgbouncer', 'python-fuse', 'libhif', 'file', 'perl-Module-ScanDeps', 'curl', 'check', 'lsscsi', 'python-zope.interface', 'bzip2', 'python-six', 'cmake', 'gnuplot', 'python-defusedxml', 'gettext', 'open-vm-tools', 'llvm', 'python-mistune', 'kubernetes', 'lttng-tools', 'likewise-open', 'asciidoc', 'intltool', 'libtool', 'hawkey', 'libffi', 'readline', 'slang', 'netmgmt', 'perl-JSON-Any', 'ddclient', 'pycrypto', 'perl-YAML', 'diffutils', 'usbutils', 'e2fsprogs', 'rubygem-mini_portile', 'commons-daemon', 'libksba', 'nasm'] 2017-04-24 09:56:41,022 - PackageManager - INFO - -------------------------------------------------------------------------- 2017-04-24 09:56:41,022 - PackageManager - INFO - Starting Log 2017-04-24 09:56:41,022 - PackageManager - INFO - -------------------------------------------------------------------------- 2017-04-24 09:56:41,022 - Toolchain Utils - INFO - -------------------------------------------------------------------------- 2017-04-24 09:56:41,023 - Toolchain Utils - INFO - Starting Log 2017-04-24 09:56:41,023 - Toolchain Utils - INFO - -------------------------------------------------------------------------- 2017-04-24 09:56:41,023 - Toolchain Utils - INFO - Building core tool chain packages..... 2017-04-24 09:56:41,034 - Toolchain Utils - INFO - Preparing build environment 2017-04-24 09:56:41,096 - Toolchain Utils - INFO - Successfully prepared chroot:/home/vmware/GIT_REPOS/photon/stage/photonroot/build-core-toolchain 2017-04-24 09:56:41,097 - Toolchain Utils - INFO - Installing Tool Chain RPMS....... 2017-04-24 09:56:41,274 - Toolchain Utils - INFO - No old version of bzip2-libs exists, skip until the new version is built 2017-04-24 09:56:41,410 - Toolchain Utils - INFO - No old version of xz-libs exists, skip until the new version is built 2017-04-24 09:56:41,485 - Toolchain Utils - INFO - No old version of nspr-devel exists, skip until the new version is built 2017-04-24 09:56:41,500 - Toolchain Utils - INFO - No old version of sqlite-libs exists, skip until the new version is built 2017-04-24 09:56:41,518 - Toolchain Utils - INFO - No old version of nss-libs exists, skip until the new version is built 2017-04-24 09:56:41,567 - Toolchain Utils - INFO - No old version of expat-libs exists, skip until the new version is built 2017-04-24 09:56:41,682 - Toolchain Utils - INFO - No old version of rpm-libs exists, skip until the new version is built 2017-04-24 09:56:41,709 - Toolchain Utils - DEBUG - Installing rpms: filesystem linux-api-headers glibc glibc-devel zlib zlib-devel file binutils binutils-devel gmp gmp-devel mpfr mpfr-devel mpc libgcc libgcc-devel libstdc++ libstdc++-devel libgomp libgomp-devel gcc pkg-config ncurses bash bzip2 bzip2-devel sed ncurses-devel procps-ng coreutils m4 grep readline diffutils gawk findutils gettext gzip make patch util-linux util-linux-devel tar xz libtool flex flex-devel bison readline-devel lua lua-devel popt popt-devel nspr sqlite-autoconf nss nss-devel elfutils-libelf elfutils elfutils-libelf-devel elfutils-devel expat libffi libpipeline gdbm perl texinfo autoconf automake openssl openssl-devel python2 python2-libs python2-devel libcap rpm rpm-build rpm-devel groff man-pages cpio go rpm: RPM should not be used directly install RPM packages, use Alien instead! rpm: However assuming you know what you are doing... warning: /home/vmware/GIT_REPOS/photon/stage/PUBLISHRPMS/x86_64/filesystem-1.0-7.ph1.x86_64.rpm: Header V3 RSA/SHA1 Signature, key ID 66fd4949: NOKEY warning: %post(bash-4.4-1.ph1.x86_64) scriptlet failed, exit status 127 2017-04-24 09:56:47,433 - Toolchain Utils - INFO - Successfully installed all Tool Chain RPMS in Chroot:/home/vmware/GIT_REPOS/photon/stage/photonroot/build-core-toolchain 2017-04-24 09:56:47,467 - Toolchain Utils - INFO - Building rpm's for package:filesystem 2017-04-24 09:56:47,467 - Toolchain Utils - INFO - Building rpm.... 2017-04-24 09:56:47,467 - Toolchain Utils - INFO - rpmbuild -ba --clean --define \"dist .ph2\" --nocheck --define \"KERNEL_RELEASE 2.ph2\" --define \"dist .ph2\" --define \"photon_build_number dcf60ec\" --define \"KERNEL_VERSION 4.9.13\" --define \"kernelsubrelease .12577282\" --define \"photon_release_version 1.0\" /usr/src/photon/SPECS/filesystem.spec 2017-04-24 09:56:47,754 - Toolchain Utils - INFO - RPM build is successful 2017-04-24 09:56:48,116 - Toolchain Utils - INFO - Successfully destroyed chroot:/home/vmware/GIT_REPOS/photon/stage/photonroot/build-core-toolchain 2017-04-24 09:56:48,126 - Toolchain Utils - INFO - Preparing build environment 2017-04-24 09:56:48,186 - Toolchain Utils - INFO - Successfully prepared chroot:/home/vmware/GIT_REPOS/photon/stage/photonroot/build-core-toolchain 2017-04-24 09:56:48,187 - Toolchain Utils - INFO - Installing Tool Chain RPMS....... 2017-04-24 09:56:48,364 - Toolchain Utils - INFO - No old version of bzip2-libs exists, skip until the new version is built 2017-04-24 09:56:48,501 - Toolchain Utils - INFO - No old version of xz-libs exists, skip until the new version is built 2017-04-24 09:56:48,577 - Toolchain Utils - INFO - No old version of nspr-devel exists, skip until the new version is built 2017-04-24 09:56:48,592 - Toolchain Utils - INFO - No old version of sqlite-libs exists, skip until the new version is built 2017-04-24 09:56:48,606 - Toolchain Utils - INFO - No old version of nss-libs exists, skip until the new version is built 2017-04-24 09:56:48,655 - Toolchain Utils - INFO - No old version of expat-libs exists, skip until the new version is built 2017-04-24 09:56:48,773 - Toolchain Utils - INFO - No old version of rpm-libs exists, skip until the new version is built 2017-04-24 09:56:48,801 - Toolchain Utils - DEBUG - Installing rpms: filesystem linux-api-headers glibc glibc-devel zlib zlib-devel file binutils binutils-devel gmp gmp-devel mpfr mpfr-devel mpc libgcc libgcc-devel libstdc++ libstdc++-devel libgomp libgomp-devel gcc pkg-config ncurses bash bzip2 bzip2-devel sed ncurses-devel procps-ng coreutils m4 grep readline diffutils gawk findutils gettext gzip make patch util-linux util-linux-devel tar xz libtool flex flex-devel bison readline-devel lua lua-devel popt popt-devel nspr sqlite-autoconf nss nss-devel elfutils-libelf elfutils elfutils-libelf-devel elfutils-devel expat libffi libpipeline gdbm perl texinfo autoconf automake openssl openssl-devel python2 python2-libs python2-devel libcap rpm rpm-build rpm-devel groff man-pages cpio go rpm: RPM should not be used directly install RPM packages, use Alien instead! rpm: However assuming you know what you are doing... warning: /home/vmware/GIT_REPOS/photon/stage/PUBLISHRPMS/noarch/linux-api-headers-4.2.0-1.ph1.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 66fd4949: NOKEY warning: %post(bash-4.4-1.ph1.x86_64) scriptlet failed, exit status 127 2017-04-24 09:56:54,988 - Toolchain Utils - INFO - Successfully installed all Tool Chain RPMS in Chroot:/home/vmware/GIT_REPOS/photon/stage/photonroot/build-core-toolchain 2017-04-24 09:56:55,023 - Toolchain Utils - INFO - Building rpm's for package:linux-api-headers 2017-04-24 09:56:55.027018: Downloading https://dl.bintray.com/vmware/photon_sources/1.0/linux-4.9.13.tar.xz... ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",) 2017-04-24 09:56:55,210 - Toolchain Utils - ERROR - Unable to build tool chain. Traceback (most recent call last): File "/home/vmware/GIT_REPOS/photon/support/package-builder/ToolChainUtils.py", line 97, in buildCoreToolChainPackages pkgUtils.buildRPMSForGivenPackage(package, chrootID, listBuildOptionPackages, pkgBuildOptionFile, destLogPath) File "/home/vmware/GIT_REPOS/photon/support/package-builder/PackageUtils.py", line 164, in buildRPMSForGivenPackage self.copySourcesTobuildroot(listSourcesFiles,package,chrootSourcePath) File "/home/vmware/GIT_REPOS/photon/support/package-builder/PackageUtils.py", line 115, in copySourcesTobuildroot PullSources.get(source, sha1, constants.sourcePath, constants.pullsourcesConfig) File "/home/vmware/GIT_REPOS/photon/support/package-builder/PullSources.py", line 48, in get raise Exception("Missing source: "+source) Exception: Missing source: linux-4.9.13.tar.xz 2017-04-24 09:56:55,211 - PackageManager - ERROR - Unable to build tool chain 2017-04-24 09:56:55,212 - PackageManager - ERROR - Missing source: linux-4.9.13.tar.xz 2017-04-24 09:56:55,212 - /home/vmware/GIT_REPOS/photon/stage/LOGS/Main - ERROR - Caught an exception 2017-04-24 09:56:55,212 - /home/vmware/GIT_REPOS/photon/stage/LOGS/Main - ERROR - Missing source: linux-4.9.13.tar.xz Traceback (most recent call last): File "/home/vmware/GIT_REPOS/photon/support/package-builder/builder.py", line 130, in main buildPackagesForAllSpecs(listBuildOptionPackages, options.pkgBuildOptionFile, logger, options.buildThreads, pkgInfoJsonFile) File "/home/vmware/GIT_REPOS/photon/support/package-builder/builder.py", line 265, in buildPackagesForAllSpecs pkgManager.buildPackages(listPackages, listBuildOptionPackages, pkgBuildOptionFile, buildThreads) File "/home/vmware/GIT_REPOS/photon/support/package-builder/PackageManager.py", line 145, in buildPackages self.buildToolChainPackages(listBuildOptionPackages, pkgBuildOptionFile, buildThreads) File "/home/vmware/GIT_REPOS/photon/support/package-builder/PackageManager.py", line 128, in buildToolChainPackages self.buildToolChain() File "/home/vmware/GIT_REPOS/photon/support/package-builder/PackageManager.py", line 125, in buildToolChain raise e Exception: Missing source: linux-4.9.13.tar.xz Makefile:228: recipe for target 'packages' failed make: *** [packages] Error 1

---

[dthaluru]

@sudhesam Seems like you are not able to download source from this below url? https://dl.bintray.com/vmware/photon_sources/1.0/linux-4.9.13.tar.xz Can you try downloading the file manually on your build system? If you are able to download can you retry the command "sudo make all"?

If you are not able to download the file, can you give us details about your geographic location of your build system?

[ppadmavilasom]

wget --spider https://dl.bintray.com/vmware/photon_sources/1.0/linux-4.9.13.tar.xz Spider mode enabled. Check if remote file exists. --2017-04-24 18:00:38-- https://dl.bintray.com/vmware/photon_sources/1.0/linux-4.9.13.tar.xz Resolving dl.bintray.com... 35.165.107.204, 52.37.135.119, 35.165.107.204 Connecting to dl.bintray.com|35.165.107.204|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 93204620 (89M) [application/octet-stream] Remote file exists.

Please try again

[YustasSwamp]

2017-04-24 09:56:55.027018: Downloading https://dl.bintray.com/vmware/photon_sources/1.0/linux-4.9.13.tar.xz... ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",)

Client certificates missconfiguration?

[sudhesam]

vmware@fmbuild:~/workspaces/photon$ wget --spider https://dl.bintray.com/vmware/photon_sources/1.0/linux-4.9.13.tar.xz Spider mode enabled. Check if remote file exists. --2017-04-25 09:39:10-- https://dl.bintray.com/vmware/photon_sources/1.0/linux-4.9.13.tar.xz Resolving dl.bintray.com (dl.bintray.com)... 35.165.107.204, 52.37.135.119 Connecting to dl.bintray.com (dl.bintray.com)|35.165.107.204|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 93204620 (89M) [application/octet-stream] Remote file exists.

Still seeing the same build issue.

To the question of "Client certificates missconfiguration", I believe the wget should also fail if the client cert is missconfigured. Just my 2 cents.

[sudhesam]

Sorry for the piecemeal.

@dthaluru : Geographic location of build system is "San Diego, CA"

[dthaluru]

@sudhesam I believe client certificates are not able to verified as @YustasSwamp mentioned.

To get around this issue, can you modify the below line in your code and pass argument "verify=false". Otherwise you need to fix your certificates. https://github.com/vmware/photon/blob/master/support/package-builder/PullSources.py#L78

[sudhesam]

New issue after passing "verify=false" in PullSources.py :

warning: %post(bash-4.4-2.ph2.x86_64) scriptlet failed, exit status 127 /bin/sh: error while loading shared libraries: libreadline.so.7: cannot open shared object file: No such file or directory warning: %post(glibc-2.25-1.ph2.x86_64) scriptlet failed, exit status 127 2017-04-25 14:24:32,499 - build-nodejs - INFO - Successfully installed all Tool Chain RPMS in Chroot:/home/vmware/GIT_REPOS/photon/stage/photonroot/build-nodejs 2017-04-25 14:24:32,555 - build-nodejs - INFO - Installing the build time dependent packages...... 2017-04-25 14:24:32,556 - build-nodejs - INFO - Finished installing the build time dependent packages...... 2017-04-25 14:24:32,598 - build-nodejs - INFO - Building rpm's for package:nodejs 2017-04-25 14:24:32.604826: Downloading https://dl.bintray.com/vmware/photon_sources/1.0/node-7.7.4.tar.gz... /usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning) /usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning) ('Connection broken: error("(104, \'ECONNRESET\')",)', error("(104, 'ECONNRESET')",)) 2017-04-25 14:25:11,767 - build-nodejs - ERROR - Failed while building package:nodejs 2017-04-25 14:25:11,767 - build-nodejs - DEBUG - Chroot with ID: /home/vmware/GIT_REPOS/photon/stage/photonroot/build-nodejs not deleted for debugging. tail: cannot open '/home/vmware/GIT_REPOS/photon/stage/LOGS/build-nodejs/nodejs.log' for reading: No such file or directory 2017-04-25 14:25:11,773 - build-nodejs - DEBUG - 2017-04-25 14:25:11,773 - build-nodejs - ERROR - Missing source: node-7.7.4.tar.gz 2017-04-25 14:25:11,773 - PackageManager - INFO - Thread WorkerThread0 stopped building the nodejs package 2017-04-25 14:25:11,773 - PackageManager - INFO - Thread WorkerThread0 is going to rest 2017-04-25 14:25:11,773 - PackageManager - INFO - Waiting for all remaining worker threads 2017-04-25 14:25:11,774 - PackageManager - ERROR - Some of the packages failed: 2017-04-25 14:25:11,774 - PackageManager - ERROR - ['nodejs'] 2017-04-25 14:25:11,774 - /home/vmware/GIT_REPOS/photon/stage/LOGS/Main - ERROR - Caught an exception 2017-04-25 14:25:11,774 - /home/vmware/GIT_REPOS/photon/stage/LOGS/Main - ERROR - Failed during building package Traceback (most recent call last): File "/home/vmware/GIT_REPOS/photon/support/package-builder/builder.py", line 130, in main buildPackagesForAllSpecs(listBuildOptionPackages, options.pkgBuildOptionFile, logger, options.buildThreads, pkgInfoJsonFile) File "/home/vmware/GIT_REPOS/photon/support/package-builder/builder.py", line 265, in buildPackagesForAllSpecs pkgManager.buildPackages(listPackages, listBuildOptionPackages, pkgBuildOptionFile, buildThreads) File "/home/vmware/GIT_REPOS/photon/support/package-builder/PackageManager.py", line 146, in buildPackages self.buildGivenPackages(listPackages, buildThreads) File "/home/vmware/GIT_REPOS/photon/support/package-builder/PackageManager.py", line 202, in buildGivenPackages raise Exception("Failed during building package") Exception: Failed during building package Makefile:228: recipe for target 'packages' failed make: *** [packages] Error 1

[ppadmavilasom]

can you update stage/LOGS/build-nodejs.log

[sudhesam]

build-nodejs.log contents:

vmware@fmbuild:~/workspaces/photon/stage/LOGS$ cat build-nodejs.log 2017-04-25 14:24:25,810 - build-nodejs - INFO - -------------------------------------------------------------------------- 2017-04-25 14:24:25,811 - build-nodejs - INFO - Starting Log 2017-04-25 14:24:25,811 - build-nodejs - INFO - -------------------------------------------------------------------------- 2017-04-25 14:24:25,828 - build-nodejs - DEBUG - Created new chroot: /home/vmware/GIT_REPOS/photon/stage/photonroot/build-nodejs 2017-04-25 14:24:25,829 - build-nodejs - INFO - Preparing build environment 2017-04-25 14:24:25,945 - build-nodejs - INFO - Successfully prepared chroot:/home/vmware/GIT_REPOS/photon/stage/photonroot/build-nodejs 2017-04-25 14:24:25,946 - build-nodejs - INFO - Installing Tool Chain RPMS....... 2017-04-25 14:24:26,994 - build-nodejs - DEBUG - Installing rpms: filesystem linux-api-headers glibc glibc-devel zlib zlib-devel file binutils binutils-devel gmp gmp-devel mpfr mpfr-devel mpc libgcc libgcc-devel libstdc++ libstdc++-devel libgomp libgomp-devel gcc pkg-config ncurses bash bzip2 bzip2-libs bzip2-devel sed ncurses-devel procps-ng coreutils m4 grep readline diffutils gawk findutils gettext gzip make patch util-linux util-linux-devel tar xz xz-libs libtool flex flex-devel bison readline-devel lua lua-devel popt popt-devel nspr nspr-devel sqlite sqlite-libs nss nss-libs nss-devel elfutils-libelf elfutils elfutils-libelf-devel elfutils-devel expat expat-libs libffi libpipeline gdbm perl texinfo autoconf automake openssl openssl-devel python2 python2-libs python2-devel libcap rpm rpm-build rpm-devel rpm-libs groff man-pages cpio go 2017-04-25 14:24:32,499 - build-nodejs - INFO - Successfully installed all Tool Chain RPMS in Chroot:/home/vmware/GIT_REPOS/photon/stage/photonroot/build-nodejs 2017-04-25 14:24:32,555 - build-nodejs - INFO - Installing the build time dependent packages...... 2017-04-25 14:24:32,556 - build-nodejs - INFO - Finished installing the build time dependent packages...... 2017-04-25 14:24:32,598 - build-nodejs - INFO - Building rpm's for package:nodejs 2017-04-25 14:25:11,767 - build-nodejs - ERROR - Failed while building package:nodejs 2017-04-25 14:25:11,767 - build-nodejs - DEBUG - Chroot with ID: /home/vmware/GIT_REPOS/photon/stage/photonroot/build-nodejs not deleted for debugging. 2017-04-25 14:25:11,773 - build-nodejs - DEBUG - 2017-04-25 14:25:11,773 - build-nodejs - ERROR - Missing source: node-7.7.4.tar.gz vmware@fmbuild:~/workspaces/photon/stage/LOGS$

[ppadmavilasom]

ah - missing source node-7.7.4.tar.gz Looks for sources under stage/SOURCES

[sudhesam]

Sorry for the previous update.

vmware@fmbuild:~/workspaces/photon/stage/LOGS$ cat build-nodejs.log 2017-04-25 14:24:25,810 - build-nodejs - INFO - -------------------------------------------------------------------------- 2017-04-25 14:24:25,811 - build-nodejs - INFO - Starting Log 2017-04-25 14:24:25,811 - build-nodejs - INFO - -------------------------------------------------------------------------- 2017-04-25 14:24:25,828 - build-nodejs - DEBUG - Created new chroot: /home/vmware/GIT_REPOS/photon/stage/photonroot/build-nodejs 2017-04-25 14:24:25,829 - build-nodejs - INFO - Preparing build environment 2017-04-25 14:24:25,945 - build-nodejs - INFO - Successfully prepared chroot:/home/vmware/GIT_REPOS/photon/stage/photonroot/build-nodejs 2017-04-25 14:24:25,946 - build-nodejs - INFO - Installing Tool Chain RPMS....... 2017-04-25 14:24:26,994 - build-nodejs - DEBUG - Installing rpms: filesystem linux-api-headers glibc glibc-devel zlib zlib-devel file binutils binutils-devel gmp gmp-devel mpfr mpfr-devel mpc libgcc libgcc-devel libstdc++ libstdc++-devel libgomp libgomp-devel gcc pkg-config ncurses bash bzip2 bzip2-libs bzip2-devel sed ncurses-devel procps-ng coreutils m4 grep readline diffutils gawk findutils gettext gzip make patch util-linux util-linux-devel tar xz xz-libs libtool flex flex-devel bison readline-devel lua lua-devel popt popt-devel nspr nspr-devel sqlite sqlite-libs nss nss-libs nss-devel elfutils-libelf elfutils elfutils-libelf-devel elfutils-devel expat expat-libs libffi libpipeline gdbm perl texinfo autoconf automake openssl openssl-devel python2 python2-libs python2-devel libcap rpm rpm-build rpm-devel rpm-libs groff man-pages cpio go 2017-04-25 14:24:32,499 - build-nodejs - INFO - Successfully installed all Tool Chain RPMS in Chroot:/home/vmware/GIT_REPOS/photon/stage/photonroot/build-nodejs 2017-04-25 14:24:32,555 - build-nodejs - INFO - Installing the build time dependent packages...... 2017-04-25 14:24:32,556 - build-nodejs - INFO - Finished installing the build time dependent packages...... 2017-04-25 14:24:32,598 - build-nodejs - INFO - Building rpm's for package:nodejs 2017-04-25 14:25:11,767 - build-nodejs - ERROR - Failed while building package:nodejs 2017-04-25 14:25:11,767 - build-nodejs - DEBUG - Chroot with ID: /home/vmware/GIT_REPOS/photon/stage/photonroot/build-nodejs not deleted for debugging. 2017-04-25 14:25:11,773 - build-nodejs - DEBUG - 2017-04-25 14:25:11,773 - build-nodejs - ERROR - Missing source: node-7.7.4.tar.gz vmware@fmbuild:~/workspaces/photon/stage/LOGS$

[sudhesam]

I see node-7.7.4.tar.gz. Here it is : vmware@fmbuild:~/workspaces/photon/stage/SOURCES$ ls -l node* -rw-r--r-- 1 root root 38076416 Apr 25 14:25 node-7.7.4.tar.gz vmware@fmbuild:~/workspaces/photon/stage/SOURCES$

[ppadmavilasom]

you could check the logs in the chroot - stage/photonroot/build-nodejs/usr/src/photon/LOGS to see if there is anything different. rpmbuild happends in the stage/photonroot/build-nodejs chroot so it will look for sources under usr/src/photon/SOURCES there.

[sudhesam]

SOURCES and LOGS is empty and rpm.log also doesn't give me any clue as to what could be wrong in my environment.

vmware@fmbuild:~/workspaces/photon/stage/photonroot/build-nodejs/usr/src/photon/SOURCES$ ls -l total 0

vmware@fmbuild:~/workspaces/photon/stage/photonroot/build-nodejs/usr/src/photon/SOURCES$ cd ../LOGS/ vmware@fmbuild:~/workspaces/photon/stage/photonroot/build-nodejs/usr/src/photon/LOGS$ ls -l total 0

vmware@fmbuild:~/workspaces/photon/stage/photonroot/build-nodejs/usr/src/photon/LOGS$ more ~/workspaces/photon/stage/photonroot/build-nodejs/usr/lib/rpm/rpm.log /var/log/rpmpkgs { weekly notifempty missingok } vmware@fmbuild:~/workspaces/photon/stage/photonroot/build-nodejs/usr/src/photon/LOGS$

[ppadmavilasom]

you could try "make nodejs" which should rebuild the chroot.

[sudhesam]

Not sure what connection nodejs is trying to make. Also, it is complaining about missing nodejs.log.

('Connection broken: error("(104, \'ECONNRESET\')",)', error("(104, 'ECONNRESET')",)) 2017-04-25 15:53:37,821 - build-nodejs - ERROR - Failed while building package:nodejs 2017-04-25 15:53:37,822 - build-nodejs - DEBUG - Chroot with ID: /home/vmware/GIT_REPOS/photon/stage/photonroot/build-nodejs not deleted for debugging. tail: cannot open '/home/vmware/GIT_REPOS/photon/stage/LOGS/build-nodejs/nodejs.log' for reading: No such file or directory

More debug info:

2017-04-25 15:53:37,825 - build-nodejs - DEBUG - 2017-04-25 15:53:37,825 - build-nodejs - ERROR - Missing source: node-7.7.4.tar.gz 2017-04-25 15:53:37,826 - PackageManager - INFO - Thread WorkerThread0 stopped building the nodejs package 2017-04-25 15:53:37,826 - PackageManager - INFO - Thread WorkerThread0 is going to rest 2017-04-25 15:53:37,826 - PackageManager - INFO - Waiting for all remaining worker threads 2017-04-25 15:53:37,826 - PackageManager - ERROR - Some of the packages failed: 2017-04-25 15:53:37,826 - PackageManager - ERROR - ['nodejs'] 2017-04-25 15:53:37,827 - /home/vmware/GIT_REPOS/photon/stage/LOGS/Main - ERROR - Caught an exception 2017-04-25 15:53:37,827 - /home/vmware/GIT_REPOS/photon/stage/LOGS/Main - ERROR - Failed during building package Traceback (most recent call last): File "/home/vmware/GIT_REPOS/photon/support/package-builder/builder.py", line 128, in main buildAPackage(package, listBuildOptionPackages, options.pkgBuildOptionFile, options.buildThreads) File "/home/vmware/GIT_REPOS/photon/support/package-builder/builder.py", line 255, in buildAPackage pkgManager.buildPackages(listPackages, listBuildOptionPackages, pkgBuildOptionFile, buildThreads) File "/home/vmware/GIT_REPOS/photon/support/package-builder/PackageManager.py", line 146, in buildPackages self.buildGivenPackages(listPackages, buildThreads) File "/home/vmware/GIT_REPOS/photon/support/package-builder/PackageManager.py", line 202, in buildGivenPackages raise Exception("Failed during building package") Exception: Failed during building package Makefile:520: recipe for target 'nodejs' failed make: *** [nodejs] Error 1

[sudhesam]

Missing nodejs.log

vmware@fmbuild:~/GIT_REPOS/photon/stage/LOGS/build-nodejs$ ls -l total 4 -rw-r--r-- 1 root root 15 Apr 25 15:52 adjustGCCSpecScript.log vmware@fmbuild:~/GIT_REPOS/photon/stage/LOGS/build-nodejs$ cat adjustGCCSpecScript.log Using options: vmware@fmbuild:~/GIT_REPOS/photon/stage/LOGS/build-nodejs$

[ppadmavilasom]

i am not sure why there is a connection reset log - is the build directory a local directory or some shared external resource

[sudhesam]

Build directory is local.

[ppadmavilasom]

please check if you have enough disk space left. if that's not the case, please do a make clean and make iso

[sudhesam]

Yes, I do have enough disk space.

vmware@fmbuild:~/GIT_REPOS/photon$ df -kh . Filesystem Size Used Avail Use% Mounted on /dev/sda1 32G 9.6G 21G 32% / vmware@fmbuild:~/GIT_REPOS/photon$

[ppadmavilasom]

Can you update your build machine distro - i will try it with your env following the blog you posted above, you are building the master branch - right?

[sudhesam]

vmware@fmbuild:~/GIT_REPOS/photon$ cat /etc/issue Ubuntu 16.04.1 LTS \n \l

vmware@fmbuild:~/GIT_REPOS/photon$ cat /etc/debian_version stretch/sid vmware@fmbuild:~/GIT_REPOS/photon$ git branch

• master vmware@fmbuild:~/GIT_REPOS/photon$

Did not do anything fancy.

1. Fired up the machine
2. Updated all pre-requisite software as per https://blogs.vmware.com/cloudnative/building-photon-os-from-source/
3. cloned the repository
4. updated CA certificate
5. Turned verify=false in PullSources.py (As per @dthaluru instructions)
6. started the build

[ppadmavilasom]

Thanks for the update. i read the logs you posted again - connection reset must be the source download not completing? could you wget nodejs source from https://dl.bintray.com/vmware/photon_sources/1.0/node-7.7.4.tar.gz to /stage/SOURCES, then try a "make nodejs"

[sudhesam]

I got this read error in the middle of download :

node-7.7.4.tar.gz.1 76%[=====================================================================================================> ] 33.92M 841KB/s in 43s

2017-04-26 12:39:44 (813 KB/s) - Read error at byte 35572173/46666325 (Connection reset by peer). Retrying.

--2017-04-26 12:39:45-- (try: 2) https://akamai.bintray.com/41/41ad7cb94554c5263dc1cad8dbb984702bd7552677df981d50e9822624a6ea63?__gda__=exp=1493236260~hmac=bb1604217457470980da49e99679c77e4d1a97fcfd8b03fd100ec61d54883b7f&response-content-disposition=attachment%3Bfilename%3D%22node-7.7.4.tar.gz%22&response-content-type=application%2Foctet-stream&requestInfo=U2FsdGVkX1_NGXVW31U-22Jnc29FVoJeXcsfwxbIn7_RQf8Km-1Vo0knLIlrYwlnKXvKXJqIlo8gpzEgEJYqYkiYqXKQStFG3wAFT5jCRKCkYH0tmNz9qMRAYbH-D-YKsJGbvYLWKjhaAObSwGCQuQ Connecting to akamai.bintray.com (akamai.bintray.com)|23.72.202.160|:443... connected. HTTP request sent, awaiting response... 206 Partial Content Length: 46666325 (45M), 11094152 (11M) remaining [application/octet-stream] Saving to: ‘node-7.7.4.tar.gz.1’

node-7.7.4.tar.gz.1 100%[++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++================================>] 44.50M 1.27MB/s in 8.4s

2017-04-26 12:39:53 (1.27 MB/s) - ‘node-7.7.4.tar.gz.1’ saved [46666325/46666325]

Will try doing 'make nodejs' and will post the results.

[sudhesam]

Got the same error again.

[ppadmavilasom]

i tried following your setup. didnt have issues. i know that doesnt help. the only diff i can see is i ran this without sudo. in addition to the blog instructions, i had to install fakeroot-ng so i can run from my user. make nodejs is what i tried. Here is my source dir details priyesh@priyesh-ubuntu:~/photon$ ls -al stage/SOURCES/node-7.7.4.tar.gz -rw-rw-r-- 1 priyesh priyesh 46666325 Apr 26 13:23 stage/SOURCES/node-7.7.4.tar.gz priyesh@priyesh-ubuntu:~/photon$ shasum stage/SOURCES/node-7.7.4.tar.gz 31f7b14476b90037516449aec233c68d9b21edd3 stage/SOURCES/node-7.7.4.tar.gz

[sudhesam]

Hmm...For some reason I am seeing 2 files : node-7.7.4.tar.gz node-7.7.4.tar.gz.1

The ".1" file checksum matches with that of what you have above.

vmware@fmbuild:~/GIT_REPOS/photon$ shasum stage/SOURCES/node-7.7.4.tar.gz* d7d2b4db858f21405245496f9d139c3dbc506def stage/SOURCES/node-7.7.4.tar.gz 31f7b14476b90037516449aec233c68d9b21edd3 stage/SOURCES/node-7.7.4.tar.gz.1 vmware@fmbuild:~/GIT_REPOS/photon$

I copied node-7.7.4.tar.gz.1 to node-7.7.4.tar.gz and now "make nodejs" works fine.

I guess the fix is to replace node-7.7.4.tar.gz.1 -> node-7.7.4.tar.gz if the download is interrupted and restarted.

[ppadmavilasom]

makes sense. glad you sorted it out.

[sudhesam]

Apache Maven build is now failing. Below is the error :

2017-04-26 17:25:21,588 - build-apache-maven - INFO - rpmbuild -ba --clean --define \"dist .ph2\" --nocheck --define \"KERNEL_RELEASE 2.ph2\" --define \"dist .ph2\" --define \"photon_build_number dcf60ec\" --define \"KERNEL_VERSION 4.9.13\" --define \"kernelsubrelease .12577282\" --define \"photon_release_version 1.0\" /usr/src/photon/SPECS/apache-maven.spec 2017-04-26 17:26:00,832 - build-apache-maven - ERROR - Building rpm is failed /usr/src/photon/SPECS/apache-maven.spec 2017-04-26 17:26:00,832 - build-apache-maven - ERROR - Failed while building rpm:apache-maven 2017-04-26 17:26:00,832 - build-apache-maven - ERROR - Failed while building package:apache-maven 2017-04-26 17:26:00,832 - build-apache-maven - DEBUG - Chroot with ID: /home/vmware/GIT_REPOS/photon/stage/photonroot/build-apache-maven not deleted for debugging. 2017-04-26 17:26:00,847 - build-apache-maven - DEBUG - [java] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80) [java] at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) [java] at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) [java] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307) [java] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193) [java] at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106) [java] at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863) [java] at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288) [java] at org.apache.maven.cli.MavenCli.main(MavenCli.java:199) [java] at org.apache.maven.cli.MavenCli.main(MavenCli.java:188) [java] Caused by: org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.apache.maven.plugins:maven-remote-resources-plugin:jar:1.5 [java] at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:282) [java] at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:198) [java] at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:287) [java] at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolve(DefaultPluginDependenciesResolver.java:103) [java] ... 20 more [java] Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.apache.maven.plugins:maven-remote-resources-plugin:pom:1.5 from/to central (https://repo.maven.apache.org/maven2): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target [java] at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444) [java] at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246) [java] at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:223) [java] at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:267) [java] ... 23 more [java] Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact org.apache.maven.plugins:maven-remote-resources-plugin:pom:1.5 from/to central (https://repo.maven.apache.org/maven2): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target [java] at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:43) [java] at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:355) [java] at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:67) [java] at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:581) [java] at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:249) [java] at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:520) [java] at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:421) [java] ... 26 more [java] Caused by: org.apache.maven.wagon.TransferFailedException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target [java] at org.apache.maven.wagon.providers.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:1066) [java] at org.apache.maven.wagon.providers.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:960) [java] at org.apache.maven.wagon.StreamWagon.getInputStream(StreamWagon.java:116) [java] at org.apache.maven.wagon.StreamWagon.getIfNewer(StreamWagon.java:88) [java] at org.apache.maven.wagon.StreamWagon.get(StreamWagon.java:61) [java] at org.eclipse.aether.transport.wagon.WagonTransporter$GetTaskRunner.run(WagonTransporter.java:560) [java] at org.eclipse.aether.transport.wagon.WagonTransporter.execute(WagonTransporter.java:427) [java] at org.eclipse.aether.transport.wagon.WagonTransporter.get(WagonTransporter.java:404) [java] at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask(BasicRepositoryConnector.java:447) [java] at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:350) [java] ... 31 more [java] Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target [java] at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [java] at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949) [java] at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302) [java] at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296) [java] at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509) [java] at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216) [java] at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979) [java] at sun.security.ssl.Handshaker.process_record(Handshaker.java:914) [java] at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062) [java] at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) [java] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) [java] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) [java] at org.apache.maven.wagon.providers.http.httpclient.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:275) [java] at org.apache.maven.wagon.providers.http.httpclient.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:254) [java] at org.apache.maven.wagon.providers.http.httpclient.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:123) [java] at org.apache.maven.wagon.providers.http.httpclient.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:318) [java] at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:363) [java] at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.MainClientExec.execute(MainClientExec.java:219) [java] at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195) [java] at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec.execute(RetryExec.java:86) [java] at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RedirectExec.execute(RedirectExec.java:108) [java] at org.apache.maven.wagon.providers.http.httpclient.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) [java] at org.apache.maven.wagon.providers.http.httpclient.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) [java] at org.apache.maven.wagon.providers.http.AbstractHttpClientWagon.execute(AbstractHttpClientWagon.java:832) [java] at org.apache.maven.wagon.providers.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:983) [java] ... 40 more [java] Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target [java] at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387) [java] at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [java] at sun.security.validator.Validator.validate(Validator.java:260) [java] at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324) [java] at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229) [java] at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124) [java] at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1491) [java] ... 60 more [java] Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target [java] at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) [java] at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) [java] at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) [java] at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382) [java] ... 66 more [java] [main] ERROR org.apache.maven.cli.MavenCli - [java] [main] ERROR org.apache.maven.cli.MavenCli - Re-run Maven using the -X switch to enable full debug logging. [java] [main] ERROR org.apache.maven.cli.MavenCli - [java] [main] ERROR org.apache.maven.cli.MavenCli - For more information about the errors and possible solutions, please read the following articles: [java] [main] ERROR org.apache.maven.cli.MavenCli - [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException

BUILD FAILED /usr/src/photon/BUILD/apache-maven-3.3.9/build.xml:254: Java returned: 1

Total time: 38 seconds error: Bad exit status from /var/tmp/rpm-tmp.npl5sP (%build)

RPM build errors: Bad exit status from /var/tmp/rpm-tmp.npl5sP (%build)

2017-04-26 17:26:00,848 - build-apache-maven - ERROR - RPM build failed 2017-04-26 17:26:00,848 - PackageManager - INFO - Thread WorkerThread0 stopped building the apache-maven package 2017-04-26 17:26:00,849 - PackageManager - INFO - Thread WorkerThread0 is going to rest 2017-04-26 17:26:00,849 - PackageManager - INFO - Waiting for all remaining worker threads 2017-04-26 17:26:00,849 - PackageManager - ERROR - Some of the packages failed: 2017-04-26 17:26:00,849 - PackageManager - ERROR - ['apache-maven'] 2017-04-26 17:26:00,849 - /home/vmware/GIT_REPOS/photon/stage/LOGS/Main - ERROR - Caught an exception 2017-04-26 17:26:00,849 - /home/vmware/GIT_REPOS/photon/stage/LOGS/Main - ERROR - Failed during building package Traceback (most recent call last): File "/home/vmware/GIT_REPOS/photon/support/package-builder/builder.py", line 130, in main buildPackagesForAllSpecs(listBuildOptionPackages, options.pkgBuildOptionFile, logger, options.buildThreads, pkgInfoJsonFile) File "/home/vmware/GIT_REPOS/photon/support/package-builder/builder.py", line 265, in buildPackagesForAllSpecs pkgManager.buildPackages(listPackages, listBuildOptionPackages, pkgBuildOptionFile, buildThreads) File "/home/vmware/GIT_REPOS/photon/support/package-builder/PackageManager.py", line 146, in buildPackages self.buildGivenPackages(listPackages, buildThreads) File "/home/vmware/GIT_REPOS/photon/support/package-builder/PackageManager.py", line 202, in buildGivenPackages raise Exception("Failed during building package") Exception: Failed during building package Makefile:228: recipe for target 'packages' failed make: *** [packages] Error 1 vmware@fmbuild:~/GIT_REPOS/photon$

[ppadmavilasom]

[java] Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

[sudhesam]

Apologies for asking something obvious.

What is considered as 'valid certification path' ? This is not clear from the message above. Can you pl. expand ?

[ppadmavilasom]

maven builds are attempting to download resources from maven repos like this - https://repo.maven.apache.org/maven2. the certificate validation for this should happen with the build chroot's ca certs which is supplied by openssl. i dont understand why this failure

[sudhesam]

I am able to 'wget' the file it is trying to download.

vmware@fmbuild:~$ wget https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-remote-resources-plugin/1.5/maven-remote-resources-plugin-1.5.pom --2017-04-27 09:13:14-- https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-remote-resources-plugin/1.5/maven-remote-resources-plugin-1.5.pom Resolving repo.maven.apache.org (repo.maven.apache.org)... 151.101.40.215 Connecting to repo.maven.apache.org (repo.maven.apache.org)|151.101.40.215|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 13353 (13K) [text/xml] Saving to: ‘maven-remote-resources-plugin-1.5.pom’

maven-remote-resources-plugin-1.5.pom 100%[======================================================================================================================================>] 13.04K --.-KB/s in 0.002s

2017-04-27 09:13:14 (6.38 MB/s) - ‘maven-remote-resources-plugin-1.5.pom’ saved [13353/13353]

vmware@fmbuild:~$

Can you pl. verify shasum of apache-maven-3.3.9-src.tar.gz ? Below is the shasum in my environment :

vmware@fmbuild:~/GIT_REPOS/photon/stage/SOURCES$ ls -l mave -rw-r--r-- 1 root root 3845425 Apr 26 17:25 apache-maven-3.3.9-src.tar.gz -rw-r--r-- 1 root root 1169712 Apr 26 16:11 maven-ant-tasks-2.1.3.tar.gz vmware@fmbuild:~/GIT_REPOS/photon/stage/SOURCES$ shasum apache-maven-3.3.9-src.tar.gz 1912316078f1f7041dd8cd2580f210d30f898162 apache-maven-3.3.9-src.tar.gz vmware@fmbuild:~/GIT_REPOS/photon/stage/SOURCES$

[ppadmavilasom]

download from the build chroot is where its running into missing local certs. checksum looks good %define sha1 apache-maven=1912316078f1f7041dd8cd2580f210d30f898162 fyi, checksums are in the spec file. spec file for this - SPECS/apache-maven/apache-maven.spec

you could also try "make minimal-iso" which will skip building most of these packages but give you a bootable media which you can use if thats an option

[sudhesam]

Just curious.

What is the average build time for 'minimal-iso' ? The reason I am asking is I started the build today around 10:00 AM this morning and the build is still going on.

Below is the build system CPU and MEMINFO :

vmware@fmbuild:~/GIT_REPOS/photon$ sudo cat /proc/cpuinfo [sudo] password for vmware: processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 63 model name : Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz stepping : 2 microcode : 0x2d cpu MHz : 2599.998 cache size : 25600 KB physical id : 0 siblings : 1 core id : 0 cpu cores : 1 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 15 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf pni pclmulqdq vmx ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx hypervisor lahf_lm epb tpr_shadow vnmi ept vpid cqm_llc cqm_occup_llc dtherm ida arat pln pts bugs : bogomips : 5199.99 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management:

processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 63 model name : Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz stepping : 2 microcode : 0x2d cpu MHz : 2599.998 cache size : 25600 KB physical id : 2 siblings : 1 core id : 0 cpu cores : 1 apicid : 2 initial apicid : 2 fpu : yes fpu_exception : yes cpuid level : 15 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf pni pclmulqdq vmx ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx hypervisor lahf_lm epb tpr_shadow vnmi ept vpid cqm_llc cqm_occup_llc dtherm ida arat pln pts bugs : bogomips : 5199.99 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management:

vmware@fmbuild:~/GIT_REPOS/photon$ sudo cat /proc/meminfo MemTotal: 8175468 kB MemFree: 4096728 kB MemAvailable: 7424476 kB Buffers: 403724 kB Cached: 2989300 kB SwapCached: 8 kB Active: 1907884 kB Inactive: 1834456 kB Active(anon): 295860 kB Inactive(anon): 64792 kB Active(file): 1612024 kB Inactive(file): 1769664 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 8386556 kB SwapFree: 8386540 kB Dirty: 104 kB Writeback: 0 kB AnonPages: 349424 kB Mapped: 160316 kB Shmem: 11336 kB Slab: 279676 kB SReclaimable: 252044 kB SUnreclaim: 27632 kB KernelStack: 3424 kB PageTables: 8180 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 12474288 kB Committed_AS: 1358036 kB VmallocTotal: 34359738367 kB VmallocUsed: 0 kB VmallocChunk: 0 kB HardwareCorrupted: 0 kB AnonHugePages: 75776 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 69568 kB DirectMap2M: 8318976 kB vmware@fmbuild:~/GIT_REPOS/photon$

The reason I am asking is if we have to make the minimal-iso build as part of continous integration process, I am looking for ways to minimize the build time like using 'cached' copies of most of the system related packages.

Any input appreciated.

Thanks.

[ppadmavilasom]

shouldnt be close to 4 hours or so for the setup you have for minimal you can download rpms from the bintray repos to the stage/RPMS location to get a build to complete faster. repo urls are: https://vmware.bintray.com/photon_release_1.0_x86_64/ and https://vmware.bintray.com/photon_updates_1.0_x86_64/

[sudhesam]

minimal-iso failed again at maven build with the same error.

BUILD FAILED /usr/src/photon/BUILD/apache-maven-3.3.9/build.xml:254: Java returned: 1

Total time: 41 seconds error: Bad exit status from /var/tmp/rpm-tmp.0FTR8e (%build)

RPM build errors: Bad exit status from /var/tmp/rpm-tmp.0FTR8e (%build)

2017-04-27 20:27:22,155 - build-apache-maven - ERROR - RPM build failed 2017-04-27 20:27:22,155 - PackageManager - INFO - Thread WorkerThread0 stopped building the apache-maven package 2017-04-27 20:27:22,156 - PackageManager - INFO - Thread WorkerThread0 is going to rest 2017-04-27 20:27:22,156 - PackageManager - INFO - Waiting for all remaining worker threads 2017-04-27 20:27:22,156 - PackageManager - ERROR - Some of the packages failed: 2017-04-27 20:27:22,156 - PackageManager - ERROR - ['apache-maven'] 2017-04-27 20:27:22,156 - /home/vmware/GIT_REPOS/photon/stage/LOGS/Main - ERROR - Caught an exception 2017-04-27 20:27:22,157 - /home/vmware/GIT_REPOS/photon/stage/LOGS/Main - ERROR - Failed during building package Traceback (most recent call last): File "/home/vmware/GIT_REPOS/photon/support/package-builder/builder.py", line 130, in main buildPackagesForAllSpecs(listBuildOptionPackages, options.pkgBuildOptionFile, logger, options.buildThreads, pkgInfoJsonFile) File "/home/vmware/GIT_REPOS/photon/support/package-builder/builder.py", line 265, in buildPackagesForAllSpecs pkgManager.buildPackages(listPackages, listBuildOptionPackages, pkgBuildOptionFile, buildThreads) File "/home/vmware/GIT_REPOS/photon/support/package-builder/PackageManager.py", line 146, in buildPackages self.buildGivenPackages(listPackages, buildThreads) File "/home/vmware/GIT_REPOS/photon/support/package-builder/PackageManager.py", line 202, in buildGivenPackages raise Exception("Failed during building package") Exception: Failed during building package Makefile:149: recipe for target 'packages-minimal' failed make: *** [packages-minimal] Error 1

[sudhesam]

Any update on how I can proceed further ?

As per the update "download from the build chroot is where its running into missing local certs.", I believe this should be an issue for every build in all environment. Pardon my ignorance. I say this is because the chroot environment is created for each package and then the build is executed from within 'without any external interference' to copy any local certs into chroot.

Please correct me if my understanding is incorrect.

[ppadmavilasom]

i tried building maven and it did succeed. this is using your build setup ubuntu base.

[ppadmavilasom]

could you tar the build-apache-maven folder and attach please?

[sudhesam]

Does the size of build-apache-maven build get to 20+Gig ? That's awfully huge.

vmware@fmbuild:~/GIT_REPOS/photon/stage/photonroot$ du -sh build-apache-maven.tar 20G build-apache-maven.tar vmware@fmbuild:~/GIT_REPOS/photon/stage/photonroot$ ls -l build-apache-maven.tar -rw-r--r-- 1 root root 20942233600 Apr 28 14:57 build-apache-maven.tar vmware@fmbuild:~/GIT_REPOS/photon/stage/photonroot$

[ppadmavilasom]

compress it - tar -czf build-apache-maven.tar.gz build-apache-maven

[sudhesam]

Wow !!! This takes a lot of time to create a tarball. I started this last night and it is still trying to compress the image.

vmware@fmbuild:~/GIT_REPOS/photon/stage/photonroot$ df -kh . Filesystem Size Used Avail Use% Mounted on /dev/sda1 32G 20G 11G 66% / vmware@fmbuild:~/GIT_REPOS/photon/stage/photonroot$ du -sh build-apache-maven.tar.gz 7.9G build-apache-maven.tar.gz vmware@fmbuild:~/GIT_REPOS/photon/stage/photonroot$

It is about 7.9 Gig (until now). Any idea about the size that this image can get to before it is done.

Appreciate your input.

Thanks,

[ppadmavilasom]

are there other build folders in the photonroot? whats the list of photon/stage/photonroot?

[sudhesam]

vmware@fmbuild:~/GIT_REPOS/photon/stage/photonroot$ ls -lart total 8783280 drwxr-xr-x 10 root root 4096 Apr 27 10:54 .. drwxr-xr-x 14 root root 4096 Apr 28 14:22 build-apache-maven drwxr-xr-x 3 root root 4096 Apr 30 21:08 . -rw-r--r-- 1 root root 8994062336 May 1 14:03 build-apache-maven.tar.gz vmware@fmbuild:~/GIT_REPOS/photon/stage/photonroot$

[sudhesam]

command executed :

root 26352 5331 0 Apr30 pts/1 00:00:00 sudo tar -czf build-apache-maven.tar.gz build-apache-maven root 26358 26352 4 Apr30 pts/1 00:40:41 tar -czf build-apache-maven.tar.gz build-apache-maven

[sudhesam]

I had to 'umount' the following volumes before tarring it up.

vmware@fmbuild:~/GIT_REPOS/photon/stage/photonroot$ history | grep umount 524 sudo umount /home/vmware/GIT_REPOS/photon/stage/photonroot/build-apache-maven/dev 525 sudo umount /home/vmware/GIT_REPOS/photon/stage/photonroot/build-apache-maven/proc 526 sudo umount /home/vmware/GIT_REPOS/photon/stage/photonroot/build-apache-maven/sys 527 sudo umount /home/vmware/GIT_REPOS/photon/stage/photonroot/build-apache-maven/run 543 history | grep umount vmware@fmbuild:~/GIT_REPOS/photon/stage/photonroot$

Otherwise, the size of the file was just growing.

When attaching the file, I get an error message : "Yowza, that's a bug file. Try again with a file smaller than 10MB.".

The current size of build-apache-maven.tar.gz is 307 MB.

vmware@fmbuild:~/GIT_REPOS/photon/stage/photonroot$ sudo du -sh build-apache-maven.tar.gz 307M build-apache-maven.tar.gz vmware@fmbuild:~/GIT_REPOS/photon/stage/photonroot$

[sudhesam]

Any input appreciated.

[sudhesam]

From within 'chroot' environment, when I try 'wget', I noticed the following error :

root [ / ]# wget https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-remote-resources-plugin/1.5/maven-remote-resources-plugin-1.5.pom --2017-05-03 16:31:47-- https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-remote-resources-plugin/1.5/maven-remote-resources-plugin-1.5.pom Resolving repo.maven.apache.org... 151.101.40.215 Connecting to repo.maven.apache.org|151.101.40.215|:443... connected. ERROR: cannot verify repo.maven.apache.org's certificate, issued by ‘CN=105.128.134.248’: Self-signed certificate encountered. To connect to repo.maven.apache.org insecurely, use `--no-check-certificate'. root [ / ]#

Now, my question is how does the 'chroot' environment get the CA cert from my 'host' environment ?

I am guessing that this issue is NOT isolated only to my setup. This should be an issue in every build.

Please let me know if my understanding is incorrect.

We need to get the build working soon since we are running into a deadline to release our product. Appreciate your time and input.