search

microsoft / pai

Resource scheduling and cluster management for AI
https://openpai.readthedocs.io
MIT License
2.63k stars 548 forks source link

[IB Drivers] IB drivers' installation will failed on Azure VM. #2470

Closed ydye closed 5 years ago

ydye commented 5 years ago

Important Note

A mitigation for the issue had been provided in the following PR: Mitigate PR: #2514 @ydye. please refer to the following instruction to apply the fix: https://github.com/Microsoft/pai/issues/2470#issuecomment-481125214

This issue is for a more formal resolution for the issue.

Organization Name: OpenPAI

Short summary about the issue/question:

With OpenPAI's offical image, the installation will failed due to ubuntu's kernel upgrade. Some interface has been changed, which cause the failure. After upgrade the OFED version to 4.5-1.0.1.0, the installation will failed. And the log is following.

Checking SW Requirements...
Removing old packages...
Installing new packages
Installing libibumad-43.1.1.MLNX20180612.87b4d9b...
Installing libopensm-5.3.0.MLNX20181108.33944a2...
Installing libibmad-5.0.0.MLNX20181022.0361c15...
Installing infiniband-diags-5.0.0.MLNX20181101.2a96f26...
Installing ofed-scripts-4.5...
Installing mlnx-ofed-kernel-utils-4.5...
Installing mlnx-ofed-kernel-modules-4.5...

Error: mlnx-ofed-kernel-modules installation failed!
Collecting debug info...
See:
        /tmp/MLNX_OFED_LINUX.18160.logs/mlnx-ofed-kernel-modules.debinstall.log
Removing newly installed packages...

Failed to install MLNX_OFED_LINUX-4.5-1.0.1.0-ubuntu16.04-ext for 4.15.0-1040-azure

File /tmp/MLNX_OFED_LINUX.18160.logs/mlnx-ofed-kernel-modules.debinstall.log


/usr/bin/dpkg -i --force-confnew --force-confmiss /tmp/MLNX_OFED_LINUX-4.5-1.0.1.0-4.15.0-1040-azure/MLNX_OFED_LINUX-4.5-1.0.1.0-ubuntu16.04-ext/DEBS/mlnx-ofed-kernel-modules_4.5-OFED.4.5.1.0.1.1.gb4fdfac.kver.4.15.0-1040-azure_amd64.deb
Selecting previously unselected package mlnx-ofed-kernel-modules.
(Reading database ... 32015 files and directories currently installed.)
Preparing to unpack .../mlnx-ofed-kernel-modules_4.5-OFED.4.5.1.0.1.1.gb4fdfac.kver.4.15.0-1040-azure_amd64.deb ...
Unpacking mlnx-ofed-kernel-modules (4.5-OFED.4.5.1.0.1.1.gb4fdfac.kver.4.15.0-1040-azure) ...
Setting up mlnx-ofed-kernel-modules (4.5-OFED.4.5.1.0.1.1.gb4fdfac.kver.4.15.0-1040-azure) ...

---------------- START OF DEBUG INFO -------------------
Install command: /tmp/MLNX_OFED_LINUX-4.5-1.0.1.0-4.15.0-1040-azure/MLNX_OFED_LINUX-4.5-1.0.1.0-ubuntu16.04-ext/mlnxofedinstall --force --without-dkms --force --kernel-only --without-dkms --without-fw-update --with-infiniband-diags

Vars dump:
- ofedlogs: /tmp/MLNX_OFED_LINUX.18160.logs
- MLNX_OFED_LINUX_VERSION: 4.5-1.0.1.0
- MLNX_OFED_ARCH: x86_64
- MLNX_OFED_DISTRO: skip-distro-check
- distro: ubuntu16.04
- arch: x86_64
- kernel: 4.15.0-1040-azure
- config: /tmp/ofed.conf
- update_firmware: 0

Setup info:

- uname -r: 4.15.0-1040-azure

- uname -m: x86_64

- lsb_release -a: No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.5 LTS
Release:        16.04
Codename:       xenial

- cat /etc/issue: Ubuntu 16.04.5 LTS \n \l

- cat /proc/version: Linux version 4.15.0-1040-azure (buildd@lgw01-amd64-030) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)) #44-Ubuntu SMP Thu Feb 21 14:24:01 UTC 2019

- gcc --version: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

- lspci -n | grep 15b3: 0002:00:02.0 0280: 15b3:1004

- dpkg --list: Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                               Version                                              Architecture Description
+++-==================================-====================================================-============-===============================================================================
ii  adduser                            3.113+nmu3ubuntu4                                    all          add and remove users and groups
ii  apt                                1.2.31                                               amd64        commandline package manager
ii  apt-transport-https                1.2.29ubuntu0.1                                      amd64        https download transport for APT
ii  apt-utils                          1.2.31                                               amd64        package management related utility programs
ii  autoconf                           2.69-9                                               all          automatic configure script builder
ii  automake                           1:1.15-4ubuntu1                                      all          Tool for generating GNU Standards-compliant Makefiles
ii  autotools-dev                      20150820.1                                           all          Update infrastructure for config.{guess,sub} files
ii  base-files                         9.4ubuntu4.7                                         amd64        Debian base system miscellaneous files
ii  base-passwd                        3.5.39                                               amd64        Debian base system master password and group files
ii  bash                               4.3-14ubuntu1.2                                      amd64        GNU Bourne Again SHell
ii  bc                                 1.06.95-9build1                                      amd64        GNU bc arbitrary precision calculator language
ii  bind9-host                         1:9.10.3.dfsg.P4-8ubuntu1.12                         amd64        Version of 'host' bundled with BIND 9.X
ii  binutils                           2.26.1-1ubuntu1~16.04.8                              amd64        GNU assembler, linker and binary utilities
ii  binutils-dev                       2.26.1-1ubuntu1~16.04.8                              amd64        GNU binary utilities (BFD development files)
ii  bison                              2:3.0.4.dfsg-1                                       amd64        YACC-compatible parser generator
ii  bsdmainutils                       9.0.6ubuntu3                                         amd64        collection of more utilities from FreeBSD
ii  bsdutils                           1:2.27.1-6ubuntu3.6                                  amd64        basic utilities from 4.4BSD-Lite
ii  build-essential                    12.1ubuntu2                                          amd64        Informational list of build-essential packages
ii  bzip2                              1.0.6-8                                              amd64        high-quality block-sorting file compressor - utilities
ii  ca-certificates                    20170717~16.04.2                                     all          Common CA certificates
ii  chrpath                            0.16-1                                               amd64        Tool to edit the rpath in ELF binaries
ii  coreutils                          8.25-2ubuntu3~16.04                                  amd64        GNU core utilities
ii  cpp                                4:5.3.1-1ubuntu1                                     amd64        GNU C preprocessor (cpp)
ii  cpp-5                              5.4.0-6ubuntu1~16.04.11                              amd64        GNU C preprocessor
ii  cron                               3.0pl1-128ubuntu2                                    amd64        process scheduling daemon
ii  cuda-command-line-tools-9-1        9.1.85-1                                             amd64        CUDA command-line tools
ii  cuda-compiler-9-1                  9.1.85-1                                             amd64        CUDA compiler
ii  cuda-cublas-9-1                    9.1.85.3-1                                           amd64        CUBLAS native runtime libraries
ii  cuda-cublas-dev-9-1                9.1.85.3-1                                           amd64        CUBLAS native dev links, headers
ii  cuda-cudart-9-1                    9.1.85-1                                             amd64        CUDA Runtime native Libraries
ii  cuda-cudart-dev-9-1                9.1.85-1                                             amd64        CUDA Runtime native dev links, headers
ii  cuda-cufft-9-1                     9.1.85-1                                             amd64        CUFFT native runtime libraries
ii  cuda-cufft-dev-9-1                 9.1.85-1                                             amd64        CUFFT native dev links, headers
ii  cuda-cuobjdump-9-1                 9.1.85-1                                             amd64        CUDA cuobjdump
ii  cuda-cupti-9-1                     9.1.85-1                                             amd64        CUDA profiling tools interface.
ii  cuda-curand-9-1                    9.1.85-1                                             amd64        CURAND native runtime libraries
ii  cuda-curand-dev-9-1                9.1.85-1                                             amd64        CURAND native dev links, headers
ii  cuda-cusolver-9-1                  9.1.85-1                                             amd64        CUDA solver native runtime libraries
ii  cuda-cusolver-dev-9-1              9.1.85-1                                             amd64        CUDA solver native dev links, headers
ii  cuda-cusparse-9-1                  9.1.85-1                                             amd64        CUSPARSE native runtime libraries
ii  cuda-cusparse-dev-9-1              9.1.85-1                                             amd64        CUSPARSE native dev links, headers
ii  cuda-driver-dev-9-1                9.1.85-1                                             amd64        CUDA Driver native dev stub library
ii  cuda-gdb-9-1                       9.1.85-1                                             amd64        CUDA-GDB
ii  cuda-gpu-library-advisor-9-1       9.1.85-1                                             amd64        CUDA GPU Library Advisor.
ii  cuda-libraries-9-1                 9.1.85-1                                             amd64        CUDA Libraries 9.1 meta-package
ii  cuda-libraries-dev-9-1             9.1.85-1                                             amd64        CUDA Libraries 9.1 development meta-package
ii  cuda-license-9-1                   9.1.85-1                                             amd64        CUDA licenses
ii  cuda-memcheck-9-1                  9.1.85-1                                             amd64        CUDA-MEMCHECK
ii  cuda-minimal-build-9-1             9.1.85-1                                             amd64        Minimal CUDA 9.1 toolkit build packages.
ii  cuda-misc-headers-9-1              9.1.85-1                                             amd64        CUDA miscellaneous headers
ii  cuda-npp-9-1                       9.1.85-1                                             amd64        NPP native runtime libraries
ii  cuda-npp-dev-9-1                   9.1.85-1                                             amd64        NPP native dev links, headers
ii  cuda-nvcc-9-1                      9.1.85.2-1                                           amd64        CUDA nvcc
ii  cuda-nvdisasm-9-1                  9.1.85-1                                             amd64        CUDA disassembler
ii  cuda-nvgraph-9-1                   9.1.85-1                                             amd64        NVGRAPH native runtime libraries
ii  cuda-nvgraph-dev-9-1               9.1.85-1                                             amd64        NVGRAPH native dev links, headers
ii  cuda-nvml-dev-9-1                  9.1.85-1                                             amd64        NVML native dev links, headers
ii  cuda-nvprof-9-1                    9.1.85-1                                             amd64        CUDA Profiler tools
ii  cuda-nvprune-9-1                   9.1.85-1                                             amd64        CUDA nvprune
ii  cuda-nvrtc-9-1                     9.1.85-1                                             amd64        NVRTC native runtime libraries
ii  cuda-nvrtc-dev-9-1                 9.1.85-1                                             amd64        NVRTC native dev links, headers
ii  cuda-nvtx-9-1                      9.1.85-1                                             amd64        NVIDIA Tools Extension
ii  curl                               7.47.0-1ubuntu2.12                                   amd64        command line tool for transferring data with URL syntax
ii  dash                               0.5.8-2.1ubuntu2                                     amd64        POSIX-compliant shell
ii  dbus                               1.10.6-1ubuntu3.3                                    amd64        simple interprocess messaging system (daemon and utilities)
ii  debconf                            1.5.58ubuntu1                                        all          Debian configuration management system
ii  debhelper                          9.20160115ubuntu3                                    all          helper programs for debian/rules
ii  debianutils                        4.7                                                  amd64        Miscellaneous utilities specific to Debian
ii  dh-python                          2.20151103ubuntu1.1                                  all          Debian helper tools for packaging Python libraries and applications
ii  dh-strip-nondeterminism            0.015-1                                              all          debhelper add-on to strip non-determinism from files
ii  diffstat                           1.61-1                                               amd64        produces graph of changes introduced by a diff file
ii  diffutils                          1:3.3-3                                              amd64        File comparison utilities
ii  distro-info-data                   0.28ubuntu0.10                                       all          information about the distributions' releases (data files)
ii  dkms                               2.2.0.3-2ubuntu11.6                                  all          Dynamic Kernel Module Support Framework
ii  dpatch                             2.0.38                                               all          patch maintenance system for Debian source packages
ii  dpkg                               1.18.4ubuntu1.5                                      amd64        Debian package management system
ii  dpkg-dev                           1.18.4ubuntu1.5                                      all          Debian package development tools
ii  e2fslibs:amd64                     1.42.13-1ubuntu1                                     amd64        ext2/ext3/ext4 file system libraries
ii  e2fsprogs                          1.42.13-1ubuntu1                                     amd64        ext2/ext3/ext4 file system utilities
ii  ethtool                            1:4.5-1                                              amd64        display or change Ethernet device settings
ii  fakeroot                           1.20.2-1ubuntu1                                      amd64        tool for simulating superuser privileges
ii  file                               1:5.25-2ubuntu1.2                                    amd64        Determines file type using "magic" numbers
ii  findutils                          4.6.0+git+20160126-2                                 amd64        utilities for finding files--find, xargs
ii  flex                               2.6.0-11                                             amd64        fast lexical analyzer generator
ii  fontconfig                         2.11.94-0ubuntu1.1                                   amd64        generic font configuration library - support binaries
ii  fontconfig-config                  2.11.94-0ubuntu1.1                                   all          generic font configuration library - configuration
ii  fonts-dejavu-core                  2.35-1                                               all          Vera font family derivate with additional characters
ii  fonts-liberation                   1.07.4-1                                             all          Fonts with the same metrics as Times, Arial and Courier
ii  g++                                4:5.3.1-1ubuntu1                                     amd64        GNU C++ compiler
ii  g++-5                              5.4.0-6ubuntu1~16.04.11                              amd64        GNU C++ compiler
ii  gawk                               1:4.1.3+dfsg-0.1                                     amd64        GNU awk, a pattern scanning and processing language
ii  gcc                                4:5.3.1-1ubuntu1                                     amd64        GNU C compiler
ii  gcc-5                              5.4.0-6ubuntu1~16.04.11                              amd64        GNU C compiler
ii  gcc-5-base:amd64                   5.4.0-6ubuntu1~16.04.11                              amd64        GCC, the GNU Compiler Collection (base package)
ii  gcc-6-base:amd64                   6.0.1-0ubuntu1                                       amd64        GCC, the GNU Compiler Collection (base package)
ii  geoip-database                     20160408-1                                           all          IP lookup command line tools that use the GeoIP library (country database)
ii  gettext                            0.19.7-2ubuntu3.1                                    amd64        GNU Internationalization utilities
ii  gettext-base                       0.19.7-2ubuntu3.1                                    amd64        GNU Internationalization utilities for the base system
ii  gfortran                           4:5.3.1-1ubuntu1                                     amd64        GNU Fortran 95 compiler
ii  gfortran-5                         5.4.0-6ubuntu1~16.04.11                              amd64        GNU Fortran compiler
ii  gir1.2-atk-1.0                     2.18.0-1                                             amd64        ATK accessibility toolkit (GObject introspection)
ii  gir1.2-freedesktop:amd64           1.46.0-3ubuntu1                                      amd64        Introspection data for some FreeDesktop components
ii  gir1.2-gdkpixbuf-2.0:amd64         2.32.2-1ubuntu1.6                                    amd64        GDK Pixbuf library - GObject-Introspection
ii  gir1.2-glib-2.0:amd64              1.46.0-3ubuntu1                                      amd64        Introspection data for GLib, GObject, Gio and GModule
ii  gir1.2-gtk-2.0                     2.24.30-1ubuntu1.16.04.2                             amd64        GTK+ graphical user interface library -- gir bindings
ii  gir1.2-pango-1.0:amd64             1.38.1-1                                             amd64        Layout and rendering of internationalized text - gir bindings
ii  git                                1:2.7.4-0ubuntu1.6                                   amd64        fast, scalable, distributed revision control system
ii  git-man                            1:2.7.4-0ubuntu1.6                                   all          fast, scalable, distributed revision control system (manual pages)
ii  gnupg                              1.4.20-1ubuntu3.3                                    amd64        GNU privacy guard - a free PGP replacement
ii  gnupg-curl                         1.4.20-1ubuntu3.3                                    amd64        GNU privacy guard - a free PGP replacement (cURL)
ii  gpgv                               1.4.20-1ubuntu3.3                                    amd64        GNU privacy guard - signature verification tool
ii  graphviz                           2.38.0-12ubuntu2.1                                   amd64        rich set of graph drawing tools
ii  grep                               2.25-1~16.04.1                                       amd64        GNU grep, egrep and fgrep
ii  groff-base                         1.22.3-7                                             amd64        GNU troff text-formatting system (base system components)
ii  gzip                               1.6-4ubuntu1                                         amd64        GNU compression utilities
ii  hicolor-icon-theme                 0.15-0ubuntu1.1                                      all          default fallback theme for FreeDesktop.org icon themes
ii  hostname                           3.16ubuntu2                                          amd64        utility to set/show the host name or domain name
ii  infiniband-diags                   5.0.0.MLNX20181101.2a96f26-0.1.45101                 amd64        OpenFabrics Alliance InfiniBand Diagnostic Tools
ii  init                               1.29ubuntu4                                          amd64        System-V-like init utilities - metapackage
ii  init-system-helpers                1.29ubuntu4                                          all          helper tools for all init systems
ii  initscripts                        2.88dsf-59.3ubuntu2                                  amd64        scripts for initializing and shutting down the system
ii  insserv                            1.14.0-5ubuntu3                                      amd64        boot sequence organizer using LSB init.d script dependency information
ii  intltool-debian                    0.35.0+20060710.4                                    all          Help i18n of RFC822 compliant config files
ii  iproute2                           4.3.0-1ubuntu3.16.04.4                               amd64        networking and traffic control tools
ii  iso-codes                          3.65-1                                               all          ISO language, territory, currency, script codes and their translations
ii  jq                                 1.5+dfsg-1ubuntu0.1                                  amd64        lightweight and flexible command-line JSON processor
ii  kmod                               22-1ubuntu5.2                                        amd64        tools for managing Linux kernel modules
ii  less                               481-2.1ubuntu0.2                                     amd64        pager program similar to more
ii  libacl1:amd64                      2.2.52-3                                             amd64        Access control list shared library
ii  libapparmor1:amd64                 2.10.95-0ubuntu2.10                                  amd64        changehat AppArmor library
ii  libapt-inst2.0:amd64               1.2.31                                               amd64        deb package format runtime library
ii  libapt-pkg5.0:amd64                1.2.31                                               amd64        package management runtime library
ii  libarchive-zip-perl                1.56-2ubuntu0.1                                      all          Perl module for manipulation of ZIP archives
ii  libasan2:amd64                     5.4.0-6ubuntu1~16.04.11                              amd64        AddressSanitizer -- a fast memory error detector
ii  libasn1-8-heimdal:amd64            1.7~git20150920+dfsg-4ubuntu1.16.04.1                amd64        Heimdal Kerberos - ASN.1 library
ii  libasprintf-dev:amd64              0.19.7-2ubuntu3.1                                    amd64        GNU Internationalization library development files
ii  libasprintf0v5:amd64               0.19.7-2ubuntu3.1                                    amd64        GNU library to use fprintf and friends in C++
ii  libatk1.0-0:amd64                  2.18.0-1                                             amd64        ATK accessibility toolkit
ii  libatk1.0-data                     2.18.0-1                                             all          Common files for the ATK accessibility toolkit
ii  libatk1.0-dev                      2.18.0-1                                             amd64        Development files for the ATK accessibility toolkit
ii  libatm1:amd64                      1:2.5.1-1.5                                          amd64        shared library for ATM (Asynchronous Transfer Mode)
ii  libatomic1:amd64                   5.4.0-6ubuntu1~16.04.11                              amd64        support library providing __atomic built-in functions
ii  libattr1:amd64                     1:2.4.47-2                                           amd64        Extended attribute shared library
ii  libaudit-common                    1:2.4.5-1ubuntu2.1                                   all          Dynamic library for security auditing - common files
ii  libaudit-dev:amd64                 1:2.4.5-1ubuntu2.1                                   amd64        Header files and static library for security auditing
ii  libaudit1:amd64                    1:2.4.5-1ubuntu2.1                                   amd64        Dynamic library for security auditing
ii  libavahi-client3:amd64             0.6.32~rc+dfsg-1ubuntu2.3                            amd64        Avahi client library
ii  libavahi-common-data:amd64         0.6.32~rc+dfsg-1ubuntu2.3                            amd64        Avahi common data files
ii  libavahi-common3:amd64             0.6.32~rc+dfsg-1ubuntu2.3                            amd64        Avahi common library
ii  libbind9-140:amd64                 1:9.10.3.dfsg.P4-8ubuntu1.12                         amd64        BIND9 Shared Library used by BIND
ii  libbison-dev:amd64                 2:3.0.4.dfsg-1                                       amd64        YACC-compatible parser generator - development library
ii  libblkid1:amd64                    2.27.1-6ubuntu3.6                                    amd64        block device ID library
ii  libbsd0:amd64                      0.8.2-1                                              amd64        utility functions from BSD systems - shared library
ii  libbz2-1.0:amd64                   1.0.6-8                                              amd64        high-quality block-sorting file compressor library - runtime
ii  libc-bin                           2.23-0ubuntu10                                       amd64        GNU C Library: Binaries
ii  libc-dev-bin                       2.23-0ubuntu10                                       amd64        GNU C Library: Development binaries
ii  libc6:amd64                        2.23-0ubuntu10                                       amd64        GNU C Library: Shared libraries
ii  libc6-dev:amd64                    2.23-0ubuntu10                                       amd64        GNU C Library: Development Libraries and Header Files
ii  libcairo-gobject2:amd64            1.14.6-1                                             amd64        Cairo 2D vector graphics library (GObject library)
ii  libcairo-script-interpreter2:amd64 1.14.6-1                                             amd64        Cairo 2D vector graphics library (script interpreter)
ii  libcairo2:amd64                    1.14.6-1                                             amd64        Cairo 2D vector graphics library
ii  libcairo2-dev                      1.14.6-1                                             amd64        Development files for the Cairo 2D graphics library
ii  libcap-ng0:amd64                   0.7.7-1                                              amd64        An alternate POSIX capabilities library
ii  libcap2:amd64                      1:2.24-12                                            amd64        POSIX 1003.1e capabilities (library)
ii  libcap2-bin                        1:2.24-12                                            amd64        POSIX 1003.1e capabilities (utilities)
ii  libcc1-0:amd64                     5.4.0-6ubuntu1~16.04.11                              amd64        GCC cc1 plugin for GDB
ii  libcdt5                            2.38.0-12ubuntu2.1                                   amd64        rich set of graph drawing tools - cdt library
ii  libcgraph6                         2.38.0-12ubuntu2.1                                   amd64        rich set of graph drawing tools - cgraph library
ii  libcilkrts5:amd64                  5.4.0-6ubuntu1~16.04.11                              amd64        Intel Cilk Plus language extensions (runtime)
ii  libcomerr2:amd64                   1.42.13-1ubuntu1                                     amd64        common error description library
ii  libcr-dev                          0.8.5-2.2                                            amd64        Development files for BLCR
ii  libcr0                             0.8.5-2.2                                            amd64        Libraries to Checkpoint and Restart Linux processes
ii  libcroco3:amd64                    0.6.11-1                                             amd64        Cascading Style Sheet (CSS) parsing and manipulation toolkit
ii  libcryptsetup4:amd64               2:1.6.6-5ubuntu2.1                                   amd64        disk encryption support - shared library
hi  libcudnn7                          7.1.2.21-1+cuda9.1                                   amd64        cuDNN runtime libraries
ii  libcudnn7-dev                      7.1.2.21-1+cuda9.1                                   amd64        cuDNN development libraries and headers
ii  libcups2:amd64                     2.1.3-4ubuntu0.7                                     amd64        Common UNIX Printing System(tm) - Core library
ii  libcurl3-gnutls:amd64              7.47.0-1ubuntu2.12                                   amd64        easy-to-use client-side URL transfer library (GnuTLS flavour)
ii  libdatrie1:amd64                   0.2.10-2                                             amd64        Double-array trie library
ii  libdb5.3:amd64                     5.3.28-11ubuntu0.1                                   amd64        Berkeley v5.3 Database Libraries [runtime]
ii  libdbus-1-3:amd64                  1.10.6-1ubuntu3.3                                    amd64        simple interprocess messaging system (library)
ii  libdbus-glib-1-2:amd64             0.106-1                                              amd64        simple interprocess messaging system (GLib-based shared library)
ii  libdebconfclient0:amd64            0.198ubuntu1                                         amd64        Debian Configuration Management System (C-implementation library)
ii  libdevmapper1.02.1:amd64           2:1.02.110-1ubuntu10                                 amd64        Linux Kernel Device Mapper userspace library
ii  libdns162:amd64                    1:9.10.3.dfsg.P4-8ubuntu1.12                         amd64        DNS Shared Library used by BIND
ii  libdpkg-perl                       1.18.4ubuntu1.5                                      all          Dpkg perl modules
ii  libdrm-amdgpu1:amd64               2.4.91-2~16.04.1                                     amd64        Userspace interface to amdgpu-specific kernel DRM services -- runtime
ii  libdrm-common                      2.4.91-2~16.04.1                                     all          Userspace interface to kernel DRM services -- common files
ii  libdrm-intel1:amd64                2.4.91-2~16.04.1                                     amd64        Userspace interface to intel-specific kernel DRM services -- runtime
ii  libdrm-nouveau2:amd64              2.4.91-2~16.04.1                                     amd64        Userspace interface to nouveau-specific kernel DRM services -- runtime
ii  libdrm-radeon1:amd64               2.4.91-2~16.04.1                                     amd64        Userspace interface to radeon-specific kernel DRM services -- runtime
ii  libdrm2:amd64                      2.4.91-2~16.04.1                                     amd64        Userspace interface to kernel DRM services -- runtime
ii  libdw-dev:amd64                    0.165-3ubuntu1.1                                     amd64        libdw1 development libraries and header files
ii  libdw1:amd64                       0.165-3ubuntu1.1                                     amd64        library that provides access to the DWARF debug information
ii  libedit2:amd64                     3.1-20150325-1ubuntu2                                amd64        BSD editline and history libraries
ii  libelf-dev:amd64                   0.165-3ubuntu1.1                                     amd64        libelf1 development libraries and header files
ii  libelf1:amd64                      0.165-3ubuntu1.1                                     amd64        library to read and write ELF files
ii  liberror-perl                      0.17-1.2                                             all          Perl module for error/exception handling in an OO-ish way
ii  libexpat1:amd64                    2.1.0-7ubuntu0.16.04.3                               amd64        XML parsing C library - runtime library
ii  libexpat1-dev:amd64                2.1.0-7ubuntu0.16.04.3                               amd64        XML parsing C library - development kit
ii  libfakeroot:amd64                  1.20.2-1ubuntu1                                      amd64        tool for simulating superuser privileges - shared libraries
ii  libfdisk1:amd64                    2.27.1-6ubuntu3.6                                    amd64        fdisk partitioning library
ii  libffi6:amd64                      3.2.1-4                                              amd64        Foreign Function Interface library runtime
ii  libfile-stripnondeterminism-perl   0.015-1                                              all          Perl module for stripping non-determinism from files
ii  libfl-dev:amd64                    2.6.0-11                                             amd64        static library for flex (a fast lexical analyzer generator)
ii  libfontconfig1:amd64               2.11.94-0ubuntu1.1                                   amd64        generic font configuration library - runtime
ii  libfontconfig1-dev:amd64           2.11.94-0ubuntu1.1                                   amd64        generic font configuration library - development
ii  libfontenc1:amd64                  1:1.1.3-1                                            amd64        X11 font encoding library
ii  libfreetype6:amd64                 2.6.1-0.1ubuntu2.3                                   amd64        FreeType 2 font engine, shared library files
ii  libfreetype6-dev:amd64             2.6.1-0.1ubuntu2.3                                   amd64        FreeType 2 font engine, development files
ii  libgcc-5-dev:amd64                 5.4.0-6ubuntu1~16.04.11                              amd64        GCC support library (development files)
ii  libgcc1:amd64                      1:6.0.1-0ubuntu1                                     amd64        GCC support library
ii  libgcrypt20:amd64                  1.6.5-2ubuntu0.5                                     amd64        LGPL Crypto library - runtime library
ii  libgd3:amd64                       2.1.1-4ubuntu0.16.04.11                              amd64        GD Graphics Library
ii  libgdbm3:amd64                     1.8.3-13.1                                           amd64        GNU dbm database routines (runtime version)
ii  libgdk-pixbuf2.0-0:amd64           2.32.2-1ubuntu1.6                                    amd64        GDK Pixbuf library
ii  libgdk-pixbuf2.0-common            2.32.2-1ubuntu1.6                                    all          GDK Pixbuf library - data files
ii  libgdk-pixbuf2.0-dev               2.32.2-1ubuntu1.6                                    amd64        GDK Pixbuf library (development files)
ii  libgeoip1:amd64                    1.6.9-1                                              amd64        non-DNS IP-to-country resolver library
ii  libgettextpo-dev:amd64             0.19.7-2ubuntu3.1                                    amd64        GNU Internationalization library development files
ii  libgettextpo0:amd64                0.19.7-2ubuntu3.1                                    amd64        GNU Internationalization library
ii  libgfortran-5-dev:amd64            5.4.0-6ubuntu1~16.04.11                              amd64        Runtime library for GNU Fortran applications (development files)
ii  libgfortran3:amd64                 5.4.0-6ubuntu1~16.04.11                              amd64        Runtime library for GNU Fortran applications
ii  libgirepository-1.0-1:amd64        1.46.0-3ubuntu1                                      amd64        Library for handling GObject introspection data (runtime library)
ii  libgl1-mesa-dri:amd64              18.0.5-0ubuntu0~16.04.1                              amd64        free implementation of the OpenGL API -- DRI modules
ii  libgl1-mesa-glx:amd64              18.0.5-0ubuntu0~16.04.1                              amd64        free implementation of the OpenGL API -- GLX runtime
ii  libglapi-mesa:amd64                18.0.5-0ubuntu0~16.04.1                              amd64        free implementation of the GL API -- shared library
ii  libglib2.0-0:amd64                 2.48.2-0ubuntu4.1                                    amd64        GLib library of C routines
ii  libglib2.0-bin                     2.48.2-0ubuntu4.1                                    amd64        Programs for the GLib library
ii  libglib2.0-data                    2.48.2-0ubuntu4.1                                    all          Common files for GLib library
ii  libglib2.0-dev                     2.48.2-0ubuntu4.1                                    amd64        Development files for the GLib library
ii  libgmp10:amd64                     2:6.1.0+dfsg-2                                       amd64        Multiprecision arithmetic library
ii  libgnutls30:amd64                  3.4.10-4ubuntu1.4                                    amd64        GNU TLS library - main runtime library
ii  libgomp1:amd64                     5.4.0-6ubuntu1~16.04.11                              amd64        GCC OpenMP (GOMP) support library
ii  libgpg-error0:amd64                1.21-2ubuntu1                                        amd64        library for common error values and messages in GnuPG components
ii  libgpm2:amd64                      1.20.4-6.1                                           amd64        General Purpose Mouse - shared library
ii  libgraphite2-3:amd64               1.3.10-0ubuntu0.16.04.1                              amd64        Font rendering engine for Complex Scripts -- library
ii  libgssapi-krb5-2:amd64             1.13.2+dfsg-5ubuntu2.1                               amd64        MIT Kerberos runtime libraries - krb5 GSS-API Mechanism
ii  libgssapi3-heimdal:amd64           1.7~git20150920+dfsg-4ubuntu1.16.04.1                amd64        Heimdal Kerberos - GSSAPI support library
ii  libgtk2.0-0:amd64                  2.24.30-1ubuntu1.16.04.2                             amd64        GTK+ graphical user interface library
ii  libgtk2.0-bin                      2.24.30-1ubuntu1.16.04.2                             amd64        programs for the GTK+ graphical user interface library
ii  libgtk2.0-common                   2.24.30-1ubuntu1.16.04.2                             all          common files for the GTK+ graphical user interface library
ii  libgtk2.0-dev                      2.24.30-1ubuntu1.16.04.2                             amd64        development files for the GTK+ library
ii  libgvc6                            2.38.0-12ubuntu2.1                                   amd64        rich set of graph drawing tools - gvc library
ii  libgvpr2                           2.38.0-12ubuntu2.1                                   amd64        rich set of graph drawing tools - gvpr library
ii  libharfbuzz-dev                    1.0.1-1ubuntu0.1                                     amd64        Development files for OpenType text shaping engine
ii  libharfbuzz-gobject0:amd64         1.0.1-1ubuntu0.1                                     amd64        OpenType text shaping engine ICU backend (GObject library)
ii  libharfbuzz-icu0:amd64             1.0.1-1ubuntu0.1                                     amd64        OpenType text shaping engine ICU backend
ii  libharfbuzz0b:amd64                1.0.1-1ubuntu0.1                                     amd64        OpenType text shaping engine (shared library)
ii  libhcrypto4-heimdal:amd64          1.7~git20150920+dfsg-4ubuntu1.16.04.1                amd64        Heimdal Kerberos - crypto library
ii  libheimbase1-heimdal:amd64         1.7~git20150920+dfsg-4ubuntu1.16.04.1                amd64        Heimdal Kerberos - Base library
ii  libheimntlm0-heimdal:amd64         1.7~git20150920+dfsg-4ubuntu1.16.04.1                amd64        Heimdal Kerberos - NTLM support library
ii  libhogweed4:amd64                  3.2-1ubuntu0.16.04.1                                 amd64        low level cryptographic library (public-key cryptos)
ii  libhx509-5-heimdal:amd64           1.7~git20150920+dfsg-4ubuntu1.16.04.1                amd64        Heimdal Kerberos - X509 support library
ii  libiberty-dev:amd64                20160215-1ubuntu0.2                                  amd64        library of utility functions used by GNU programs
ii  libibmad                           5.0.0.MLNX20181022.0361c15-0.1.45101                 amd64        OpenFabrics Alliance InfiniBand MAD library
ii  libibumad                          43.1.1.MLNX20180612.87b4d9b-0.1.45101                amd64        OpenFabrics Alliance InfiniBand umad (user MAD) library
ii  libice-dev:amd64                   2:1.0.9-1                                            amd64        X11 Inter-Client Exchange library (development headers)
ii  libice6:amd64                      2:1.0.9-1                                            amd64        X11 Inter-Client Exchange library
ii  libicu55:amd64                     55.1-7ubuntu0.4                                      amd64        International Components for Unicode
ii  libidn11:amd64                     1.32-3ubuntu1.2                                      amd64        GNU Libidn library, implementation of IETF IDN specifications
ii  libisc160:amd64                    1:9.10.3.dfsg.P4-8ubuntu1.12                         amd64        ISC Shared Library used by BIND
ii  libisccc140:amd64                  1:9.10.3.dfsg.P4-8ubuntu1.12                         amd64        Command Channel Library used by BIND
ii  libisccfg140:amd64                 1:9.10.3.dfsg.P4-8ubuntu1.12                         amd64        Config File Handling Library used by BIND
ii  libisl15:amd64                     0.16.1-1                                             amd64        manipulating sets and relations of integer points bounded by linear constraints
ii  libitm1:amd64                      5.4.0-6ubuntu1~16.04.11                              amd64        GNU Transactional Memory Library
ii  libjbig0:amd64                     2.1-3.1                                              amd64        JBIGkit libraries
ii  libjpeg-turbo8:amd64               1.4.2-0ubuntu3.1                                     amd64        IJG JPEG compliant runtime library.
ii  libjpeg8:amd64                     8c-2ubuntu8                                          amd64        Independent JPEG Group's JPEG runtime library (dependency package)
ii  libk5crypto3:amd64                 1.13.2+dfsg-5ubuntu2.1                               amd64        MIT Kerberos runtime libraries - Crypto Library
ii  libkeyutils1:amd64                 1.5.9-8ubuntu1                                       amd64        Linux Key Management Utilities (library)
ii  libkmod2:amd64                     22-1ubuntu5.2                                        amd64        libkmod shared library
ii  libkrb5-26-heimdal:amd64           1.7~git20150920+dfsg-4ubuntu1.16.04.1                amd64        Heimdal Kerberos - libraries
ii  libkrb5-3:amd64                    1.13.2+dfsg-5ubuntu2.1                               amd64        MIT Kerberos runtime libraries
ii  libkrb5support0:amd64              1.13.2+dfsg-5ubuntu2.1                               amd64        MIT Kerberos runtime libraries - Support library
ii  libldap-2.4-2:amd64                2.4.42+dfsg-2ubuntu3.4                               amd64        OpenLDAP libraries
ii  libllvm6.0:amd64                   1:6.0-1ubuntu2~16.04.1                               amd64        Modular compiler and toolchain technologies, runtime library
ii  liblsan0:amd64                     5.4.0-6ubuntu1~16.04.11                              amd64        LeakSanitizer -- a memory leak detector (runtime)
ii  libltdl-dev:amd64                  2.4.6-0.1                                            amd64        System independent dlopen wrapper for GNU libtool
ii  libltdl7:amd64                     2.4.6-0.1                                            amd64        System independent dlopen wrapper for GNU libtool
ii  liblwres141:amd64                  1:9.10.3.dfsg.P4-8ubuntu1.12                         amd64        Lightweight Resolver Library used by BIND
ii  liblz4-1:amd64                     0.0~r131-2ubuntu2                                    amd64        Fast LZ compression algorithm library - runtime
ii  liblzma-dev:amd64                  5.1.1alpha+20120614-2ubuntu2                         amd64        XZ-format compression library - development files
ii  liblzma5:amd64                     5.1.1alpha+20120614-2ubuntu2                         amd64        XZ-format compression library
ii  liblzo2-2:amd64                    2.08-1.2                                             amd64        data compression library
ii  libmagic1:amd64                    1:5.25-2ubuntu1.2                                    amd64        File type determination library using "magic" numbers
ii  libmail-sendmail-perl              0.79.16-1                                            all          Send email from a perl script
ii  libmnl0:amd64                      1.0.3-5                                              amd64        minimalistic Netlink communication library
ii  libmount1:amd64                    2.27.1-6ubuntu3.6                                    amd64        device mounting library
ii  libmpc3:amd64                      1.0.3-1                                              amd64        multiple precision complex floating-point library
ii  libmpdec2:amd64                    2.4.2-1                                              amd64        library for decimal floating point arithmetic (runtime library)
ii  libmpfr4:amd64                     3.1.4-1                                              amd64        multiple precision floating-point computation
ii  libmpx0:amd64                      5.4.0-6ubuntu1~16.04.11                              amd64        Intel memory protection extensions (runtime)
ii  libnccl-dev                        2.2.12-1+cuda9.1                                     amd64        NVIDIA Collectives Communication Library (NCCL) Development Files
hi  libnccl2                           2.2.12-1+cuda9.1                                     amd64        NVIDIA Collectives Communication Library (NCCL) Runtime
ii  libncurses5:amd64                  6.0+20160213-1ubuntu1                                amd64        shared libraries for terminal handling
ii  libncursesw5:amd64                 6.0+20160213-1ubuntu1                                amd64        shared libraries for terminal handling (wide character support)
ii  libnettle6:amd64                   3.2-1ubuntu0.16.04.1                                 amd64        low level cryptographic library (symmetric and one-way cryptos)
ii  libnl-3-200:amd64                  3.2.27-1ubuntu0.16.04.1                              amd64        library for dealing with netlink sockets
ii  libnl-3-dev:amd64                  3.2.27-1ubuntu0.16.04.1                              amd64        development library and headers for libnl-3
ii  libnl-route-3-200:amd64            3.2.27-1ubuntu0.16.04.1                              amd64        library for dealing with netlink sockets - route interface
ii  libnl-route-3-dev:amd64            3.2.27-1ubuntu0.16.04.1                              amd64        development library and headers for libnl-route-3
ii  libnuma-dev:amd64                  2.0.11-1ubuntu1.1                                    amd64        Development files for libnuma
ii  libnuma1:amd64                     2.0.11-1ubuntu1.1                                    amd64        Libraries for controlling NUMA policy
ii  libonig2:amd64                     5.9.6-1ubuntu0.1                                     amd64        regular expressions library
ii  libopensm                          5.3.0.MLNX20181108.33944a2-0.1.45101                 amd64        Infiniband subnet manager libraries
ii  libp11-kit0:amd64                  0.23.2-5~ubuntu16.04.1                               amd64        library for loading and coordinating access to PKCS#11 modules - runtime
ii  libpam-modules:amd64               1.1.8-3.2ubuntu2.1                                   amd64        Pluggable Authentication Modules for PAM
ii  libpam-modules-bin                 1.1.8-3.2ubuntu2.1                                   amd64        Pluggable Authentication Modules for PAM - helper binaries
ii  libpam-runtime                     1.1.8-3.2ubuntu2.1                                   all          Runtime support for the PAM library
ii  libpam0g:amd64                     1.1.8-3.2ubuntu2.1                                   amd64        Pluggable Authentication Modules library
ii  libpango-1.0-0:amd64               1.38.1-1                                             amd64        Layout and rendering of internationalized text
ii  libpango1.0-dev                    1.38.1-1                                             amd64        Development files for the Pango
ii  libpangocairo-1.0-0:amd64          1.38.1-1                                             amd64        Layout and rendering of internationalized text
ii  libpangoft2-1.0-0:amd64            1.38.1-1                                             amd64        Layout and rendering of internationalized text
ii  libpangoxft-1.0-0:amd64            1.38.1-1                                             amd64        Layout and rendering of internationalized text
ii  libpathplan4                       2.38.0-12ubuntu2.1                                   amd64        rich set of graph drawing tools - pathplan library
ii  libpci3:amd64                      1:3.3.1-1.1ubuntu1.3                                 amd64        Linux PCI Utilities (shared library)
ii  libpciaccess0:amd64                0.13.4-1                                             amd64        Generic PCI access library for X
ii  libpcre16-3:amd64                  2:8.38-3.1                                           amd64        Perl 5 Compatible Regular Expression Library - 16 bit runtime files
ii  libpcre3:amd64                     2:8.38-3.1                                           amd64        Perl 5 Compatible Regular Expression Library - runtime files
ii  libpcre3-dev:amd64                 2:8.38-3.1                                           amd64        Perl 5 Compatible Regular Expression Library - development files
ii  libpcre32-3:amd64                  2:8.38-3.1                                           amd64        Perl 5 Compatible Regular Expression Library - 32 bit runtime files
ii  libpcrecpp0v5:amd64                2:8.38-3.1                                           amd64        Perl 5 Compatible Regular Expression Library - C++ runtime files
ii  libperl-dev                        5.22.1-9ubuntu0.6                                    amd64        Perl library: development files
ii  libperl5.22:amd64                  5.22.1-9ubuntu0.6                                    amd64        shared Perl library
ii  libpipeline1:amd64                 1.4.1-2                                              amd64        pipeline manipulation library
ii  libpixman-1-0:amd64                0.33.6-1                                             amd64        pixel-manipulation library for X and cairo
ii  libpixman-1-dev                    0.33.6-1                                             amd64        pixel-manipulation library for X and cairo (development files)
ii  libpng12-0:amd64                   1.2.54-1ubuntu1.1                                    amd64        PNG library - runtime
ii  libpng12-dev:amd64                 1.2.54-1ubuntu1.1                                    amd64        PNG library - development
ii  libpopt0:amd64                     1.16-10                                              amd64        lib for parsing cmdline parameters
ii  libprocps4:amd64                   2:3.3.10-4ubuntu2.4                                  amd64        library for accessing process information from /proc
ii  libpthread-stubs0-dev:amd64        0.3-4                                                amd64        pthread stubs not provided by native libc, development files
ii  libpython-all-dev:amd64            2.7.12-1~16.04                                       amd64        package depending on all supported Python development packages
ii  libpython-dev:amd64                2.7.12-1~16.04                                       amd64        header files and a static library for Python (default)
ii  libpython-stdlib:amd64             2.7.12-1~16.04                                       amd64        interactive high-level object-oriented language (default python version)
ii  libpython2.7:amd64                 2.7.12-1ubuntu0~16.04.4                              amd64        Shared Python runtime library (version 2.7)
ii  libpython2.7-dev:amd64             2.7.12-1ubuntu0~16.04.4                              amd64        Header files and a static library for Python (v2.7)
ii  libpython2.7-minimal:amd64         2.7.12-1ubuntu0~16.04.4                              amd64        Minimal subset of the Python language (version 2.7)
ii  libpython2.7-stdlib:amd64          2.7.12-1ubuntu0~16.04.4                              amd64        Interactive high-level object-oriented language (standard library, version 2.7)
ii  libpython3-stdlib:amd64            3.5.1-3                                              amd64        interactive high-level object-oriented language (default python3 version)
ii  libpython3.5:amd64                 3.5.2-2ubuntu0~16.04.5                               amd64        Shared Python runtime library (version 3.5)
ii  libpython3.5-minimal:amd64         3.5.2-2ubuntu0~16.04.5                               amd64        Minimal subset of the Python language (version 3.5)
ii  libpython3.5-stdlib:amd64          3.5.2-2ubuntu0~16.04.5                               amd64        Interactive high-level object-oriented language (standard library, version 3.5)
ii  libquadmath0:amd64                 5.4.0-6ubuntu1~16.04.11                              amd64        GCC Quad-Precision Math Library
ii  libreadline6:amd64                 6.3-8ubuntu2                                         amd64        GNU readline and history libraries, run-time libraries
ii  libroken18-heimdal:amd64           1.7~git20150920+dfsg-4ubuntu1.16.04.1                amd64        Heimdal Kerberos - roken support library
ii  librtmp1:amd64                     2.4+20151223.gitfa8646d-1ubuntu0.1                   amd64        toolkit for RTMP streams (shared library)
ii  libsasl2-2:amd64                   2.1.26.dfsg1-14ubuntu0.1                             amd64        Cyrus SASL - authentication abstraction library
ii  libsasl2-modules-db:amd64          2.1.26.dfsg1-14ubuntu0.1                             amd64        Cyrus SASL - pluggable authentication modules (DB)
ii  libseccomp2:amd64                  2.3.1-2.1ubuntu2~16.04.1                             amd64        high level interface to Linux seccomp filter
ii  libselinux1:amd64                  2.4-3build2                                          amd64        SELinux runtime shared libraries
ii  libsemanage-common                 2.3-1build3                                          all          Common files for SELinux policy management libraries
ii  libsemanage1:amd64                 2.3-1build3                                          amd64        SELinux policy management library
ii  libsensors4:amd64                  1:3.4.0-2                                            amd64        library to read temperature/voltage/fan sensors
ii  libsepol1:amd64                    2.4-2                                                amd64        SELinux library for manipulating binary security policies
ii  libsigsegv2:amd64                  2.10-4                                               amd64        Library for handling page faults in a portable way
ii  libslang2:amd64                    2.3.0-2ubuntu1.1                                     amd64        S-Lang programming library - runtime version
ii  libslang2-dev:amd64                2.3.0-2ubuntu1.1                                     amd64        S-Lang programming library, development version
ii  libsm-dev:amd64                    2:1.2.2-1                                            amd64        X11 Session Management library (development headers)
ii  libsm6:amd64                       2:1.2.2-1                                            amd64        X11 Session Management library
ii  libsmartcols1:amd64                2.27.1-6ubuntu3.6                                    amd64        smart column output alignment library
ii  libsqlite3-0:amd64                 3.11.0-1ubuntu1                                      amd64        SQLite 3 shared library
ii  libss2:amd64                       1.42.13-1ubuntu1                                     amd64        command-line interface parsing library
ii  libssl-dev:amd64                   1.0.2g-1ubuntu4.15                                   amd64        Secure Sockets Layer toolkit - development files
ii  libssl-doc                         1.0.2g-1ubuntu4.15                                   all          Secure Sockets Layer toolkit - development documentation
ii  libssl1.0.0:amd64                  1.0.2g-1ubuntu4.15                                   amd64        Secure Sockets Layer toolkit - shared libraries
ii  libstdc++-5-dev:amd64              5.4.0-6ubuntu1~16.04.11                              amd64        GNU Standard C++ Library v3 (development files)
ii  libstdc++6:amd64                   5.4.0-6ubuntu1~16.04.11                              amd64        GNU Standard C++ Library v3
ii  libsys-hostname-long-perl          1.5-1                                                all          Figure out the long (fully-qualified) hostname
ii  libsystemd0:amd64                  229-4ubuntu21.15                                     amd64        systemd utility library
ii  libtasn1-6:amd64                   4.7-3ubuntu0.16.04.3                                 amd64        Manage ASN.1 structures (runtime)
ii  libtcl8.4:amd64                    8.4.20-8                                             amd64        Tcl (the Tool Command Language) v8.5 - run-time library files
ii  libtcl8.6:amd64                    8.6.5+dfsg-2                                         amd64        Tcl (the Tool Command Language) v8.6 - run-time library files
ii  libthai-data                       0.1.24-2                                             all          Data files for Thai language support library
ii  libthai0:amd64                     0.1.24-2                                             amd64        Thai language support library
ii  libtiff5:amd64                     4.0.6-1ubuntu0.6                                     amd64        Tag Image File Format (TIFF) library
ii  libtimedate-perl                   2.3000-2                                             all          collection of modules to manipulate date/time information
ii  libtinfo5:amd64                    6.0+20160213-1ubuntu1                                amd64        shared low-level terminfo library for terminal handling
ii  libtk8.4:amd64                     8.4.20-8                                             amd64        Tk toolkit for Tcl and X11 v8.4 - run-time files
ii  libtk8.6:amd64                     8.6.5-1                                              amd64        Tk toolkit for Tcl and X11 v8.6 - run-time files
ii  libtool                            2.4.6-0.1                                            all          Generic library support script
ii  libtsan0:amd64                     5.4.0-6ubuntu1~16.04.11                              amd64        ThreadSanitizer -- a Valgrind-based detector of data races (runtime)
ii  libtxc-dxtn-s2tc0:amd64            0~git20131104-1.1                                    amd64        Texture compression library for Mesa
ii  libubsan0:amd64                    5.4.0-6ubuntu1~16.04.11                              amd64        UBSan -- undefined behaviour sanitizer (runtime)
ii  libudev1:amd64                     229-4ubuntu21.15                                     amd64        libudev shared library
ii  libunistring0:amd64                0.9.3-5.2ubuntu1                                     amd64        Unicode string library for C
ii  libunwind-dev                      1.1-4.1                                              amd64        library to determine the call-chain of a program - development
ii  libunwind8                         1.1-4.1                                              amd64        library to determine the call-chain of a program - runtime
ii  libunwind8-dev                     1.1-4.1                                              amd64        library to determine the call-chain of a program - development
ii  libusb-0.1-4:amd64                 2:0.1.12-28                                          amd64        userspace USB programming library
ii  libustr-1.0-1:amd64                1.0.4-5                                              amd64        Micro string library: shared library
ii  libutempter0:amd64                 1.1.6-3                                              amd64        privileged helper for utmp/wtmp updates (runtime)
ii  libuuid1:amd64                     2.27.1-6ubuntu3.6                                    amd64        Universally Unique ID library
ii  libvpx3:amd64                      1.5.0-2ubuntu1                                       amd64        VP8 and VP9 video codec (shared library)
ii  libwind0-heimdal:amd64             1.7~git20150920+dfsg-4ubuntu1.16.04.1                amd64        Heimdal Kerberos - stringprep implementation
ii  libx11-6:amd64                     2:1.6.3-1ubuntu2.1                                   amd64        X11 client-side library
ii  libx11-data                        2:1.6.3-1ubuntu2.1                                   all          X11 client-side library
ii  libx11-dev:amd64                   2:1.6.3-1ubuntu2.1                                   amd64        X11 client-side library (development headers)
ii  libx11-doc                         2:1.6.3-1ubuntu2.1                                   all          X11 client-side library (development documentation)
ii  libx11-xcb1:amd64                  2:1.6.3-1ubuntu2.1                                   amd64        Xlib/XCB interface library
ii  libxau-dev:amd64                   1:1.0.8-1                                            amd64        X11 authorisation library (development headers)
ii  libxau6:amd64                      1:1.0.8-1                                            amd64        X11 authorisation library
ii  libxaw7:amd64                      2:1.0.13-1                                           amd64        X11 Athena Widget library
ii  libxcb-dri2-0:amd64                1.11.1-1ubuntu1                                      amd64        X C Binding, dri2 extension
ii  libxcb-dri3-0:amd64                1.11.1-1ubuntu1                                      amd64        X C Binding, dri3 extension
ii  libxcb-glx0:amd64                  1.11.1-1ubuntu1                                      amd64        X C Binding, glx extension
ii  libxcb-present0:amd64              1.11.1-1ubuntu1                                      amd64        X C Binding, present extension
ii  libxcb-render0:amd64               1.11.1-1ubuntu1                                      amd64        X C Binding, render extension
ii  libxcb-render0-dev:amd64           1.11.1-1ubuntu1                                      amd64        X C Binding, render extension, development files
ii  libxcb-shape0:amd64                1.11.1-1ubuntu1                                      amd64        X C Binding, shape extension
ii  libxcb-shm0:amd64                  1.11.1-1ubuntu1                                      amd64        X C Binding, shm extension
ii  libxcb-shm0-dev:amd64              1.11.1-1ubuntu1                                      amd64        X C Binding, shm extension, development files
ii  libxcb-sync1:amd64                 1.11.1-1ubuntu1                                      amd64        X C Binding, sync extension
ii  libxcb1:amd64                      1.11.1-1ubuntu1                                      amd64        X C Binding
ii  libxcb1-dev:amd64                  1.11.1-1ubuntu1                                      amd64        X C Binding, development files
ii  libxcomposite-dev                  1:0.4.4-1                                            amd64        X11 Composite extension library (development headers)
ii  libxcomposite1:amd64               1:0.4.4-1                                            amd64        X11 Composite extension library
ii  libxcursor-dev:amd64               1:1.1.14-1ubuntu0.16.04.2                            amd64        X cursor management library (development files)
ii  libxcursor1:amd64                  1:1.1.14-1ubuntu0.16.04.2                            amd64        X cursor management library
ii  libxdamage-dev:amd64               1:1.1.4-2                                            amd64        X11 damaged region extension library (development headers)
ii  libxdamage1:amd64                  1:1.1.4-2                                            amd64        X11 damaged region extension library
ii  libxdmcp-dev:amd64                 1:1.1.2-1.1                                          amd64        X11 authorisation library (development headers)
ii  libxdmcp6:amd64                    1:1.1.2-1.1                                          amd64        X11 Display Manager Control Protocol library
ii  libxext-dev:amd64                  2:1.3.3-1                                            amd64        X11 miscellaneous extensions library (development headers)
ii  libxext6:amd64                     2:1.3.3-1                                            amd64        X11 miscellaneous extension library
ii  libxfixes-dev:amd64                1:5.0.1-2                                            amd64        X11 miscellaneous 'fixes' extension library (development headers)
ii  libxfixes3:amd64                   1:5.0.1-2                                            amd64        X11 miscellaneous 'fixes' extension library
ii  libxft-dev                         2.3.2-1                                              amd64        FreeType-based font drawing library for X (development files)
ii  libxft2:amd64                      2.3.2-1                                              amd64        FreeType-based font drawing library for X
ii  libxi-dev                          2:1.7.6-1                                            amd64        X11 Input extension library (development headers)
ii  libxi6:amd64                       2:1.7.6-1                                            amd64        X11 Input extension library
ii  libxinerama-dev:amd64              2:1.1.3-1                                            amd64        X11 Xinerama extension library (development headers)
ii  libxinerama1:amd64                 2:1.1.3-1                                            amd64        X11 Xinerama extension library
ii  libxml2:amd64                      2.9.3+dfsg1-1ubuntu0.6                               amd64        GNOME XML library
ii  libxml2-utils                      2.9.3+dfsg1-1ubuntu0.6                               amd64        XML utilities
ii  libxmu6:amd64                      2:1.1.2-2                                            amd64        X11 miscellaneous utility library
ii  libxmuu1:amd64                     2:1.1.2-2                                            amd64        X11 miscellaneous micro-utility library
ii  libxpm4:amd64                      1:3.5.11-1ubuntu0.16.04.1                            amd64        X11 pixmap library
ii  libxrandr-dev:amd64                2:1.5.0-1                                            amd64        X11 RandR extension library (development headers)
ii  libxrandr2:amd64                   2:1.5.0-1                                            amd64        X11 RandR extension library
ii  libxrender-dev:amd64               1:0.9.9-0ubuntu1                                     amd64        X Rendering Extension client library (development files)
ii  libxrender1:amd64                  1:0.9.9-0ubuntu1                                     amd64        X Rendering Extension client library
ii  libxshmfence1:amd64                1.2-1                                                amd64        X shared memory fences - shared library
ii  libxss1:amd64                      1:1.2.2-1                                            amd64        X11 Screen Saver extension library
ii  libxt6:amd64                       1:1.1.5-0ubuntu1                                     amd64        X11 toolkit intrinsics library
ii  libxtables11:amd64                 1.6.0-2ubuntu3                                       amd64        netfilter xtables library
ii  libxtst6:amd64                     2:1.2.2-1                                            amd64        X11 Testing -- Record extension library
ii  libxv1:amd64                       2:1.0.10-1                                           amd64        X11 Video extension library
ii  libxxf86dga1:amd64                 2:1.1.4-1                                            amd64        X11 Direct Graphics Access extension library
ii  libxxf86vm1:amd64                  1:1.1.4-1                                            amd64        X11 XFree86 video mode extension library
ii  libyaml-0-2:amd64                  0.1.6-3                                              amd64        Fast YAML 1.1 parser and emitter library
ii  linux-libc-dev:amd64               4.4.0-141.167                                        amd64        Linux Kernel Headers for development
ii  login                              1:4.2-3.1ubuntu5.3                                   amd64        system login tools
ii  lsb-base                           9.20160110ubuntu0.2                                  all          Linux Standard Base init script functionality
ii  lsb-release                        9.20160110ubuntu0.2                                  all          Linux Standard Base version reporting utility
ii  lsof                               4.89+dfsg-0.1                                        amd64        Utility to list open files
ii  m4                                 1.4.17-5                                             amd64        macro processing language
ii  make                               4.1-6                                                amd64        utility for directing compilation
ii  makedev                            2.3.1-93ubuntu2~ubuntu16.04.1                        all          creates device files in /dev
ii  man-db                             2.7.5-1                                              amd64        on-line manual pager
ii  mawk                               1.3.3-17ubuntu2                                      amd64        a pattern scanning and text processing language
ii  mime-support                       3.59ubuntu1                                          all          MIME files 'mime.types' & 'mailcap', and support programs
ii  mlnx-ofed-kernel-modules           4.5-OFED.4.5.1.0.1.1.gb4fdfac.kver.4.15.0-1040-azure amd64        mlnx-ofed kernel modules
ii  mlnx-ofed-kernel-utils             4.5-OFED.4.5.1.0.1.1.gb4fdfac.kver.4.15.0-1040-azure amd64        Userspace tools to restart and tune mlnx-ofed kernel modules
ii  module-init-tools                  22-1ubuntu5.2                                        all          transitional dummy package (module-init-tools to kmod)
ii  mount                              2.27.1-6ubuntu3.6                                    amd64        tools for mounting and manipulating filesystems
ii  multiarch-support                  2.23-0ubuntu10                                       amd64        Transitional package to ensure multiarch compatibility
ii  ncurses-base                       6.0+20160213-1ubuntu1                                all          basic terminal type definitions
ii  ncurses-bin                        6.0+20160213-1ubuntu1                                amd64        terminal-related programs and man pages
ii  net-tools                          1.60-26ubuntu1                                       amd64        NET-3 networking toolkit
ii  ofed-scripts                       4.5-OFED.4.5.1.0.1                                   amd64        MLNX_OFED utilities
ii  openssh-client                     1:7.2p2-4ubuntu2.8                                   amd64        secure shell (SSH) client, for secure access to remote machines
ii  openssl                            1.0.2g-1ubuntu4.14                                   amd64        Secure Sockets Layer toolkit - cryptographic utility
ii  passwd                             1:4.2-3.1ubuntu5.3                                   amd64        change and administer password and group data
ii  patch                              2.7.5-1ubuntu0.16.04.1                               amd64        Apply a diff file to an original
ii  patchutils                         0.3.4-1                                              amd64        Utilities to work with patches
ii  pciutils                           1:3.3.1-1.1ubuntu1.3                                 amd64        Linux PCI Utilities
ii  perl                               5.22.1-9ubuntu0.6                                    amd64        Larry Wall's Practical Extraction and Report Language
ii  perl-base                          5.22.1-9ubuntu0.6                                    amd64        minimal Perl system
ii  perl-modules-5.22                  5.22.1-9ubuntu0.6                                    all          Core Perl modules
ii  pkg-config                         0.29.1-0ubuntu1                                      amd64        manage compile and link flags for libraries
ii  po-debconf                         1.0.19                                               all          tool for managing templates file translations with gettext
ii  procps                             2:3.3.10-4ubuntu2.4                                  amd64        /proc file system utilities
ii  psmisc                             22.21-2.1build1                                      amd64        utilities that use the proc file system
ii  python                             2.7.12-1~16.04                                       amd64        interactive high-level object-oriented language (default version)
ii  python-all                         2.7.12-1~16.04                                       amd64        package depending on all supported Python runtime versions
ii  python-all-dev                     2.7.12-1~16.04                                       amd64        package depending on all supported Python development packages
ii  python-apt-common                  1.1.0~beta1ubuntu0.16.04.2                           all          Python interface to libapt-pkg (locales)
ii  python-cffi-backend                1.5.2-1ubuntu1                                       amd64        Foreign Function Interface for Python calling C code - backend
ii  python-cryptography                1.2.3-1ubuntu0.2                                     amd64        Python library exposing cryptographic recipes and primitives (Python 2)
ii  python-dev                         2.7.12-1~16.04                                       amd64        header files and a static library for Python (default)
ii  python-enum34                      1.1.2-1                                              all          backport of Python 3.4's enum package
ii  python-idna                        2.0-3                                                all          Python IDNA2008 (RFC 5891) handling (Python 2)
ii  python-ipaddress                   1.0.16-1                                             all          Backport of Python 3 ipaddress module (Python 2)
ii  python-jinja2                      2.8-1                                                all          small but fast and easy to use stand-alone template engine
ii  python-libxml2                     2.9.3+dfsg1-1ubuntu0.6                               amd64        Python bindings for the GNOME XML library
ii  python-markupsafe                  0.23-2build2                                         amd64        HTML/XHTML/XML string library for Python
ii  python-minimal                     2.7.12-1~16.04                                       amd64        minimal subset of the Python language (default version)
ii  python-ndg-httpsclient             0.4.0-3                                              all          enhanced HTTPS support for httplib and urllib2 using PyOpenSSL for Python2
ii  python-netifaces                   0.10.4-0.1build2                                     amd64        portable network interface information - Python 2.x
ii  python-nose                        1.3.7-1                                              all          test discovery and running of Python's unittest
ii  python-openssl                     0.15.1-2ubuntu0.2                                    all          Python 2 wrapper around the OpenSSL library
ii  python-pip                         8.1.1-2ubuntu0.4                                     all          alternative Python package installer
ii  python-pip-whl                     8.1.1-2ubuntu0.4                                     all          alternative Python package installer
ii  python-pkg-resources               20.7.0-1                                             all          Package Discovery and Resource Access using pkg_resources
ii  python-prettytable                 0.7.2-3                                              all          library to represent tabular data in visually appealing ASCII tables
ii  python-pyasn1                      0.1.9-1                                              all          ASN.1 library for Python (Python 2 module)
ii  python-setuptools                  20.7.0-1                                             all          Python Distutils Enhancements
ii  python-six                         1.10.0-3                                             all          Python 2 and 3 compatibility library (Python 2 interface)
ii  python-tz                          2014.10~dfsg1-0ubuntu2                               all          Python version of the Olson timezone database
ii  python-urllib3                     1.13.1-2ubuntu0.16.04.2                              all          HTTP library with thread-safe connection pooling for Python
ii  python-wheel                       0.29.0-1                                             all          built-package format for Python
ii  python-yaml                        3.11-3build1                                         amd64        YAML parser and emitter for Python
ii  python2.7                          2.7.12-1ubuntu0~16.04.4                              amd64        Interactive high-level object-oriented language (version 2.7)
ii  python2.7-dev                      2.7.12-1ubuntu0~16.04.4                              amd64        Header files and a static library for Python (v2.7)
ii  python2.7-minimal                  2.7.12-1ubuntu0~16.04.4                              amd64        Minimal subset of the Python language (version 2.7)
ii  python3                            3.5.1-3                                              amd64        interactive high-level object-oriented language (default python3 version)
ii  python3-apt                        1.1.0~beta1ubuntu0.16.04.2                           amd64        Python 3 interface to libapt-pkg
ii  python3-dbus                       1.2.0-3                                              amd64        simple interprocess messaging system (Python 3 interface)
ii  python3-gi                         3.20.0-0ubuntu1                                      amd64        Python 3 bindings for gobject-introspection libraries
ii  python3-minimal                    3.5.1-3                                              amd64        minimal subset of the Python language (default python3 version)
ii  python3-pycurl                     7.43.0-1ubuntu1                                      amd64        Python bindings to libcurl (Python 3)
ii  python3-software-properties        0.96.20.8                                            all          manage the repositories that you install software from
ii  python3.5                          3.5.2-2ubuntu0~16.04.5                               amd64        Interactive high-level object-oriented language (version 3.5)
ii  python3.5-minimal                  3.5.2-2ubuntu0~16.04.5                               amd64        Minimal subset of the Python language (version 3.5)
ii  quilt                              0.63-3                                               all          Tool to work with series of patches
ii  readline-common                    6.3-8ubuntu2                                         all          GNU readline and history libraries, common files
ii  realpath                           8.25-2ubuntu3~16.04                                  all          coreutils realpath transitional package
ii  rsync                              3.1.1-3ubuntu1.2                                     amd64        fast, versatile, remote (and local) file-copying tool
ii  sed                                4.2.2-7                                              amd64        The GNU sed stream editor
ii  sensible-utils                     0.0.9ubuntu0.16.04.1                                 all          Utilities for sensible alternative selection
ii  sgml-base                          1.26+nmu4ubuntu1                                     all          SGML infrastructure and SGML catalog file support
ii  shared-mime-info                   1.5-2ubuntu0.2                                       amd64        FreeDesktop.org shared MIME database and spec
ii  software-properties-common         0.96.20.8                                            all          manage the repositories that you install software from (common)
ii  sudo                               1.8.16-0ubuntu1.5                                    amd64        Provide limited super user privileges to specific users
ii  swig                               3.0.8-0ubuntu3                                       amd64        Generate scripting interfaces to C/C++ code
ii  swig3.0                            3.0.8-0ubuntu3                                       amd64        Generate scripting interfaces to C/C++ code
ii  systemd                            229-4ubuntu21.15                                     amd64        system and service manager
ii  systemd-sysv                       229-4ubuntu21.15                                     amd64        system and service manager - SysV links
ii  sysv-rc                            2.88dsf-59.3ubuntu2                                  all          System-V-like runlevel change mechanism
ii  sysvinit-utils                     2.88dsf-59.3ubuntu2                                  amd64        System-V-like utilities
ii  tar                                1.28-2.1ubuntu0.1                                    amd64        GNU version of the tar archiving utility
ii  tcl                                8.6.0+9                                              amd64        Tool Command Language (default version) - shell
ii  tcl8.4                             8.4.20-8                                             amd64        Tcl (the Tool Command Language) v8.4 - shell
ii  tcl8.6                             8.6.5+dfsg-2                                         amd64        Tcl (the Tool Command Language) v8.6 - shell
ii  tk                                 8.6.0+9                                              amd64        Toolkit for Tcl and X11 (default version) - windowing shell
ii  tk8.4                              8.4.20-8                                             amd64        Tk toolkit for Tcl and X11, v8.4 - windowing shell
ii  tk8.6                              8.6.5-1                                              amd64        Tk toolkit for Tcl and X11 v8.6 - windowing shell
ii  tzdata                             2018i-0ubuntu0.16.04                                 all          time zone and daylight-saving time data
ii  ubuntu-keyring                     2012.05.19                                           all          GnuPG keys of the Ubuntu archive
ii  ucf                                3.0036                                               all          Update Configuration File(s): preserve user changes to config files
ii  unattended-upgrades                0.90ubuntu0.10                                       all          automatic installation of security upgrades
ii  util-linux                         2.27.1-6ubuntu3.6                                    amd64        miscellaneous system utilities
ii  vim                                2:7.4.1689-3ubuntu1.2                                amd64        Vi IMproved - enhanced vi editor
ii  vim-common                         2:7.4.1689-3ubuntu1.2                                amd64        Vi IMproved - Common files
ii  vim-runtime                        2:7.4.1689-3ubuntu1.2                                all          Vi IMproved - Runtime files
ii  wget                               1.17.1-1ubuntu1.4                                    amd64        retrieves files from the web
ii  x11-common                         1:7.7+13ubuntu3.1                                    all          X Window System (X.Org) infrastructure
ii  x11-utils                          7.7+3                                                amd64        X11 utilities
ii  x11proto-composite-dev             1:0.4.2-2                                            all          X11 Composite extension wire protocol
ii  x11proto-core-dev                  7.0.31-1~ubuntu16.04.2                               all          X11 core wire protocol and auxiliary headers
ii  x11proto-damage-dev                1:1.2.1-2                                            all          X11 Damage extension wire protocol
ii  x11proto-fixes-dev                 1:5.0-2ubuntu2                                       all          X11 Fixes extension wire protocol
ii  x11proto-input-dev                 2.3.1-1                                              all          X11 Input extension wire protocol
ii  x11proto-kb-dev                    1.0.7-0ubuntu1                                       all          X11 XKB extension wire protocol
ii  x11proto-randr-dev                 1.5.0-1                                              all          X11 RandR extension wire protocol
ii  x11proto-render-dev                2:0.11.1-2                                           all          X11 Render extension wire protocol
ii  x11proto-xext-dev                  7.3.0-1                                              all          X11 various extension wire protocol
ii  x11proto-xinerama-dev              1.2.1-2                                              all          X11 Xinerama extension wire protocol
ii  xauth                              1:1.0.9-1ubuntu2                                     amd64        X authentication utility
ii  xbitmaps                           1.1.1-2                                              all          Base X bitmaps
ii  xdg-user-dirs                      0.15-2ubuntu6.16.04.1                                amd64        tool to manage well known user directories
ii  xml-core                           0.13+nmu2                                            all          XML infrastructure and XML catalog file support
ii  xorg-sgml-doctools                 1:1.11-1                                             all          Common tools for building X.Org SGML documentation
ii  xterm                              322-1ubuntu1                                         amd64        X terminal emulator
ii  xtrans-dev                         1.3.5-1                                              all          X transport library (development files)
ii  xz-utils                           5.1.1alpha+20120614-2ubuntu2                         amd64        XZ-format compression utilities
ii  zlib1g:amd64                       1:1.2.8.dfsg-2ubuntu4.1                              amd64        compression library - runtime
ii  zlib1g-dev:amd64                   1:1.2.8.dfsg-2ubuntu4.1                              amd64        compression library - development
---------------- END OF DEBUG INFO -------------------

Output of lspci

0000:00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (AGP disabled) (rev 03)
0000:00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 01)
0000:00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
0000:00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 02)
0000:00:08.0 VGA compatible controller: Microsoft Corporation Hyper-V virtual VGA
0002:00:02.0 Network controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
06ad:00:00.0 3D controller: NVIDIA Corporation GP100GL [Tesla P100 PCIe 16GB] (rev a1)
1b0d:00:00.0 3D controller: NVIDIA Corporation GP100GL [Tesla P100 PCIe 16GB] (rev a1)
cf2d:00:00.0 3D controller: NVIDIA Corporation GP100GL [Tesla P100 PCIe 16GB] (rev a1)
f641:00:00.0 3D controller: NVIDIA Corporation GP100GL [Tesla P100 PCIe 16GB] (rev a1)

Brief what process you are following:

How to reproduce it:

OpenPAI Environment:

Anything else we need to know:

ydye commented 5 years ago

Pending this issue, and still investigate why the installation will failed after upgrade OFED to the latest version. @fanyangCS @scarlett2018 @sterowang

squirrelsc commented 5 years ago

There are some compiler errors.

CC [M] /tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx5/core/debugfs.o /tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx4/catas.c: In function 'mlx4_start_catas_poll': /tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx4/catas.c:282:2: error: implicit declaration of function 'init_timer' [-Werror=implicit-function-declaration] init_timer(&priv->catas_err.timer); ^ /tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx4/catas.c:298:23: error: 'struct timer_list' has no member named 'data' priv->catas_err.timer.data = (unsigned long) dev; ^ /tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx4/catas.c:299:33: error: assignment from incompatible pointer type [-Werror=incompatible-pointer-types] priv->catas_err.timer.function = poll_catas; ^ cc1: all warnings being treated as errors scripts/Makefile.build:332: recipe for target '/tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx4/catas.o' failed make[5]: [/tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx4/catas.o] Error 1 make[5]: Waiting for unfinished jobs....

ydye commented 5 years ago

There are some compiler errors.

CC [M] /tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx5/core/debugfs.o /tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx4/catas.c: In function 'mlx4_start_catas_poll': /tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx4/catas.c:282:2: error: implicit declaration of function 'init_timer' [-Werror=implicit-function-declaration] init_timer(&priv->catas_err.timer); ^ /tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx4/catas.c:298:23: error: 'struct timer_list' has no member named 'data' priv->catas_err.timer.data = (unsigned long) dev; ^ /tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx4/catas.c:299:33: error: assignment from incompatible pointer type [-Werror=incompatible-pointer-types] priv->catas_err.timer.function = poll_catas; ^ cc1: all warnings being treated as errors scripts/Makefile.build:332: recipe for target '/tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx4/catas.o' failed make[5]: [/tmp/mlnx_iso.50/mlnx-ofed-kernel/mlnx-ofed-kernel-4.2/drivers/net/ethernet/mellanox/mlx4/catas.o] Error 1 make[5]: Waiting for unfinished jobs....

This due to linux kernel update, and the interface has been changed. Upgrade OFED version could solve this issue. But After upgrade the OFED version, another problem will occurs. That's what I mentioned in this issue.

ydye commented 5 years ago

Mitigate PR: #2514

scarlett2018 commented 5 years ago

shall we put the mitigation in Mid April or End April? @sterowang @ydye

ydye commented 5 years ago

@scarlett2018 @sterowang I think it should be in.

scarlett2018 commented 5 years ago

cc @Gerhut @qfyin - may we put this in Mid April? since this is only related to installation, should not have a big impact on the entire release plan.

Gerhut commented 5 years ago

cc @Gerhut @qfyin - may we put this in Mid April? since this is only related to installation, should not have a big impact on the entire release plan.

How much is the test effort of this fix? We have only 1 day to do the test in mid April plan.

scarlett2018 commented 5 years ago
ydye commented 5 years ago

cc @Gerhut @qfyin - may we put this in Mid April? since this is only related to installation, should not have a big impact on the entire release plan.

How much is the test effort of this fix? We have only 1 day to do the test in mid April plan.

1) Rebuild Image 2) Before deployment, configure to disable ib installation

drivers:
    skip-ib-installation: true

3) If drivers is passed, then it will be ok.

scarlett2018 commented 5 years ago

@ydye - may you take the test case on Mid April test date? if so, I guess it does not have too much impact on Mid April release. cc @Gerhut @qfyin

ydye commented 5 years ago

ok

scarlett2018 commented 5 years ago

1 thing that can lower the chances to hit the problem is to by default turn off the setting. while on the otherside, we need to consider users who are already using IB when working on the design.

abuccts commented 5 years ago

For Azure VM,

There're builtin InfiniBand kernel modules in vmlinux image, so DO NOT need to install Mellanox OFED. If there's Mellanox device in lspci, then IB device can be used directly. Otherwise, there's no IB in VM.

Corresponding config in /boot/config-$(uname -r):

CONFIG_MLX4_CORE=y
CONFIG_MLX5_CORE=y

Disabled IB driver (OFED) installation in #2595.

abuccts commented 5 years ago

To use InfiniBand in a non-privileged Docker container, we need two flags:

Added in #2657.