quic / cloud-ai-sdk

Qualcomm Cloud AI SDK (Platform and Apps) enable high performance deep learning inference on Qualcomm Cloud AI platforms delivering high throughput and low latency across Computer Vision, Object Detection, Natural Language Processing and Generative AI models.
https://quic.github.io/cloud-ai-sdk-pages/latest/
Other
52 stars 5 forks source link

Platform SDK Update failed on Amazon Linux (on dl2q.24xlarge AWS instance) #3

Closed arjunsuresh closed 7 months ago

arjunsuresh commented 7 months ago

After installing 12.2 Platform SDK the devices are not coming up. Even a system reboot didn't help.

Same issue is happening while installing SDK on Ubuntu 22.04 and Ubuntu 20.04 instances on AWS. It worked fine on a Thundercomm RB6 running Ubuntu 18.04.

[ec2-user@ip-172-31-58-28 rpm]$ bash install.sh
Failed to set locale, defaulting to C
Loaded plugins: dkms-build-requires, extras_suggestions, langpacks, priorities, update-motd, versionlock
amzn2-core                                                                                                                                                                                               | 3.6 kB  00:00:00

dbus.x86_64                                                                                                  1:1.10.24-7.amzn2.0.4                                                                                    amzn2-core
dbus-libs.x86_64                                                                                             1:1.10.24-7.amzn2.0.4                                                                                    amzn2-core
ec2-hibinit-agent.noarch                                                                                     1.0.8-0.amzn2                                                                                            amzn2-core
freetype.x86_64                                                                                              2.8-14.amzn2.1.2                                                                                         amzn2-core
indent.x86_64                                                                                                2.2.11-13.amzn2.0.4                                                                                      amzn2-core
nss-softokn.x86_64                                                                                           3.90.0-6.amzn2.0.1                                                                                       amzn2-core
nss-softokn-freebl.x86_64                                                                                    3.90.0-6.amzn2.0.1                                                                                       amzn2-core
pam.x86_64                                                                                                   1.1.8-23.amzn2.0.2                                                                                       amzn2-core
pcre2.x86_64                                                                                                 10.23-11.amzn2.0.2                                                                                       amzn2-core
python-jinja2.noarch                                                                                         2.7.2-3.amzn2.0.1                                                                                        amzn2-core
systemd.x86_64                                                                                               219-78.amzn2.0.23                                                                                        amzn2-core
systemd-devel.x86_64                                                                                         219-78.amzn2.0.23                                                                                        amzn2-core
systemd-libs.x86_64                                                                                          219-78.amzn2.0.23                                                                                        amzn2-core
systemd-sysv.x86_64                                                                                          219-78.amzn2.0.23                                                                                        amzn2-core
traceroute.x86_64                                                                                            3:2.0.22-2.amzn2.0.2                                                                                     amzn2-core
tzdata.noarch                                                                                                2023d-1.amzn2.0.1                                                                                        amzn2-core
tzdata-java.noarch                                                                                           2023d-1.amzn2.0.1                                                                                        amzn2-core
Failed to set locale, defaulting to C
Loaded plugins: dkms-build-requires, extras_suggestions, langpacks, priorities, update-motd, versionlock
Package kernel-devel-5.10.205-195.804.amzn2.x86_64 already installed and latest version
Package dkms-2.6.1-1.amzn2.0.1.noarch already installed and latest version
Package pciutils-3.5.1-3.amzn2.x86_64 already installed and latest version
No package libnsl available.
Package ncurses-libs-6.0-8.20170212.amzn2.1.7.x86_64 already installed and latest version
Package ncurses-compat-libs-6.0-8.20170212.amzn2.1.7.x86_64 already installed and latest version
Nothing to do
Not running under HyperV
Failed to set locale, defaulting to C
Loaded plugins: dkms-build-requires, extras_suggestions, langpacks, priorities, update-motd, versionlock
Examining qaic-fw-1.12.2.0-1.el7.x86_64.rpm: qaic-fw-1.12.2.0-1.el7.x86_64
Marking qaic-fw-1.12.2.0-1.el7.x86_64.rpm to be installed
Examining qaic-kmd-1.12.2.0-1.el7.noarch.rpm: qaic-kmd-1.12.2.0-1.el7.noarch
Marking qaic-kmd-1.12.2.0-1.el7.noarch.rpm to be installed
Examining qaic-rt-1.12.2.0-1.el7.x86_64.rpm: qaic-rt-1.12.2.0-1.el7.x86_64
Marking qaic-rt-1.12.2.0-1.el7.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package qaic-fw.x86_64 0:1.12.2.0-1.el7 will be installed
---> Package qaic-kmd.noarch 0:1.12.2.0-1.el7 will be installed
---> Package qaic-rt.x86_64 0:1.12.2.0-1.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================================================================================================================================================================
 Package                                        Arch                                         Version                                                Repository                                                             Size
================================================================================================================================================================================================================================
Installing:
 qaic-fw                                        x86_64                                       1.12.2.0-1.el7                                         /qaic-fw-1.12.2.0-1.el7.x86_64                                        4.1 M
 qaic-kmd                                       noarch                                       1.12.2.0-1.el7                                         /qaic-kmd-1.12.2.0-1.el7.noarch                                       821 k
 qaic-rt                                        x86_64                                       1.12.2.0-1.el7                                         /qaic-rt-1.12.2.0-1.el7.x86_64                                         58 M

Transaction Summary
================================================================================================================================================================================================================================
Install  3 Packages

Total size: 63 M
Installed size: 63 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
  Installing : qaic-fw-1.12.2.0-1.el7.x86_64                                                                                                                                                                                1/3
  Installing : qaic-kmd-1.12.2.0-1.el7.noarch                                                                                                                                                                               2/3

Creating symlink /var/lib/dkms/qaic/1.12.2.0/source ->
                 /usr/src/qaic-1.12.2.0

DKMS: add completed.

Kernel preparation unnecessary for this kernel.  Skipping...

Building module:
cleaning build area....
make -j96 KERNELRELEASE=5.10.205-195.804.amzn2.x86_64.....
cleaning build area...

DKMS: build completed.

mhi.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/5.10.205-195.804.amzn2.x86_64/kernel/qaic//

qaic.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/5.10.205-195.804.amzn2.x86_64/kernel/qaic//
Adding any weak-modules
/usr/bin/dracut: line 656: warning: setlocale: LC_CTYPE: cannot change locale (): No such file or directory
/usr/bin/dracut: line 656: warning: setlocale: LC_CTYPE: cannot change locale (): No such file or directory

gzip: /boot/initramfs-5.10.205-195.804.amzn2.x86_64.img: not in gzip format

gzip: /boot/initramfs-5.10.205-195.804.amzn2.x86_64.tmp: not in gzip format

depmod...

DKMS: install completed.
  Installing : qaic-rt-1.12.2.0-1.el7.x86_64                                                                                                                                                                                3/3
  Verifying  : qaic-kmd-1.12.2.0-1.el7.noarch                                                                                                                                                                               1/3
  Verifying  : qaic-rt-1.12.2.0-1.el7.x86_64                                                                                                                                                                                2/3
  Verifying  : qaic-fw-1.12.2.0-1.el7.x86_64                                                                                                                                                                                3/3

Installed:
  qaic-fw.x86_64 0:1.12.2.0-1.el7                                           qaic-kmd.noarch 0:1.12.2.0-1.el7                                           qaic-rt.x86_64 0:1.12.2.0-1.el7

Complete!
didn't install for these kernels: 5.10.199-190.747.amzn2.x86_64
Warning: KMD did not install on all kernels on the system.
Extracting qaic-test-data...
Waiting for devices to be in QSM state (may take upto 15 minutes max) ...
Device status:
QidNNC  0000:00:18.0    Error
QidQID  0000:00:19.0    Error
Qid0    0000:00:1a.0    Error
QidNNC  0000:00:1b.0    Error
QidQID  0000:00:1c.0    Error
Qid1    0000:00:1d.0    Error
QidNNC  0000:00:1e.0    Error
QidQID  0000:00:1f.0    Error
Installation complete. Resetting devices ...
Resetting /sys/bus/mhi/devices/mhi0/soc_reset
Resetting /sys/bus/mhi/devices/mhi1/soc_reset
Resetting /sys/bus/mhi/devices/mhi2/soc_reset
Resetting /sys/bus/mhi/devices/mhi3/soc_reset
Resetting /sys/bus/mhi/devices/mhi4/soc_reset
Resetting /sys/bus/mhi/devices/mhi5/soc_reset
Resetting /sys/bus/mhi/devices/mhi6/soc_reset
Resetting /sys/bus/mhi/devices/mhi7/soc_reset
Reset complete.