search

nvdla / sw

NVDLA SW
Other
477 stars 190 forks source link

KMD Compilation Error #51

Closed silvaurus closed 6 years ago

silvaurus commented 6 years ago

Hi! I'm having an error compiling the KMD. I followed instructions in here Issue 14, downloading linux 4.13.3 and the toolchain, run make KDIR= ARCH=arm64 CROSS_COMPILE=, but it returns an error immediately.

My gcc version is 4.4.7, ldd version 2.17.

Could you help me look into this? Thank you so much!

Error message:

make -C /mnt/y-lab/henry/linux-4.13.3 M=$PWD modules make[1]: Entering directory '/mnt/y-lab/henry/linux-4.13.3'

ERROR: Kernel configuration is invalid. include/generated/autoconf.h or include/config/auto.conf are missing. Run 'make oldconfig && make prepare' on kernel src to fix it.

WARNING: Symbol version dump ./Module.symvers is missing; modules will have no dependencies and modversions.

CC [M] /mnt/y-lab/henry/sw/kmd/port/linux/../../firmware/scheduler.o In file included from :0:0: ././include/linux/kconfig.h:4:10: fatal error: generated/autoconf.h: No such file or directory

include <generated/autoconf.h>

      ^~~~~~~~~~~~~~~~~~~~~~

compilation terminated.

prasshantg commented 6 years ago

@silvaurus can you please try with steps mentioned at http://nvdla.org/vp.html#building-linux-kernel-for-nvdla-virtual-simulator ?

JunningWu commented 6 years ago

@prasshantg I guess @silvaurus want to build nvdla.ko, not a kernel image using Buildroot.

prasshantg commented 6 years ago

that requires kernel build hence pointed to instructions to build kernel using buildroot

silvaurus commented 6 years ago

Thank you so much @prasshantg @JunningWu ! I actually still don't quite understand the whole compile flow. Do I need both virtual environment and KMD/UMD to compile a neuron network implementation? Previously I thought virtual environment is a NN accelerator simulator...

JunningWu commented 6 years ago

@silvaurus you can ref these lines.

cp arch/arm64/configs/defconfig .config

make ARCH=arm64 CROSS_COMPILE=~/Downloads/gcc-linaro-7.2.1-2017.11-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- oldconfig

make ARCH=arm64 CROSS_COMPILE=~/Downloads/gcc-linaro-7.2.1-2017.11-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- -k
prasshantg commented 6 years ago

@silvaurus

You have 3 options

  1. Use docker image https://hub.docker.com/r/nvdla/vp/, it includes pre-built virtual platform including CMOD libs/bins, all KMD and UMD binaries
  2. Use pre-built binaries for KMD and UMD from https://github.com/nvdla/sw/tree/master/prebuilt/linux You will have to prepare virtual platform and build CMOD libs/bins as per http://nvdla.org/vp.html#using-the-virtual-simulator
  3. Build everything yourself, follow http://nvdla.org/vp.html#using-the-virtual-simulator to prepare virtual platform and http://nvdla.org/vp.html#building-linux-kernel-for-nvdla-virtual-simulator to prepare KMD

I am suggesting to use buildroot since we have takes care of all dependencies there.

boiihong commented 6 years ago

@silvaurus

Have you solved the problem? If you already have installed buildroot then you have all you need to build kmd at there: _1. Configured and Built Kernel: buildroot-2017.11/output/build/linux-4.13.3

  1. aarch64 cross compiler: buildroot-2017.11/output/host/bin/aarch64-linux-gnu-_

Please go to nvdla_home/sw/kmd and build it. Build command is as follow... _make KDIR=(pathto)buildroot-2017.11/output/build/linux-4.13.3 ARCH=arm64 CROSS_COMPILE=(pathto)buildroot-2017.11/output/host/bin/aarch64-linux-gnu-_

The opendla.ko is at (nvdla_home)/sw/kmd/port/linux/opendla.ko

silvaurus commented 6 years ago

Yes. Thank you so much!!!

sivagururaman commented 6 years ago

I used buildroot-2011.11. Built the kernel first with the details from: http://nvdla.org/vp.html#building-linux-kernel-for-nvdla-virtual-simulator Then built the KMD with: make KDIR=(path_to)buildroot-2017.11/output/build/linux-4.13.3 ARCH=arm64 CROSS_COMPILE=(pathto)buildroot-2017.11/output/host/bin/aarch64-linux-gnu-

Now I am able to build the KMD for this built linux. Thanks for your help!

prasshantg commented 6 years ago

Excellent, thank you guys!!

Please a leave note here on what was missing from documentation that caused so much trouble and I will get it updated in our README.

ferin08 commented 6 years ago

We followed this website to build using docker: https://hub.docker.com/r/nvdla/vp/ and then built KMD with:

make KDIR=/root/NVDLA/buildroot-2018.02.2/output/build/linux-4.13.6 ARCH=arm64 CROSS_COMPILE=/root/NVDLA/buildroot-2018.02.2/output/host/bin/aarch64-linux-gnu-u-

and we come across this error: Error: duplicate filename '100-uclibc-conf.patch' Conflicting files are: already applied: /root/NVDLA/buildroot-2018.02.2/package/gcc/6.4.0/100-uclibc-conf.patch to be applied : /root/NVDLA/buildroot-2018.02.2/package/gcc/6.4.0/100-uclibc-conf.patch make: *** [/root/NVDLA/buildroot-2018.02.2/output/build/host-gcc-initial-6.4.0/.stamp_patched] Error 1

Could you help us with this.

prasshantg commented 6 years ago

@ferin08

You don't have to use docker to build KMD, it is just to run virtual platform. You can build KMD on host system without docker.

apart from that, did you follow http://nvdla.org/vp.html#configure to build buildroot?

try using toolchain from buildroot/output/host/opt/ext-toolchain/bin

also, you can download verified buildroot from https://github.com/nvdla/buildroot.git to avoid any version issues, likely latest version of buildroot might have changes that we have not verified

ferin08 commented 6 years ago

Hey, Yes, I used this website(http://nvdla.org/vp.html#configure) to build buildroot.

In the host system, I ran this command to build KMD: make KDIR=/root/NVDLA/buildroot-2018.02.2/output/build/linux-4.13.6 ARCH=arm64 CROSS_COMPILE=/root/NVDLA/buildroot-2018.02.2/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu-

And I got this as an error: make: *** No targets specified and no makefile found. Stop.

Why isn't it creating a makefile?

prasshantg commented 6 years ago

@ferin08 It will really help if you can share all steps you followed, it is difficult to tell what you could be missing

ferin08 commented 6 years ago

Ok,

I used this website to run NVDLA vp: https://hub.docker.com/r/nvdla/vp/

I also downloaded NVDLA/hw and NVDLA/sw from githhub. I followed the instructions given here to install the hw and systemc: https://github.com/nvdla/vp (I used the docker to load vp because I faced a mount error when I tried to run it from the source code)

Then I tried to build KMD, where I faced the above mentioned error.

prasshantg commented 6 years ago

@ferin08

Please try below steps exactly as is

  1. Download buildroot https://github.com/nvdla/buildroot.git
  2. cd buildroot
  3. Configure make qemu_aarch64_virt_defconfig
  4. Options from http://nvdla.org/vp.html#configure
  5. make -j4
  6. toolchain downloaded at buildroot/output/host/opt/ext-toolchain/bin
  7. image and rootfs at buildroot/output/images
  8. linux source at buildroot/output/build/linux-master
  9. Download https://github.com/nvdla/sw
  10. cd sw/kmd
  11. Build KMD using, make KDIR= ARCH=arm64 CROSS_COMPILE=

You can use docker for loading VP but it is not required to build KMD

ferin08 commented 6 years ago

Here is the error I got after following the above mentioned steps: Command: root@wipro-Bel:~/NVDLA/sw/kmd# make KDIR=/root/NVDLA/buildroot-master/output/build/linux-4.14.4 ARCH=arm64 CROSS_COMPILE=/root/NVDLA/buildroot-master/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu-

Output: make -C /root/NVDLA/buildroot-master/output/build/linux-4.14.4 M=$PWD modules make[1]: Entering directory /root/NVDLA/buildroot-master/output/build/linux-4.14.4' CC [M] /root/NVDLA/sw/kmd/port/linux/nvdla_gem.o /root/NVDLA/sw/kmd/port/linux/nvdla_gem.c: In function ‘nvdla_drm_probe’: /root/NVDLA/sw/kmd/port/linux/nvdla_gem.c:440:16: error: ‘DMA_MEMORY_MAP’ undeclared (first use in this function) 0x40000000, DMA_MEMORY_MAP | DMA_MEMORY_EXCLUSIVE); ^~~~~~~~~~~~~~ /root/NVDLA/sw/kmd/port/linux/nvdla_gem.c:440:16: note: each undeclared identifier is reported only once for each function it appears in make[3]: *** [/root/NVDLA/sw/kmd/port/linux/nvdla_gem.o] Error 1 make[2]: *** [/root/NVDLA/sw/kmd/port/linux] Error 2 make[1]: *** [_module_/root/NVDLA/sw/kmd] Error 2 make[1]: Leaving directory/root/NVDLA/buildroot-master/output/build/linux-4.14.4' make: *** [modules] Error 2

sivagururaman commented 6 years ago

Looks like this error is mainly due to the Linux version: 4.14.4. I too faced this issue when I had used this version. Instead, can you try 4.13.3? This worked for me.

Apart from this change, your command line is fine.

Regards!

On Fri, Jun 22, 2018 at 11:31 AM, ferin08 notifications@github.com wrote:

Here is the error I got after following the above mentioned steps: Command: root@wipro-Bel:~/NVDLA/sw/kmd# make KDIR=/root/NVDLA/buildroot- master/output/build/linux-4.14.4 ARCH=arm64 CROSS_COMPILE=/root/NVDLA/ buildroot-master/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu-

Output: make -C /root/NVDLA/buildroot-master/output/build/linux-4.14.4 M=$PWD modules make[1]: Entering directory /root/NVDLA/buildroot-master/output/build/linux-4.14.4' CC [M] /root/NVDLA/sw/kmd/port/linux/nvdla_gem.o /root/NVDLA/sw/kmd/port/linux/nvdla_gem.c: In function ‘nvdla_drm_probe’: /root/NVDLA/sw/kmd/port/linux/nvdla_gem.c:440:16: error: ‘DMA_MEMORY_MAP’ undeclared (first use in this function) 0x40000000, DMA_MEMORY_MAP | DMA_MEMORY_EXCLUSIVE); ^~~~~~ /root/NVDLA/sw/kmd/port/linux/nvdla_gem.c:440:16: note: each undeclared identifier is reported only once for each function it appears in make[3]: [/root/NVDLA/sw/kmd/port/linux/nvdla_gem.o] Error 1 make[2]: [/root/NVDLA/sw/kmd/port/linux] Error 2 make[1]: [module/root/NVDLA/sw/kmd] Error 2 make[1]: Leaving directory /root/NVDLA/buildroot-master/output/build/linux-4.14.4' make: [modules] Error 2

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nvdla/sw/issues/51#issuecomment-399329457, or mute the thread https://github.com/notifications/unsubscribe-auth/AmZiiPJpKg4FVo8FdGGpwOXfakW_HDPIks5t_IgzgaJpZM4UUhDr .

prasshantg commented 6 years ago

It is because DMA_MEMORY_MAP is removed in Linux 4.14 with https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/include/linux/dma-mapping.h?h=v4.14.51&id=2436bdcda53ff4abb7897c87fa29ef3de8055344

Either you need to use Linux 4.13.3 or remove it's usage from KMD

ferin08 commented 6 years ago

Is this the way to install linux 4.13.3? http://www.techgazet.com/install-linux-kernel-ubuntu/ Or something similar to this?

Also, how do I remove it's usage from KMD?

prasshantg commented 6 years ago

If you select correct option when configuring buildroot then it takes care of it automatically

$ make menuconfig

ferin08 commented 6 years ago

In buildroot-master, when I executed "make menuconfig": I couldn't set Toolchain -> Custom kernel headers series -> 4.13.x because I didn't have "Custom kernel headers series" on the screen.

So, I went ahead and downloaded buildroot from their official website.

Now, I got linux 4.13.6 and I could build KMD. However I got this error for UMD: I also executed this command: export TOP=/root/NVDLA/sw/umd Error: Command: make Output: for dir in core/runtime tests/runtime; do \ make -C $dir; \ done make[1]: Entering directory /root/NVDLA/sw/umd/core/runtime' Makefile:32: *** Toolchain prefix missing. Stop. make[1]: Leaving directory/root/NVDLA/sw/umd/core/runtime' make[1]: Entering directory /root/NVDLA/sw/umd/tests/runtime' Makefile:32: *** Toolchain prefix missing. Stop. make[1]: Leaving directory/root/NVDLA/sw/umd/tests/runtime' make: *** [subdirs] Error 2

I did "export CROSS_COMPILE=/root/NVDLA/buildroot-2018.02.2/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu-" But it still says toolchain prefix missing. Why is that happening?

prasshantg commented 6 years ago

Please do

cd /root/NVDLA/sw/umd/ export TOP=$(pwd) make TOOLCHAIN_PREFIX=/root/NVDLA/buildroot-2018.02.2/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu-

ferin08 commented 6 years ago

This is what we get after running the commands you mentioned : /NVDLA/sw/umd# make TOOLCHAIN_PREFIX=/root/NVDLA/buildroot-2018.02.2/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu- for dir in core/runtime tests/runtime; do \ make -C $dir; \ done MODULE_CC /root/NVDLA/buildroot-2018.02.2/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu-gcc make[1]: Entering directory /root/NVDLA/sw/umd/core/runtime' make[1]: Nothing to be done forall'. make[1]: Leaving directory /root/NVDLA/sw/umd/core/runtime' MODULE_CC /root/NVDLA/buildroot-2018.02.2/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu-gcc make[1]: Entering directory/root/NVDLA/sw/umd/tests/runtime' make[1]: Nothing to be done for all'. make[1]: Leaving directory/root/NVDLA/sw/umd/tests/runtime'

After logging into vp, I get this error while mounting Command: mount -t 9p -o trans=virtio r /tmp Error: mount: mounting r on /tmp failed: No such device

smsharif1991 commented 6 years ago

Hi @ferin08 ,

Here are the steps which I followed and was able to build successfully both KMD and UMD.

1) Download OpenDla source code for KMD and UMD git clone https://github.com/nvdla/sw.git

2) KMD build a) Buildroot download: https://buildroot.org/downloads/buildroot-2017.11-rc1.tar.bz2

http://nvdla.org/vp.html#building-linux-kernel-for-nvdla-virtual-simulator
Go to that folder and run the below commands and settings
$ make qemu_aarch64_virt_defconfig
$ make menuconfig
* Target Options -> Target Architecture -> AArch64 (little endian)
* Target Options -> Target Architecture Variant -> cortex-A57
* Toolchain -> Custom kernel headers series -> 4.13.x
* Toolchain -> Toolchain type -> External toolchain
* Toolchain -> Toolchain -> Linaro AArch64 2017.08
* Toolchain -> Toolchain origin -> Toolchain to be downloaded and installed
* Kernel -> () Kernel version -> 4.13.3
* Kernel -> Kernel configuration -> Use the architecture default configuration
* System configuration -> Enable root login with password -> Y
* System configuration -> Root password -> nvdla
* Target Packages -> Show packages that are also provided by busybox -> Y
* Target Packages -> Networking applications -> openssh -> Y

$make -j4

b) Go to /kmd make -C /output/build/linux-4.13.3/ M=$TOP/kmd/ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- modules -> To build module make -C /output/build/linux-4.13.3/ M=$TOP/kmd/ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- clean -> To clean module

3) UMD build

protobuf version install 2.6.1 (protoc --version) https://github.com/google/protobuf/releases/download/v2.6.1/protobuf-2.6.1.tar.bz2

Go to protobuf folder and do the following $ ./configure $ make $ make check $ sudo make install $ sudo ldconfig # refresh shared library cache.

cd /umd/ export TOP=pwd export PATH=/output/host/opt/ext-toolchain/bin/:$PATH make TOOLCHAIN_PREFIX=/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu- make clean -> to remove the out directory for UMD

Thanks.

smsharif1991 commented 6 years ago

Here are the steps for VP: Docker instruction Install Docker CE https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#install-docker-ce

Follow the steps at https://hub.docker.com/r/nvdla/vp/

Let us know if things work fine for you , will then document the steps.

Thanks.

ferin08 commented 6 years ago

We followed the comments for installing VP through source code and were successfully able to build UMD KMD. What are the steps after launching VP because the mnt file does not have the drm.ko, opendla.ko (since we loaded it from the folder where CMakeLists.txt was present: which is in "vp" folder)? Also in the mnt file, we couldn't execute the hello test (though the file was present), this was the error we get: command: ./aarch64_hello Error: -sh: ./aarch64_hello: not found

sivagururaman commented 6 years ago

I used drm.ko from prebuilt folder.

opendla.ko is the o/p of your KMD build. It would be in kmd/port/linux/

Regards!

On Tue, Jun 26, 2018 at 11:49 AM, ferin08 notifications@github.com wrote:

We followed the comments for installing VP through source code and were successfully able to build UMD KMD. What are the steps after launching VP because the mnt file does not have the drm.ko, opendla.ko (since we loaded it from the folder where CMakeLists.txt was present: which is in "vp" folder)? Also in the mnt file, we couldn't execute the hello test (though the file was present), this was the error we get: command: ./aarch64_hello Error: -sh: ./aarch64_hello: not found

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nvdla/sw/issues/51#issuecomment-400192055, or mute the thread https://github.com/notifications/unsubscribe-auth/AmZiiFGzyX9Xzstd0VnTwnyXVMJRlqrKks5uAdJmgaJpZM4UUhDr .

anakin1028 commented 6 years ago

@prasshantg @smsharif1991 I want to build the nvdla_runtime by myself, and use this binary to run the loadable on the docker/vp.

I follow the instructions given by you two and I can build the nvdla_runtime successfully. Then I copy this file to the mount directory and run it but got the following segmentation fault:

segmentation_fault

I'm thinking if I need to build the opendla.ko and drm.ko and insmod these modules instead of the prebuilt modules on the docker? I followed the instructions from @prasshantg and try the build the kmd by myself but got the following WARNINGS:

opendla ko

I'm confused about the relationship between opendla.ko, drm.ko and nvdla_runtime. Do opendla.ko and nvdla_runtime belong to the scope of UMD and opendla.ko belongs to the scope of KMD?

When I build the nvdla_runtime in the umd directory I didn't see the output of drm.ko so where can I get this file? How can solve my initial problem to run my own built binary on the docker?

sivagururaman commented 6 years ago

drm.ko can be taken from prebuilt folder. You have to insmod drm.ko and opendla.ko (in the same order) before running nvdla_runtime.

On Tue, Jun 26, 2018 at 1:08 PM, Anakin notifications@github.com wrote:

@prasshantg https://github.com/prasshantg @smsharif1991 https://github.com/smsharif1991 I want to build the nvdla_runtime by myself, and use this binary to run the loadable on the docker/vp.

I follow the instructions given by you two and I can build the nvdla_runtime successfully. Then I copy this file to the mount directory and run it but got the following segmentation fault:

[image: segmentation_fault] https://user-images.githubusercontent.com/6182530/41895885-b79bf620-7955-11e8-96a6-b1593cfde415.PNG

I'm thinking if I need to build the opendla.ko and drm.ko and insmod these modules instead of the prebuilt modules on the docker? I followed the instructions from @prasshantg https://github.com/prasshantg and try the build the kmd by myself but got the following WARNINGS:

[image: opendla ko] https://user-images.githubusercontent.com/6182530/41896039-2bcba07c-7956-11e8-8999-68257a11555e.PNG

I'm confused about the relationship between opendla.ko, drm.ko and nvdla_runtime. Do opendla.ko and nvdla_runtime belong to the scope of UMD and opendla.ko belongs to the scope of KMD?

When I build the nvdla_runtime in the umd directory I didn't see the output of drm.ko so where can I get this file? How can solve my initial problem to run my own built binary on the docker?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nvdla/sw/issues/51#issuecomment-400210737, or mute the thread https://github.com/notifications/unsubscribe-auth/AmZiiKcBiHqcLhcP_TiDfvpOPbWB3xlPks5uAeT9gaJpZM4UUhDr .

anakin1028 commented 6 years ago

@sivagururaman Yes, I know it. The first picture I post in my previous comment is run after insmoding the opendla.ko and drm.ko. So, I wonder if I need to rebuild those kernel modules to test the newly built nvdla_runtime under umd directory.

sivagururaman commented 6 years ago

Yes, better to build the KMD. That will get you the new opendla.ko.

BTW, hope you are using 4.13.3 from the buildroot. Please do so, if you are not.

On Tue, Jun 26, 2018 at 1:24 PM, Anakin notifications@github.com wrote:

@sivagururaman https://github.com/sivagururaman Yes, I know it. The first picture I post in my previous comment is run after insmoding the opendla.ko and drm.ko. So, I wonder if I need to rebuild those kernel modules to test the newly built nvdla_runtime under umd directory.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvdla/sw/issues/51#issuecomment-400215331, or mute the thread https://github.com/notifications/unsubscribe-auth/AmZiiNtYvMCDCUTvr8xrHVwi1gR_obUgks5uAeiogaJpZM4UUhDr .

ferin08 commented 6 years ago

I did the same and these are the outputs I got:

Command: insmod drm.ko Output:

[  141.232829] drm: version magic '4.13.3 SMP preempt mod_unload aarch64' should be '4.13.11 SMP preempt mod_unload aarch64'
[  144.726549] drm: version magic '4.13.3 SMP preempt mod_unload aarch64' should be '4.13.11 SMP preempt mod_unload aarch64'
insmod: can't insert 'drm.ko': invalid module format

Command: insmod opendla.ko Output: [ 157.217912] opendla: loading out-of-tree module taints kernel. [ 157.225850] opendla: Unknown symbol drm_open (err 0) [ 157.229615] opendla: Unknown symbol drm_poll (err 0) [ 157.230623] opendla: Unknown symbol drm_gem_create_mmap_offset (err 0) [ 157.231627] opendla: Unknown symbol drm_gem_prime_handle_to_fd (err 0) [ 157.232648] opendla: Unknown symbol drm_gem_object_lookup (err 0) [ 157.233617] opendla: Unknown symbol drm_gem_private_object_init (err 0) [ 157.234618] opendla: Unknown symbol drm_gem_mmap (err 0) [ 157.235470] opendla: Unknown symbol drm_gem_prime_import (err 0) [ 157.236401] opendla: Unknown symbol drm_ioctl (err 0) [ 157.237242] opendla: Unknown symbol drm_printk (err 0) [ 157.238079] opendla: Unknown symbol drm_gem_prime_fd_to_handle (err 0) [ 157.239068] opendla: Unknown symbol drm_gem_vm_close (err 0) [ 157.239974] opendla: Unknown symbol drm_read (err 0) [ 157.240849] opendla: Unknown symbol drm_gem_object_put_unlocked (err 0) [ 157.241881] opendla: Unknown symbol drm_gem_dumb_destroy (err 0) [ 157.242815] opendla: Unknown symbol drm_gem_handle_create (err 0) [ 157.243790] opendla: Unknown symbol drm_dev_unregister (err 0) [ 157.244793] opendla: Unknown symbol drm_gem_cma_vm_ops (err 0) [ 157.257049] opendla: Unknown symbol drm_dev_unref (err 0) [ 157.258163] opendla: Unknown symbol drm_gem_free_mmap_offset (err 0) [ 157.265044] opendla: Unknown symbol drm_gem_mmap_obj (err 0) [ 157.266204] opendla: Unknown symbol drm_dev_alloc (err 0) [ 157.267976] opendla: Unknown symbol drm_compat_ioctl (err 0) [ 157.268974] opendla: Unknown symbol drm_dev_register (err 0) [ 157.343785] opendla: Unknown symbol drm_release (err 0) [ 157.344617] opendla: Unknown symbol drm_gem_prime_export (err 0) [ 157.696301] opendla: Unknown symbol drm_open (err 0) [ 157.697546] opendla: Unknown symbol drm_poll (err 0) [ 157.728022] opendla: Unknown symbol drm_gem_create_mmap_offset (err 0) [ 157.731503] opendla: Unknown symbol drm_gem_prime_handle_to_fd (err 0) [ 157.736157] opendla: Unknown symbol drm_gem_object_lookup (err 0) [ 157.737433] opendla: Unknown symbol drm_gem_private_object_init (err 0) [ 157.738622] opendla: Unknown symbol drm_gem_mmap (err 0) [ 157.742049] opendla: Unknown symbol drm_gem_prime_import (err 0) [ 157.743153] opendla: Unknown symbol drm_ioctl (err 0) [ 157.748543] opendla: Unknown symbol drm_printk (err 0) [ 157.775471] opendla: Unknown symbol drm_gem_prime_fd_to_handle (err 0) [ 157.776635] opendla: Unknown symbol drm_gem_vm_close (err 0) [ 157.783563] opendla: Unknown symbol drm_read (err 0) [ 157.784769] opendla: Unknown symbol drm_gem_object_put_unlocked (err 0) [ 157.785946] opendla: Unknown symbol drm_gem_dumb_destroy (err 0) [ 157.790164] opendla: Unknown symbol drm_gem_handle_create (err 0) [ 157.793843] opendla: Unknown symbol drm_dev_unregister (err 0) [ 157.823565] opendla: Unknown symbol drm_gem_cma_vm_ops (err 0) [ 157.824489] opendla: Unknown symbol drm_dev_unref (err 0) [ 157.825357] opendla: Unknown symbol drm_gem_free_mmap_offset (err 0) [ 157.826228] opendla: Unknown symbol drm_gem_mmap_obj (err 0) [ 157.836218] opendla: Unknown symbol drm_dev_alloc (err 0) [ 157.837306] opendla: Unknown symbol drm_compat_ioctl (err 0) [ 157.838275] opendla: Unknown symbol drm_dev_register (err 0) [ 157.871165] opendla: Unknown symbol drm_release (err 0) [ 157.872221] opendla: Unknown symbol drm_gem_prime_export (err 0) insmod: can't insert 'opendla.ko': unknown symbol in module, or unknown parameter

I wasn't able to select the kernel version explicitly as 4.13.3. I only had an option which said 4.13. Should I select custom version? Also, I didn't have this option in make menuconfig: Toolchain -> Custom kernel headers series -> 4.13.x

Please let me know how I can fix this error

sivagururaman commented 6 years ago

So it looks to be that the OS on the VP is not 4.13.3. As a result the drm.ko dos not get insmoded.

On Tue, Jun 26, 2018 at 1:41 PM, ferin08 notifications@github.com wrote:

I did the same and these are the outputs I got:

Command: insmod drm.ko Output:

[ 188.017493] drm: version magic '4.13.3 SMP preempt mod_unload aarch64' should be '4.13.11 SMP preempt mod_unload aarch64' [ 191.050284] random: crng init done [ 191.991823] drm: version magic '4.13.3 SMP preempt mod_unload aarch64' should be '4.13.11 SMP preempt mod_unload aarch64' insmod: can't insert 'drm.ko': invalid module format

Command: insmod opendla.ko Output: [ 200.494649] opendla: loading out-of-tree module taints kernel. [ 200.505461] opendla: Unknown symbol drm_open (err 0) [ 200.507843] opendla: Unknown symbol drm_poll (err 0) [ 200.534943] opendla: Unknown symbol drm_gem_create_mmap_offset (err 0) [ 200.536295] opendla: Unknown symbol drm_gem_prime_handle_to_fd (err 0) [ 200.537662] opendla: Unknown symbol drm_gem_object_lookup (err 0) [ 200.538846] opendla: Unknown symbol drm_gem_private_object_init (err 0) [ 200.540372] opendla: Unknown symbol drm_gem_mmap (err 0) [ 200.541799] opendla: Unknown symbol drm_gem_prime_import (err 0) [ 200.550783] opendla: Unknown symbol drm_ioctl (err 0) [ 200.551284] opendla: Unknown symbol drm_printk (err 0) [ 200.551709] opendla: Unknown symbol drm_gem_prime_fd_to_handle (err 0) [ 200.552120] opendla: Unknown symbol drm_gem_vm_close (err 0) [ 200.552484] opendla: Unknown symbol drm_read (err 0) [ 200.552863] opendla: Unknown symbol drm_gem_object_put_unlocked (err 0) [ 200.553215] opendla: Unknown symbol drm_gem_dumb_destroy (err 0) [ 200.553464] opendla: Unknown symbol drm_gem_handle_create (err 0) [ 200.582105] opendla: Unknown symbol drm_dev_unregister (err 0) [ 200.583683] opendla: Unknown symbol drm_gem_cma_vm_ops (err 0) [ 200.584998] opendla: Unknown symbol drm_dev_unref (err 0) [ 200.585938] opendla: Unknown symbol drm_gem_free_mmap_offset (err 0) [ 200.586486] opendla: Unknown symbol drm_gem_mmap_obj (err 0) [ 200.586890] opendla: Unknown symbol drm_dev_alloc (err 0) [ 200.587296] opendla: Unknown symbol drm_compat_ioctl (err 0) [ 200.587695] opendla: Unknown symbol drm_dev_register (err 0) [ 200.588097] opendla: Unknown symbol drm_release (err 0) [ 200.588950] opendla: Unknown symbol drm_gem_prime_export (err 0) [ 201.111921] opendla: Unknown symbol drm_open (err 0) [ 201.112802] opendla: Unknown symbol drm_poll (err 0) [ 201.120572] opendla: Unknown symbol drm_gem_create_mmap_offset (err 0) [ 201.121600] opendla: Unknown symbol drm_gem_prime_handle_to_fd (err 0) [ 201.123099] opendla: Unknown symbol drm_gem_object_lookup (err 0) [ 201.123796] opendla: Unknown symbol drm_gem_private_object_init (err 0) [ 201.124393] opendla: Unknown symbol drm_gem_mmap (err 0) [ 201.125065] opendla: Unknown symbol drm_gem_prime_import (err 0) [ 201.164230] opendla: Unknown symbol drm_ioctl (err 0) [ 201.165066] opendla: Unknown symbol drm_printk (err 0) [ 201.172446] opendla: Unknown symbol drm_gem_prime_fd_to_handle (err 0) [ 201.173737] opendla: Unknown symbol drm_gem_vm_close (err 0) [ 201.178843] opendla: Unknown symbol drm_read (err 0) [ 201.179793] opendla: Unknown symbol drm_gem_object_put_unlocked (err 0) [ 201.184487] opendla: Unknown symbol drm_gem_dumb_destroy (err 0) [ 201.184955] opendla: Unknown symbol drm_gem_handle_create (err 0) [ 201.185476] opendla: Unknown symbol drm_dev_unregister (err 0) [ 201.186043] opendla: Unknown symbol drm_gem_cma_vm_ops (err 0) [ 201.186547] opendla: Unknown symbol drm_dev_unref (err 0) [ 201.187002] opendla: Unknown symbol drm_gem_free_mmap_offset (err 0) [ 201.187511] opendla: Unknown symbol drm_gem_mmap_obj (err 0) [ 201.187990] opendla: Unknown symbol drm_dev_alloc (err 0) [ 201.188458] opendla: Unknown symbol drm_compat_ioctl (err 0) [ 201.188898] opendla: Unknown symbol drm_dev_register (err 0) [ 201.189371] opendla: Unknown symbol drm_release (err 0) [ 201.189804] opendla: Unknown symbol drm_gem_prime_export (err 0) insmod: can't insert 'opendla.ko': unknown symbol in module, or unknown parameter

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvdla/sw/issues/51#issuecomment-400220156, or mute the thread https://github.com/notifications/unsubscribe-auth/AmZiiNuJnzii-iwujNOLhIIp32a2hzpIks5uAey1gaJpZM4UUhDr .

anakin1028 commented 6 years ago

@ferin08 how do you build drm.ko? and when you build the opendla.ko do you get the same result as I in my previous post?

anakin1028 commented 6 years ago

The kernel is 4.13.3 but with PREEMPT, from stdout:

uname -a
Linux ndla 4.13.3 #1 SMP PREEMPT Mon Nov 27 09:22:34 CST 2017 aarch64 GNU/Linux

Since I don't know how to build the drm.ko, I use the prebuilt one first and install the opendla.ko, and I got the same result as @ferin08:

# insmod ./sw/kmd/port/linux/opendla.ko
opendla: version magic '4.13.3 SMP mod_unload aarch64' should be '4.13.3 SMP preempt mod_unload aarch64'

The two questions remain for me:

  1. how to build drm.ko?
  2. how to bypass the version magic problem?
smsharif1991 commented 6 years ago

Hi @anakin1028 / @ferin08 ,

Please lets use the prebuilts file to see if things are fine. I have already verified that things work fine with files from prebuilts. (Buildroot should be 4.13.3 , see my comment in this issue) Please try the below steps as is:

1)docker pull nvdla/vp 2)docker run -it -v /home:/home nvdla/vp Inside the container 3)cd /usr/local/nvdla 4) cp /sw/prebuilts/linux/opendla.ko . 5) cp /sw/prebuilts/linux/drm.ko . 6) cp /sw/prebuilts/linux/nvdla_runtime . 7) cp /sw/prebuilts/linux/libnvdla_runtime.so . 8) cp /sw/regression/flatbufs/NN/NN_L0_0_fbuf . 9) aarch64_toplevel -c aarch64_nvdla.lua # start the virtual simulator Login the kernel with account 'root' and password 'nvdla'

After login the kernel: 10)mount -t 9p -o trans=virtio r /mnt # mount pwd 11)cd /mnt 12) insmod drm.ko # install drm driver 13) insmod opendla.ko # install nvdla driver 14) ./nvdla_runtime --loadable NN_L0_0_fbuf

This should work fine for both of you.

Thanks.

anakin1028 commented 6 years ago

Hi prebuilt looks good for me. But my intention is to build the own binary so that I can add my debug message to the s/w

smsharif1991 commented 6 years ago

@anakin1028 , I saw the segmentation fault what you faced that was due that you did not copy both nvdla_runtime and libnvdla_runtime.so

So whenever there is a change in lib code update the library as well.

Are you able to build UMD properly??

Thanks.

anakin1028 commented 6 years ago

@smsharif1991 thanks for the quick reply. Oh.. Yes, I didn't move the libnvdla_runtime.so to the mount directory.

@smsharif1991 two last problems are: a. where can I build the drm.ko? from the KMD directory I can only find the opendla.ko b. when insmod opendla.ko, the kernel version of the buildroot looks different with the version of
docker? As in my previous comment:

# insmod ./sw/kmd/port/linux/opendla.ko
opendla: version magic '4.13.3 SMP mod_unload aarch64' should be '4.13.3 SMP preempt mod_unload aarch64'

Thanks!

smsharif1991 commented 6 years ago

@anakin1028 drm.ko is present in the buildroot (output/build/linux-4.13.3/drivers/gpu/drm/drm.ko)

Are you able to build opendla.ko from source code??

Try the below steps: 1) cd path_to_opendla/sw/ 2) export TOP=pwd 3) export PATH=buildroot_path/output/host/opt/ext-toolchain/bin/:$PATH 4)make -C buildroot_path/output/build/linux-4.13.3/ M=$TOP/kmd/ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- modules

opendla.ko will be generated in sw/kmd/port/linux/

Thanks.

anakin1028 commented 6 years ago

@smsharif1991 I just re-run the nvdla_runtime and the output is slightly different with the prebuilt one, but I don't know why.

The prebuilt version: nvdla_runtime_ok The nvdla_runtime I build (which I don't modify anything, just build from the sw source) my_nvdla_runtime

There is Shutdown signal receved, exiting in my built one. Does this because the version in the sw is different with prebuilt one?

smsharif1991 commented 6 years ago

@anakin1028 you can ignore that, the test is running fine on DLA.

Thanks.

anakin1028 commented 6 years ago

@smsharif1991 thanks!!! Yes, I can build opendla.ko successfully, but the size looks different with the prebuilt one in the docker.

My opendla.ko: The size is around 81K opendla ko

The one on the docker: The size is 891K prebuilt_opendla

And when I build the opendla.ko, I got the following WARNING messages:

warning_for_opendla ko

And I go to the directory you mention for drm.ko, I can't not find this .ko file: Should I build drm.ko first so that I can build opendla.ko? The chinese in the stdout just means there is no drm.ko. no_drm ko

anakin1028 commented 6 years ago

Should I enable the libdrm when build the kernel (buildroot)? libdrm

prasshantg commented 6 years ago

libdrm is not required but make sure CONFIG_DRM=m is enabled in kernel .config that you used

anakin1028 commented 6 years ago

@prasshantg thanks. Is there any mapping config in make menuconfig? Since I follow the instructions given by here: builtroot configuration, the CONFIG_DRM doesn't default enable.

anakin1028 commented 6 years ago

@prasshantg @smsharif1991,

From the .config in buildroot-2018.5 I only find the config BR2_PACKAGE_LIBDRM. Even if I manually add CONFIG_DRM=m to the .config to rebuild the kernel, the config will be re-written again and doesn't build the DRM.ko at all.

I do a search in make menuconfig and I cannot find the CONFIG_DRM configuration. Any pointer is appreciated. Thanks!

anakin1028 commented 6 years ago

@ferin08 do you solve the problem? After I change the kernel to buildroot-2017.11-rc1, the kernel version becomes correct (PREEMPT is enabled) and the drm.ko get built. Thanks everyone here!

ferin08 commented 6 years ago

Yes after we changed the Linux version we were able to build everything successfully and run the tests on VP! Thank You everyone for all the help!