search

nvdla / sw

NVDLA SW
Other
482 stars 191 forks source link

KMD Compilation Error #51

Closed silvaurus closed 6 years ago

silvaurus commented 6 years ago

Hi! I'm having an error compiling the KMD. I followed instructions in here Issue 14, downloading linux 4.13.3 and the toolchain, run make KDIR= ARCH=arm64 CROSS_COMPILE=, but it returns an error immediately.

My gcc version is 4.4.7, ldd version 2.17.

Could you help me look into this? Thank you so much!

Error message:

make -C /mnt/y-lab/henry/linux-4.13.3 M=$PWD modules make[1]: Entering directory '/mnt/y-lab/henry/linux-4.13.3'

ERROR: Kernel configuration is invalid. include/generated/autoconf.h or include/config/auto.conf are missing. Run 'make oldconfig && make prepare' on kernel src to fix it.

WARNING: Symbol version dump ./Module.symvers is missing; modules will have no dependencies and modversions.

CC [M] /mnt/y-lab/henry/sw/kmd/port/linux/../../firmware/scheduler.o In file included from :0:0: ././include/linux/kconfig.h:4:10: fatal error: generated/autoconf.h: No such file or directory

include <generated/autoconf.h>

      ^~~~~~~~~~~~~~~~~~~~~~

compilation terminated.

silvaurus commented 6 years ago

Just wrote a note on the whole building flow. Thanks a lot for the many help I got from all of you!

System Requirement:

MUST use Ubuntu 14.04 - GCC 4.8.4 !

  1. Virtual Enviroment

    Follow this http://nvdla.org/vp.html

    Download Virtual Simulator:
$ git clone https://github.com/nvdla/vp.git
$ cd vp
$ git submodule update --init --recursive

Install Dependencies
$ sudo apt-get update
$ sudo apt-get install g++ cmake libboost-dev python-dev libglib2.0-dev libpixman-1-dev liblua5.2-dev swig libcap-dev libattr1-dev

Install SystemC 2.3.0
$ wget -O systemc-2.3.0a.tar.gz http://www.accellera.org/images/downloads/standards/systemc/systemc-2.3.0a.tar.gz
$ tar -xzvf systemc-2.3.0a.tar.gz
$ cd systemc-2.3.0a
$ sudo mkdir -p /usr/local/systemc-2.3.0/
$ mkdir objdir
$ cd objdir
$ ../configure --prefix=/usr/local/systemc-2.3.0
$ make
$ sudo make install

Download and build NVDLA CMOD
Before download, install those two Perl packages!
$ sudo apt install libyaml-perl libio-tee-perl
$ git clone https://github.com/nvdla/hw.git
$ cd hw
$ make
$ tools/bin/tmake -build cmod_top

Build the Virtual Simulator
$ cmake -DCMAKE_INSTALL_PREFIX=build -DSYSTEMC_PREFIX=/usr/local/systemc-2.3.0/ -DNVDLA_HW_PREFIX=/PATH/TO/NVDLA/hw -DNVDLA_HW_PROJECT=nv_full
$ make
$ make install

  2. BuildRoot - (IF you want to use 2017-11)

    Instead of downloading buildroot from its website, now need to download from its repository, and go back to branch 2017.11.

    $ git clone git://git.buildroot.net/buildroot
$ cd buildroot
$ git branch -r list all remote branch
$ git checkout -t the branch you want (orgin/2017.11.x)

    Then can follow the link here: http://nvdla.org/vp.html#building-linux-kernel-for-nvdla-virtual-simulator

    $ make qemu_aarch64_virt_defconfig
$ make menuconfig
* Target Options -> Target Architecture -> AArch64 (little endian)
* Target Options -> Target Architecture Variant -> cortex-A57
* Toolchain -> Custom kernel headers series -> 4.13.x
* Toolchain -> Toolchain type -> External toolchain
* Toolchain -> Toolchain -> Linaro AArch64 2017.08
* Toolchain -> Toolchain origin -> Toolchain to be downloaded and installed
* Kernel -> () Kernel version -> 4.13.3
* Kernel -> Kernel configuration -> Use the architecture default configuration
* System configuration -> Enable root login with password -> Y
* System configuration -> Root password -> nvdla
* Target Packages -> Show packages that are also provided by busybox -> Y
* Target Packages -> Networking applications -> openssh -> Y

$ make -j4

  3. KMD

    $ git clone https://github.com/nvdla/sw.git
$ cd sw/kmd
$ make KDIR=(path_to)buildroot-2017.11/output/build/linux-4.13.3 ARCH=arm64 CROSS_COMPILE=(path_to_)buildroot-2017.11/output/host/bin/aarch64-linux-gnu-_

The opendla.ko is at sw/kmd/port/linux/opendla.ko

  4. UMD

    $ cd sw/umd
$ export TOP=$(pwd)
$ make TOOLCHAIN_PREFIX=(path_to)buildroot-2017.11/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu-

  5. Usage

    1. Running nvdla_compile on Ubuntu

      1. Files needed:

        1. nvdla_compile binary file from /home/yiizy/nvdla/sw/prebuilt/linux

        2. prototxtfile (User input, such as the BVLC Alexnet https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet)

        3. caffemodelfile (User input, such as the BVLC Alexnet http://dl.caffe.berkeleyvision.org/bvlc_alexnet.caffemodel)

        4. Running commands:

        ./nvdla_compiler --prototxt YOUR_prototxt_FILE --caffemodel YOUR_caffemodel_FILE -o YOUR/OUTPUT/NAME

        (Normally one only needs the basic.nvdla file generated.)

    2. Running nvdla_runtime on Virtual Environment

      1. Start the Virtual Environment following http://nvdla.org/vp.html , mount current directory ./vp to /mnt

        $ export SC_SIGNAL_WRITE_CHECK=DISABLE
$ ./build/bin/aarch64_toplevel -c conf/aarch64_nvdla.lua
Login the kernel. The demo image uses account 'root' and password 'nvdla'.
# mount -t 9p -o trans=virtio r /mnt
# cd /mnt

      2. In Ubuntu, copy the following files to vp:

        1. The nvdla_runtime binary just built in sw/umd/out/runtime/nvdla_runtime/
        2. buildroot/output/build/linux-4.13.3/drivers/gpu/drm/drm.ko
        3. sw/kmd/port/linux/opendla.ko
        4. basic.nvdla generated by compiler
        5. Your image to be inferenced

      3. Running commands inside of virtual environment:

        # insmod drm.ko
# insmod opendia.ko
# ./nvdla_runtime --loadable basic.nvdla --image ./YOUR/IMAGE --normalize 1.0 --rawdump
sivagururaman commented 6 years ago

@Prashant - I have a question about the nvdla_compiler. Not sure if this belongs to this category or any other one.

The documentation says the compiler generates and converts the data format from FP to int or equivalent that would best run on HW. But i see that all the caffemodels get used only in FP version on the HW. Is there anything that I need to give as input to the nvdla_compiler? Can you help here?

Thanks, Sivam.

On Fri, Jul 13, 2018 at 7:39 AM, Yi He notifications@github.com wrote:

Just wrote a note on the whole building flow. Thanks a lot for the many help I got from all of you!

System Requirement:

MUST use Ubuntu 14.04 - GCC 4.8.4 !

1.

Virtual Enviroment

Follow this http://nvdla.org/vp.html

Download Virtual Simulator: $ git clone https://github.com/nvdla/vp.git $ cd vp $ git submodule update --init --recursive

Install Dependencies $ sudo apt-get update $ sudo apt-get install g++ cmake libboost-dev python-dev libglib2.0-dev libpixman-1-dev liblua5.2-dev swig libcap-dev libattr1-dev

Install SystemC 2.3.0 $ wget -O systemc-2.3.0a.tar.gz http://www.accellera.org/images/downloads/standards/systemc/systemc-2.3.0a.tar.gz $ tar -xzvf systemc-2.3.0a.tar.gz $ cd systemc-2.3.0a $ sudo mkdir -p /usr/local/systemc-2.3.0/ $ mkdir objdir $ cd objdir $ ../configure --prefix=/usr/local/systemc-2.3.0 $ make $ sudo make install

Download and build NVDLA CMOD Before download, install those two Perl packages! $ sudo apt install libyaml-perl libio-tee-perl $ git clone https://github.com/nvdla/hw.git $ cd hw $ make $ tools/bin/tmake -build cmod_top

Build the Virtual Simulator $ cmake -DCMAKE_INSTALL_PREFIX=build -DSYSTEMC_PREFIX=/usr/local/systemc-2.3.0/ -DNVDLA_HW_PREFIX=/PATH/TO/NVDLA/hw -DNVDLA_HW_PROJECT=nv_full $ make $ make install

2.

BuildRoot - (IF you want to use 2017-11)

Instead of downloading buildroot from its website, now need to download from its repository, and go back to branch 2017.11.

$ git clone git://git.buildroot.net/buildroot $ cd buildroot $ git branch -r list all remote branch $ git checkout -t the branch you want (orgin/2017.11.x)

Then can follow the link here: http://nvdla.org/vp.html# building-linux-kernel-for-nvdla-virtual-simulator http://nvdla.org/vp.html#building-linux-kernel-for-nvdla-virtual-simulator

$ make qemu_aarch64_virt_defconfig $ make menuconfig Target Options -> Target Architecture -> AArch64 (little endian) Target Options -> Target Architecture Variant -> cortex-A57 Toolchain -> Custom kernel headers series -> 4.13.x Toolchain -> Toolchain type -> External toolchain Toolchain -> Toolchain -> Linaro AArch64 2017.08 Toolchain -> Toolchain origin -> Toolchain to be downloaded and installed Kernel -> () Kernel version -> 4.13.3 Kernel -> Kernel configuration -> Use the architecture default configuration System configuration -> Enable root login with password -> Y System configuration -> Root password -> nvdla Target Packages -> Show packages that are also provided by busybox -> Y Target Packages -> Networking applications -> openssh -> Y

$ make -j4

3.

KMD

$ git clone https://github.com/nvdla/sw.git $ cd sw/kmd $ make KDIR=(path_to)buildroot-2017.11/output/build/linux-4.13.3 ARCH=arm64 CROSS_COMPILE=(pathto)buildroot-2017.11/output/host/bin/aarch64-linux-gnu-_

The opendla.ko is at sw/kmd/port/linux/opendla.ko

4.

UMD

$ cd sw/umd $ export TOP=$(pwd) $ make TOOLCHAIN_PREFIX=(path_to)buildroot-2017.11/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu-

5.

Usage 1.

  Running nvdla_compile on Ubuntu
  1.

     Files needed:
     1.

        nvdla_compile binary file from /home/yiizy/nvdla/sw/prebuilt/
        linux
        2.

        prototxtfile (User input, such as the BVLC Alexnet
        https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet
        <https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet>
        )
        3.

        caffemodelfile (User input, such as the BVLC Alexnet
        http://dl.caffe.berkeleyvision.org/bvlc_alexnet.caffemodel
        <http://dl.caffe.berkeleyvision.org/bvlc_alexnet.caffemodel>)
        4.

        Running commands:

        ./nvdla_compiler --prototxt YOUR_prototxt_FILE --caffemodel YOUR_caffemodel_FILE -o YOUR/OUTPUT/NAME

        (Normally one only needs the basic.nvdla file generated.)
        2.

  Running nvdla_runtime on Virtual Environment
  1.

     Start the Virtual Environment following http://nvdla.org/vp.html
     , mount current directory ./vp to /mnt

     $ export SC_SIGNAL_WRITE_CHECK=DISABLE
     $ ./build/bin/aarch64_toplevel -c conf/aarch64_nvdla.lua
     Login the kernel. The demo image uses account 'root' and password 'nvdla'.# mount -t 9p -o trans=virtio r /mnt# cd /mnt

     2.

     In Ubuntu, copy the following files to vp:
     1. The nvdla_runtime binary just built in
        sw/umd/out/runtime/nvdla_runtime/
        2. buildroot/output/build/linux-4.13.3/drivers/gpu/drm/drm.ko
        3. sw/kmd/port/linux/opendla.ko
        4. basic.nvdla generated by compiler
        5. Your image to be inferenced
     3.

     Running commands inside of virtual environment:

     # insmod drm.ko# insmod opendia.ko# ./nvdla_runtime --loadable basic.nvdla --image ./YOUR/IMAGE --normalize 1.0 --rawdump

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvdla/sw/issues/51#issuecomment-404704012, or mute the thread https://github.com/notifications/unsubscribe-auth/AmZiiAd5fEPfODbwwKxpLtdq5wvMG--qks5uGAFIgaJpZM4UUhDr .

prasshantg commented 6 years ago

@sivagururaman your question is not very clear to me. I would answer from what I understand. Compiler is expected to support converting fp32 models to fp16 or int8 as per HW requirement. Some DLA HW configurations will support both fp16 and int8 while others will support just int8.

Currently compiler can do only fp32 to fp16, fp32 to int8 support will be added in future release. I hope it answers your question.

sivagururaman commented 6 years ago

@Prashant Thanks for the o/p format description. That is a very useful info for us. Another info I am looking for is how to access the memory from CPU to dump input and output of each of the blocks. For e.g: we wanted to dump I/o of conv stage and see how the data transaction takes place. Can you pass on the details here? I tried dla_get_dma_address() with DESTINATION_PROCESSOR. And used dla_data_read() with the address returned by the former. Not sure if this is correct. Wanted to hear back from you.

Regards, Sivam.

On Fri, 20 Jul 2018 at 1:00 PM Prashant Gaikwad notifications@github.com wrote:

@sivagururaman https://github.com/sivagururaman your question is not very clear to me. I would answer from what I understand. Compiler is expected to support converting fp32 models to fp16 or int8 as per HW requirement. Some DLA HW configurations will support both fp16 and int8 while others will support just int8.

Currently compiler can do only fp32 to fp16, fp32 to int8 support will be added in future release. I hope it answers your question.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvdla/sw/issues/51#issuecomment-406513865, or mute the thread https://github.com/notifications/unsubscribe-auth/AmZiiAUmZTIwqODpVMwO_mpRONEnjX5Wks5uIYcQgaJpZM4UUhDr .

-- Regards, Sivam.

prasshantg commented 6 years ago

@sivagururaman that should work to read data

sivagururaman commented 6 years ago

@Prashant, Thanks. But I do not see the desired values getting printed. Wanted to check if I have missed out any step. It does print out data but not what we ought to have fed the engine.

Regards, Sivam.

On Mon, 23 Jul 2018 at 1:12 PM Prashant Gaikwad notifications@github.com wrote:

@sivagururaman https://github.com/sivagururaman that should work to read data

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvdla/sw/issues/51#issuecomment-406969001, or mute the thread https://github.com/notifications/unsubscribe-auth/AmZiiA-v28KsGX08LyTOPkEJfDnnSMRvks5uJX5_gaJpZM4UUhDr .

-- Regards, Sivam.

prasshantg commented 6 years ago

When do you print the data? And how are you deciding index to use for getting address?

sivagururaman commented 6 years ago

@Prashant- I dump the data just before submit operation to Conv block. Regarding offset, I do not know. How do I get the correct offset/index? I was using offset=0 all along.

Regards, Sivam

On Mon, 23 Jul 2018 at 1:37 PM Prashant Gaikwad notifications@github.com wrote:

When do you print the data? And how are you deciding index to use for getting address?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvdla/sw/issues/51#issuecomment-406974991, or mute the thread https://github.com/notifications/unsubscribe-auth/AmZiiLqTj6eDujMcqYlvm_FubQdk8q7uks5uJYRIgaJpZM4UUhDr .

-- Regards, Sivam.

sivagururaman commented 6 years ago

@Prashant - So how do I get the index for different I/O data for each stage of the n.w? Ahy help here would be helpful.

Do I use the &(struct.element) - &(struct.base) as teh offset and pass that as an argument for the dla_dma_get_address()? Can you please help?

Thanks, Sivam.

On Mon, Jul 23, 2018 at 1:44 PM, Sivam Mahadevan < sivam.mahadevan@broadcom.com> wrote:

@Prashant- I dump the data just before submit operation to Conv block. Regarding offset, I do not know. How do I get the correct offset/index? I was using offset=0 all along.

Regards, Sivam

On Mon, 23 Jul 2018 at 1:37 PM Prashant Gaikwad notifications@github.com wrote:

When do you print the data? And how are you deciding index to use for getting address?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvdla/sw/issues/51#issuecomment-406974991, or mute the thread https://github.com/notifications/unsubscribe-auth/AmZiiLqTj6eDujMcqYlvm_FubQdk8q7uks5uJYRIgaJpZM4UUhDr .

-- Regards, Sivam.

prasshantg commented 6 years ago

You will have to decode it from engine programming. For example, to get index for input data to convolution engine, you can add debug print conv_surface->src_data.address at https://github.com/nvdla/sw/blob/master/kmd/firmware/conv.c#L319

sivagururaman commented 6 years ago

Thank you. So if I understand correctly, if I had to dump the weights, then I need to do the same from the op_desc for weights as well, and then use that. Am I correct?

Regards, Sivam.

On Mon, Jul 23, 2018 at 7:47 PM, Prashant Gaikwad notifications@github.com wrote:

You will have to decode it from engine programming. For example, to get index for input data to convolution engine, you can add debug print conv_surface->src_data.address at https://github.com/nvdla/sw/ blob/master/kmd/firmware/conv.c#L319

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvdla/sw/issues/51#issuecomment-407073796, or mute the thread https://github.com/notifications/unsubscribe-auth/AmZiiFhEnzvR4A5kV1vemvkAeX87bLZDks5uJdsDgaJpZM4UUhDr .

prasshantg commented 6 years ago

For weight data, you will get it from conv_surface->weight_data.address

sivagururaman commented 6 years ago

And then apply the same index that we get from the group of the operation?

On Mon, Jul 23, 2018 at 9:09 PM, Prashant Gaikwad notifications@github.com wrote:

For weight data, you will get it from conv_surface->weight_data.address

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvdla/sw/issues/51#issuecomment-407102496, or mute the thread https://github.com/notifications/unsubscribe-auth/AmZiiP-xsKmjwyfI3KQzTVoBoLaRmPI2ks5uJe48gaJpZM4UUhDr .

sivagururaman commented 6 years ago

For the index=0 (as I had mentioned last week), I had added the following lines to read 512bytes of weight data:

            dla_get_dma_address(engine->driver_context,
                                    engine->task->task_data,
                                    (conv_surface->weight_data.address

), (void *)&my_weight_address, DESTINATION_PROCESSOR );

            dla_data_read(engine->driver_context,
                                    engine->task->task_data,
                                    my_weight_address,
                                    (void *)&my_weight_data[0],
                                    //conv_surface->weight_data.size,
                                    512,
                                    0
                                    );

My aim was to get the data onto my_weight_data[]. Just wanted to run by you if this snippet is on expected lined. Not sure, if I get correct data onto this array with this change.

Regards, Sivam.

On Mon, Jul 23, 2018 at 9:11 PM, Sivam Mahadevan < sivam.mahadevan@broadcom.com> wrote:

And then apply the same index that we get from the group of the operation?

On Mon, Jul 23, 2018 at 9:09 PM, Prashant Gaikwad < notifications@github.com> wrote:

For weight data, you will get it from conv_surface->weight_data.address

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvdla/sw/issues/51#issuecomment-407102496, or mute the thread https://github.com/notifications/unsubscribe-auth/AmZiiP-xsKmjwyfI3KQzTVoBoLaRmPI2ks5uJe48gaJpZM4UUhDr .

chifredhong commented 6 years ago

I'm reading the UMD and KMD source code .I see the default data type of tensor is NvU8 in loadable file. Does the NVDLA Compiler convert the FP32 into uint8，and each processor operate with uint8 datatype? how to modify the data precision configuration under VP?@prasshantg