search

VVViy / VVViy.github.io

This is Max's blog, something interesting in it.
MIT License
13 stars 4 forks source link

nv_small FPGA Mapping Workflow - Max's Blog #3

Open VVViy opened 5 years ago

VVViy commented 5 years ago

https://vvviy.github.io/2018/09/17/nv_small-FPGA-Mapping-Workflow-II/

Keep self busy.

VVViy commented 5 years ago

@peterzh2018888

@VVViy

@peterzh2018888 @VVViy I can't find out opendla.ko in /lib/modules/[kernel version]-xilinx-[petalinux version]/extra/ only see mali.ko and can't be insmod

Seriously？Did u create opendla external module by following STEP 4？

Yes I create opendla external module by following STEP 4. but no opendla.ko create

That is so weird ! When u relaunched $ petalinux-build cmd, was there any information about opendla module?

ghost commented 5 years ago

@VVViy root@aidla:/# insmod /lib/modules/4.14.0-xilinx-v2018.2/extra/opendla.ko [ 170.618471] reset engine done [ 170.621785] [drm] Initialized nvdla 0.0.0 20171017 for a0000000.nvdla_small on minor 1 [ 170.629694] NVDLA a0000000.nvdla_small: failed to register drm device [ 170.636085] NVDLA: probe of a0000000.nvdla_small failed with error -12 root@aidla:/# ls /dev/dri/ card0 card1 renderD128 root@aidla:/# cat /proc/interrupts CPU0 CPU1 CPU2 CPU3 3: 2520 4132 10902 1416 GICv2 30 Level arch_timer 6: 0 0 0 0 GICv2 67 Level ff9905c0.mailbox 7: 0 0 0 0 GICv2 175 Level arm-pmu 8: 0 0 0 0 GICv2 176 Level arm-pmu 9: 0 0 0 0 GICv2 177 Level arm-pmu 10: 0 0 0 0 GICv2 178 Level arm-pmu 13: 0 0 0 0 GICv2 156 Level zynqmp-dma 14: 0 0 0 0 GICv2 157 Level zynqmp-dma 15: 0 0 0 0 GICv2 158 Level zynqmp-dma 16: 0 0 0 0 GICv2 159 Level zynqmp-dma 17: 0 0 0 0 GICv2 160 Level zynqmp-dma 18: 0 0 0 0 GICv2 161 Level zynqmp-dma 19: 0 0 0 0 GICv2 162 Level zynqmp-dma 20: 0 0 0 0 GICv2 163 Level zynqmp-dma 21: 0 0 0 0 GICv2 164 Level Mali_GP_MMU, Mali_GP, Mali_PP0_MMU, Mali_PP0, Mali_PP1_MMU, Mali_PP1 22: 0 0 0 0 GICv2 109 Level zynqmp-dma 23: 0 0 0 0 GICv2 110 Level zynqmp-dma 24: 0 0 0 0 GICv2 111 Level zynqmp-dma 25: 0 0 0 0 GICv2 112 Level zynqmp-dma 26: 0 0 0 0 GICv2 113 Level zynqmp-dma 27: 0 0 0 0 GICv2 114 Level zynqmp-dma 28: 0 0 0 0 GICv2 115 Level zynqmp-dma 29: 0 0 0 0 GICv2 116 Level zynqmp-dma 31: 0 0 0 0 GICv2 95 Level eth0, eth0 33: 484 0 0 0 GICv2 49 Level cdns-i2c 34: 43 0 0 0 GICv2 50 Level cdns-i2c 35: 0 0 0 0 GICv2 42 Level ff960000.memory-controller 36: 0 0 0 0 GICv2 57 Level axi-pmon 37: 0 0 0 0 GICv2 150 Level nwl_pcie:misc 42: 23 0 0 0 GICv2 47 Level ff0f0000.spi 43: 0 0 0 0 GICv2 58 Level ffa60000.rtc 44: 0 0 0 0 GICv2 59 Level ffa60000.rtc 45: 0 0 0 0 GICv2 165 Level ahci-ceva[fd0c0000.ahci] 46: 1749 0 0 0 GICv2 81 Level mmc0 47: 365 0 0 0 GICv2 53 Level xuartps 49: 0 0 0 0 GICv2 145 Edge fd4d0000.watchdog 50: 0 0 0 0 GICv2 88 Level ams-irq 51: 0 0 0 0 GICv2 154 Level fd4c0000.dma 52: 0 0 0 0 GICv2 151 Level fd4a0000.zynqmp-display 62: 0 0 0 0 GICv2 97 Level xhci-hcd:usb1 65: 0 0 0 0 zynq-gpio 22 Edge sw19 IPI0: 2197 1885 1916 1401 Rescheduling interrupts IPI1: 7 3 8 8 Function call interrupts IPI2: 0 0 0 0 CPU stop interrupts IPI3: 0 0 0 0 CPU stop (for crash dump) interrupts IPI4: 2192 1898 955 2321 Timer broadcast interrupts IPI5: 0 0 0 0 IRQ work interrupts IPI6: 0 0 0 0 CPU wake-up interrupts Err: 0 root@aidla:/# insmod /lib/modules/4.14.0-xilinx-v2018.2/extra/opendla.ko insmod: can't insert '/lib/modules/4.14.0-xilinx-v2018.2/extra/opendla.ko': File exists root@aidla:/# 现在改了make file 后可以正常生成openddla.ko,但是安装驱动有错。NVDLA a0000000.nvdla_small: failed to register drm device。cat /proc/interrupts也不对。另外编译umd还是出找不到error路径，我已经改在/ar/log/下了，我也有这个目录，不知道为啥还报错

ghost commented 5 years ago

root@aidla:/media/card/umd# vi /media/card/umd/include/ErrorMacros.h ..

if !defined (NVDLA_UTILS_ERROR_PATH)

define NVDLA_UTILS_ERROR_PATH "/var/log/"

endif

..

root@aidla:/media/card/umd# make TOOLCHAIN_PREFIX=/usr/bin/ for dir in core/runtime tests/runtime; do \ make -C $dir; \ done make[1]: Entering directory '/media/card/umd/core/runtime' rules.mk:58: /make/module.mk: No such file or directory make[1]: No rule to make target '/make/module.mk'. Stop. make[1]: Leaving directory '/media/card/umd/core/runtime' make[1]: Entering directory '/media/card/umd/tests/runtime' rules.mk:53: /make/module.mk: No such file or directory make[1]: No rule to make target '/make/module.mk'. Stop. make[1]: Leaving directory '/media/card/umd/tests/runtime' make: *** [Makefile:32: subdirs] Error 2 root@aidla:/media/card/umd#

在板上编译umd还是有问题呀，找不到路径rules.mk:58: /make/module.mk: No such file or directory

VVViy commented 5 years ago

@peterzh2018888 1）检查一下kmd源码修改是否正确，我在上面提到了，drm_gem_object_unreference_unlocked函数是linux kernel v4.12之前版本使用的，你的kernel 版本是4.14，所以不要修改这个函数，识别不了； 2）第二个要检查的是，是否去掉了dma_declare_coherent_memory函数中的DMA_MEMORY_MAP，这个flag在kernel v4.13之后的版本中停用了； 3）希望你将修改的地方与blog文中的对比一下，确定符合自己版本修改，再检查一下相关文件修改是否符合语法要求； 4）至于umd error path问题，我记得petalinux默认是没有“/var/log/“这个文件夹的，你有两种处理方式，一个是改成根目录，如“/var/”, 或者进入petalinux环境后，先在“/var”下mkdir一个“log”文件夹，再insmod.

还是仔细检查一下修改文件，您这些奇怪的问题，我在摸索过程中都没见过。

VVViy commented 5 years ago

@peterzh2018888 root@aidla:/media/card/umd# vi /media/card/umd/include/ErrorMacros.h ..

if !defined (NVDLA_UTILS_ERROR_PATH)

define NVDLA_UTILS_ERROR_PATH "/var/log/"

endif

..

root@aidla:/media/card/umd# make TOOLCHAIN_PREFIX=/usr/bin/ for dir in core/runtime tests/runtime; do \ make -C $dir; \ done make[1]: Entering directory '/media/card/umd/core/runtime' rules.mk:58: /make/module.mk: No such file or directory make[1]: No rule to make target '/make/module.mk'. Stop. make[1]: Leaving directory '/media/card/umd/core/runtime' make[1]: Entering directory '/media/card/umd/tests/runtime' rules.mk:53: /make/module.mk: No such file or directory make[1]: No rule to make target '/make/module.mk'. Stop. make[1]: Leaving directory '/media/card/umd/tests/runtime' make: *** [Makefile:32: subdirs] Error 2 root@aidla:/media/card/umd#

在板上编译umd还是有问题呀，找不到路径rules.mk:58: /make/module.mk: No such file or directory

你echo一下$TOP，看看路径对不对，我原文中那块写错了，刚改过来，要么用export TOP=${PWD}, 或者用export TOP=$(pwd)也可以

ghost commented 5 years ago

@VVViy

@peterzh2018888 root@aidla:/media/card/umd# vi /media/card/umd/include/ErrorMacros.h ..

if !defined (NVDLA_UTILS_ERROR_PATH)

define NVDLA_UTILS_ERROR_PATH "/var/log/"

endif

..

root@aidla:/media/card/umd# make TOOLCHAIN_PREFIX=/usr/bin/ for dir in core/runtime tests/runtime; do \ make -C $dir; \ done make[1]: Entering directory '/media/card/umd/core/runtime' rules.mk:58: /make/module.mk: No such file or directory make[1]: No rule to make target '/make/module.mk'. Stop. make[1]: Leaving directory '/media/card/umd/core/runtime' make[1]: Entering directory '/media/card/umd/tests/runtime' rules.mk:53: /make/module.mk: No such file or directory make[1]: No rule to make target '/make/module.mk'. Stop. make[1]: Leaving directory '/media/card/umd/tests/runtime' make: *** [Makefile:32: subdirs] Error 2 root@aidla:/media/card/umd#

在板上编译umd还是有问题呀，找不到路径rules.mk:58: /make/module.mk: No such file or directory

你echo一下$TOP，看看路径对不对，我原文中那块写错了，刚改过来，要么用export TOP=${PWD}, 或者用export TOP=$(pwd)也可以

好了可以编译通过，但是没有生成run_test.sh

ghost commented 5 years ago

@VVViy @peterzh2018888 1）检查一下kmd源码修改是否正确，我在上面提到了，drm_gem_object_unreference_unlocked函数是linux kernel v4.12之前版本使用的，你的kernel 版本是4.14，所以不要修改这个函数，识别不了； 2）第二个要检查的是，是否去掉了dma_declare_coherent_memory函数中的DMA_MEMORY_MAP，这个flag在kernel v4.13之后的版本中停用了； 3）希望你将修改的地方与blog文中的对比一下，确定符合自己版本修改，再检查一下相关文件修改是否符合语法要求； 4）至于umd error path问题，我记得petalinux默认是没有“/var/log/“这个文件夹的，你有两种处理方式，一个是改成根目录，如“/var/”, 或者进入petalinux环境后，先在“/var”下mkdir一个“log”文件夹，再insmod.

还是仔细检查一下修改文件，您这些奇怪的问题，我在摸索过程中都没见过。 您说的这些都改过了。可能是dts不对？

VVViy commented 5 years ago

@peterzh2018888

@VVViy

@peterzh2018888 root@aidla:/media/card/umd# vi /media/card/umd/include/ErrorMacros.h ..

if !defined (NVDLA_UTILS_ERROR_PATH)

define NVDLA_UTILS_ERROR_PATH "/var/log/"

endif

..

root@aidla:/media/card/umd# make TOOLCHAIN_PREFIX=/usr/bin/ for dir in core/runtime tests/runtime; do \ make -C $dir; \ done make[1]: Entering directory '/media/card/umd/core/runtime' rules.mk:58: /make/module.mk: No such file or directory make[1]: No rule to make target '/make/module.mk'. Stop. make[1]: Leaving directory '/media/card/umd/core/runtime' make[1]: Entering directory '/media/card/umd/tests/runtime' rules.mk:53: /make/module.mk: No such file or directory make[1]: No rule to make target '/make/module.mk'. Stop. make[1]: Leaving directory '/media/card/umd/tests/runtime' make: *** [Makefile:32: subdirs] Error 2 root@aidla:/media/card/umd#

在板上编译umd还是有问题呀，找不到路径rules.mk:58: /make/module.mk: No such file or directory

你echo一下$TOP，看看路径对不对，我原文中那块写错了，刚改过来，要么用export TOP=${PWD}, 或者用export TOP=$(pwd)也可以

好了可以编译通过，但是没有生成run_test.sh

run_test.sh是自己写的测试脚本，不是自动生成的啊，你写一个就行了。

ghost commented 5 years ago

@VVViy

@peterzh2018888 root@aidla:/media/card/umd# vi /media/card/umd/include/ErrorMacros.h ..

if !defined (NVDLA_UTILS_ERROR_PATH)

define NVDLA_UTILS_ERROR_PATH "/var/log/"

endif

..

root@aidla:/media/card/umd# make TOOLCHAIN_PREFIX=/usr/bin/ for dir in core/runtime tests/runtime; do \ make -C $dir; \ done make[1]: Entering directory '/media/card/umd/core/runtime' rules.mk:58: /make/module.mk: No such file or directory make[1]: No rule to make target '/make/module.mk'. Stop. make[1]: Leaving directory '/media/card/umd/core/runtime' make[1]: Entering directory '/media/card/umd/tests/runtime' rules.mk:53: /make/module.mk: No such file or directory make[1]: No rule to make target '/make/module.mk'. Stop. make[1]: Leaving directory '/media/card/umd/tests/runtime' make: *** [Makefile:32: subdirs] Error 2 root@aidla:/media/card/umd#

在板上编译umd还是有问题呀，找不到路径rules.mk:58: /make/module.mk: No such file or directory

你echo一下$TOP，看看路径对不对，我原文中那块写错了，刚改过来，要么用export TOP=${PWD}, 或者用export TOP=$(pwd)也可以 最后TIP中 export LD_LIBRARY_PATH=

没有生成的path-to-libnvdla_runtime.o只有libnvdla_runtime.so我export 之后,找到nvdla_runtime 执行后还是说load LB error

ghost commented 5 years ago

@VVViy

&nodename{

    compatible =  "....";

};

node or label? nvdla_small_0 or nvdla_small? nvdla_small can't pass,so must be nvdla_small_0.

VVViy commented 5 years ago

@peterzh2018888 你是个学生？ 对于device tree的所有问题，建议你直接去我的后一篇博客中看。测试脚本可以参考nvdla/hw#110

VVViy commented 5 years ago

下面就是测试脚本，另外，年后我事情比较多，可能回复的会比较晚。

#!/bin/sh

export LD_LIBRARY_PATH=umd/out/runtime/libnvdla_runtime/

RUNTIME=umd/out/runtime/nvdla_runtime/nvdla_runtime

FLATBUFS=kmd
TESTS="PDP/PDP_L0_0_small_fbuf CONV/CONV_D_L0_0_small_fbuf SDP/SDP_X1_L0_0_small_fbuf CDP/CDP_L0_0_small_fbuf NN/NN_L0_1_small_fbuf"

echo "== Tests for nv_small =="
echo "========================"

for test in ${TESTS}; do
    echo "= Run ${test}"
    ${RUNTIME} --loadable ${FLATBUFS}/${test}
done
VVViy commented 5 years ago

@peterzh2018888

@VVViy 下面就是测试脚本，另外，年后我事情比较多，可能回复的会比较晚。

#!/bin/sh

export LD_LIBRARY_PATH=umd/out/runtime/libnvdla_runtime/

RUNTIME=umd/out/runtime/nvdla_runtime/nvdla_runtime

FLATBUFS=kmd
TESTS="PDP/PDP_L0_0_small_fbuf CONV/CONV_D_L0_0_small_fbuf SDP/SDP_X1_L0_0_small_fbuf CDP/CDP_L0_0_small_fbuf NN/NN_L0_1_small_fbuf"

echo "== Tests for nv_small =="
echo "========================"

for test in ${TESTS}; do
    echo "= Run ${test}"
    ${RUNTIME} --loadable ${FLATBUFS}/${test}
done

root@aidla:/media/card# su root ./run_test.sh == Tests for nv_small ==

= Run PDP/PDP_L0_0_small_fbuf creating new runtime context... Emulator starting submitting tasks... Shutdown signal received, exiting Test pass = Run CONV/CONV_D_L0_0_small_fbuf creating new runtime context... Emulator starting submitting tasks... Shutdown signal received, exiting Test pass = Run SDP/SDP_X1_L0_0_small_fbuf creating new runtime context... Emulator starting submitting tasks... Shutdown signal received, exiting Test pass = Run CDP/CDP_L0_0_small_fbuf creating new runtime context... Emulator starting submitting tasks... Shutdown signal received, exiting Test pass = Run NN/NN_L0_1_small_fbuf creating new runtime context... Emulator starting submitting tasks...

一直停在submitting tasks... 这个对吗？

NN测试用例是网络测试，可能会比较慢，但也不会过久，如10 min。如果一直停在这里，说明你的配置还是有问题。 写博客就是想与人交流，避免别人走不必要弯路的，不过这个也是我业余时间维护，工作忙时，我一般都是晚上回家才会看。

VVViy commented 5 years ago

拷贝到SD的是UMD，只需要修改error path。

pl.dtsi里 nv_small node的reg字段是APB总线配置空间的地址，而kmd driver里的dma=dma_declare_coherent_memory(drm->dev, 0x40000000, 0x40000000,0x40000000, DMA_MEMORY_MAP | DMA_MEMORY_EXCLUSIVE);是DMA的地址空间需求，要和STEP 5中system-conf.dtsi文件里的reserved-memory node的reg字段对应。

blog里没提到的东西，就不需要修改，需要修改的，blog里都提到了。

VVViy commented 5 years ago

@peterzh2018888

@VVViy 拷贝到SD的是UMD，只需要修改error path。

pl.dtsi里 nv_small node的reg字段是APB总线配置空间的地址，而kmd driver里的dma=dma_declare_coherent_memory(drm->dev, 0x40000000, 0x40000000,0x40000000, DMA_MEMORY_MAP | DMA_MEMORY_EXCLUSIVE);是DMA的地址空间需求，要和STEP 5中system-conf.dtsi文件里的reserved-memory node的reg字段对应。

blog里没提到的东西，就不需要修改，需要修改的，blog里都提到了。 == Tests for nv_small ==

= Run PDP/PDP_L0_0_small_fbuf creating new runtime context... Emulator starting submitting tasks... [ 2203.960987] Enter:dla_read_network_config [ 2203.967666] Exit:dla_read_network_config status=0 [ 2203.973362] Enter: dla_initiate_processors [ 2203.978475] Enter: dla_submit_operation [ 2203.983334] Prepare PDP operation index 0 ROI 0 dep_count 0 [ 2203.989980] Enter: dla_prepare_operation [ 2203.994993] processor:PDP group:0, rdma_group:0 available [ 2204.001528] Enter: dla_read_config [ 2204.006088] Exit: dla_read_config [ 2204.010576] Exit: dla_prepare_operation status=0 [ 2204.016385] Enter: dla_program_operation [ 2204.021490] Program PDP operation index 0 ROI 0 Group[0] [ 2204.027968] group id 0 rdma id 0 [ 2204.032353] no desc get due to index==-1 [ 2204.037444] no desc get due to index==-1 [ 2204.042539] no desc get due to index==-1 [ 2204.047628] no desc get due to index==-1 [ 2204.052724] no desc get due to index==-1 [ 2204.057824] no desc get due to index==-1 [ 2204.062928] no desc get due to index==-1 [ 2204.068025] Enter: dla_op_programmed [ 2204.072789] Exit: dla_op_programmed [ 2204.077466] Exit: dla_program_operation status=0 [ 2204.083300] Enter: dla_enable_operation [ 2204.088345] Enable PDP operation index 0 ROI 0 [ 2204.094011] group id 0 rdma id 0 [ 2204.098465] rdma needed 1 [ 2204.102303] Enter: dla_op_enabled [ 2204.106852] Exit: dla_op_enabled [ 2204.111328] Exit: dla_enable_operation status=0 [ 2204.117120] Exit: dla_submit_operation [ 2204.122117] Enter: dla_dequeue_operation [ 2204.127298] exit PDP as there's no further operation [ 2204.133541] Exit: dla_dequeue_operation [ 2204.138656] Exit: dla_initiate_processors status=0 [ 2204.144736] Enter:dla_handle_events, processor:BDMA [ 2204.150906] Exit:dla_handle_events, ret:0 [ 2204.156214] Enter:dla_handle_events, processor:Convolution [ 2204.163018] Exit:dla_handle_events, ret:0 [ 2204.168345] Enter:dla_handle_events, processor:SDP [ 2204.174456] Exit:dla_handle_events, ret:0 [ 2204.179780] Enter:dla_handle_events, processor:PDP [ 2204.185895] Handle op complete event, processor PDP group 0 [ 2204.192810] Enter:dla_op_completion processor PDP group0 [ 2204.199460] Completed PDP operation index 0 ROI 0 [ 2204.205495] 1 HWLs done, totally 1 layers [ 2204.210842] Enter: dla_free_op_desc op desc index 0 ROI 0 [ 2204.217604] Exit: dla_free_op_desc [ 2204.222317] Exit:dla_op_completion processor PDP group0 status=0 [ 2204.229622] Exit:dla_handle_events, ret:0 [ 2204.234921] Enter:dla_handle_events, processor:CDP [ 2204.241019] Exit:dla_handle_events, ret:0 [ 2204.246320] Enter:dla_handle_events, processor:RUBIK [ 2204.252580] Exit:dla_handle_events, ret:0 [ 2204.257885] reset engine done Shutdown signal received, exiting Test pass = Run CONV/CONV_D_L0_0_small_fbuf creating new runtime context... Emulator starting submitting tasks... [ 2204.972905] Enter:dla_read_network_config [ 2204.979956] Exit:dla_read_network_config status=0 [ 2204.985960] Enter: dla_initiate_processors [ 2204.991362] Enter: dla_submit_operation [ 2204.996483] Prepare Convolution operation index 0 ROI 0 dep_count 1 [ 2205.004059] Enter: dla_prepare_operation [ 2205.009298] processor:Convolution group:0, rdma_group:0 available [ 2205.016755] Enter: dla_read_config [ 2205.021543] Exit: dla_read_config [ 2205.026252] Exit: dla_prepare_operation status=0 [ 2205.032293] Enter: dla_program_operation [ 2205.037660] Program Convolution operation index 0 ROI 0 Group[0] [ 2205.045189] no desc get due to index==-1 [ 2205.050614] no desc get due to index==-1 [ 2205.056025] no desc get due to index==-1 [ 2205.061410] no desc get due to index==-1 [ 2205.066776] no desc get due to index==-1 [ 2205.072123] no desc get due to index==-1 [ 2205.077441] Enter: dla_op_programmed [ 2205.082420] Update dependency operation index 1 ROI 0 DEP_COUNT=1 [ 2205.089955] enable SDP in dla_update_dependency as depdency are resolved [ 2205.098134] Enter: dla_enable_operation [ 2205.103446] exit dla_enable_operation without actual enable due to processor hasn't been programmed [ 2205.114066] Exit: dla_enable_operation status=0 [ 2205.120169] Exit: dla_op_programmed [ 2205.125213] Exit: dla_program_operation status=0 [ 2205.131382] Exit: dla_submit_operation [ 2205.136679] Enter: dla_dequeue_operation [ 2205.142152] exit Convolution as there's no further operation [ 2205.149378] Exit: dla_dequeue_operation [ 2205.154765] Enter: dla_submit_operation [ 2205.160127] Prepare SDP operation index 1 ROI 0 dep_count 0 [ 2205.167248] Enter: dla_prepare_operation [ 2205.172713] processor:SDP group:0, rdma_group:0 available [ 2205.179672] Enter: dla_read_config [ 2205.184617] Exit: dla_read_config [ 2205.189457] Exit: dla_prepare_operation status=0 [ 2205.195604] Enter: dla_program_operation [ 2205.201044] Program SDP operation index 1 ROI 0 Group[0] [ 2205.207893] no desc get due to index==-1 [ 2205.213331] no desc get due to index==-1 [ 2205.218741] no desc get due to index==-1 [ 2205.224111] no desc get due to index==-1 [ 2205.229459] no desc get due to index==-1 [ 2205.234777] no desc get due to index==-1 [ 2205.240067] Enter: dla_op_programmed [ 2205.245002] Exit: dla_op_programmed [ 2205.249840] Exit: dla_program_operation status=0 [ 2205.255817] Enter: dla_enable_operation [ 2205.261000] Enable SDP operation index 1 ROI 0 [ 2205.266789] Enter: dla_op_enabled [ 2205.271449] Update dependency operation index 0 ROI 0 DEP_COUNT=1 [ 2205.278915] enable Convolution in dla_update_dependency as depdency are resolved [ 2205.287701] Enter: dla_enable_operation [ 2205.292905] Enable Convolution operation index 0 ROI 0 [ 2205.299410] Enter: dla_op_enabled [ 2205.304091] Exit: dla_op_enabled [ 2205.308679] Exit: dla_enable_operation status=0 [ 2205.314579] Exit: dla_op_enabled [ 2205.319140] Exit: dla_enable_operation status=0 [ 2205.324998] Exit: dla_submit_operation [ 2205.330071] Enter: dla_dequeue_operation [ 2205.335348] exit SDP as there's no further operation [ 2205.341703] Exit: dla_dequeue_operation [ 2205.346946] Exit: dla_initiate_processors status=0 [ 2205.353176] Enter:dla_handle_events, processor:BDMA [ 2205.359497] Exit:dla_handle_events, ret:0 [ 2205.364928] Enter:dla_handle_events, processor:Convolution [ 2205.371832] Handle cdma weight done event, processor Convolution group 0 [ 2205.379972] Handle cdma data done event, processor Convolution group 0 [ 2205.387900] Handle op complete event, processor Convolution group 0 [ 2205.395541] Enter:dla_op_completion processor Convolution group0 [ 2205.402946] Completed Convolution operation index 0 ROI 0 [ 2205.409760] 1 HWLs done, totally 2 layers [ 2205.415184] Enter: dla_dequeue_operation [ 2205.420521] exit Convolution as there's no further operation [ 2205.427602] Exit: dla_dequeue_operation [ 2205.432852] Exit:dla_op_completion processor Convolution group0 status=0 [ 2205.441019] Exit:dla_handle_events, ret:0 [ 2205.446477] Enter:dla_handle_events, processor:SDP [ 2205.452709] Handle op complete event, processor SDP group 0 [ 2205.459728] Enter:dla_op_completion processor SDP group0 [ 2205.466488] Completed SDP operation index 1 ROI 0 [ 2205.472644] 2 HWLs done, totally 2 layers [ 2205.478115] Enter: dla_free_op_desc op desc index 0 ROI 0 [ 2205.484995] Exit: dla_free_op_desc [ 2205.489851] Enter: dla_free_op_desc op desc index 1 ROI 0 [ 2205.496730] Exit: dla_free_op_desc [ 2205.501621] Exit:dla_op_completion processor SDP group0 status=0 [ 2205.509181] Exit:dla_handle_events, ret:0 [ 2205.514755] Enter:dla_handle_events, processor:PDP [ 2205.521145] Exit:dla_handle_events, ret:0 [ 2205.526770] Enter:dla_handle_events, processor:CDP [ 2205.533193] Exit:dla_handle_events, ret:0 [ 2205.538838] Enter:dla_handle_events, processor:RUBIK [ 2205.545453] Exit:dla_handle_events, ret:0 [ 2205.551129] reset engine done Shutdown signal received, exiting Test pass = Run SDP/SDP_X1_L0_0_small_fbuf creating new runtime context... Emulator starting submitting tasks... [ 2205.982808] Enter:dla_read_network_config [ 2205.990245] Exit:dla_read_network_config status=0 [ 2205.996613] Enter: dla_initiate_processors [ 2206.002346] Enter: dla_submit_operation [ 2206.007775] Prepare SDP operation index 0 ROI 0 dep_count 0 [ 2206.014970] Enter: dla_prepare_operation [ 2206.020497] processor:SDP group:1, rdma_group:0 available [ 2206.027530] Enter: dla_read_config [ 2206.032567] Exit: dla_read_config [ 2206.037508] Exit: dla_prepare_operation status=0 [ 2206.043769] Enter: dla_program_operation [ 2206.049343] Program SDP operation index 0 ROI 0 Group[1] [ 2206.056343] no desc get due to index==-1 [ 2206.061925] no desc get due to index==-1 [ 2206.067478] no desc get due to index==-1 [ 2206.073011] no desc get due to index==-1 [ 2206.078522] no desc get due to index==-1 [ 2206.084002] no desc get due to index==-1 [ 2206.089461] no desc get due to index==-1 [ 2206.094862] Enter: dla_op_programmed [ 2206.099906] Exit: dla_op_programmed [ 2206.104843] Exit: dla_program_operation status=0 [ 2206.110894] Enter: dla_enable_operation [ 2206.116142] Enable SDP operation index 0 ROI 0 [ 2206.121991] Enter: dla_op_enabled [ 2206.126707] Exit: dla_op_enabled [ 2206.131315] Exit: dla_enable_operation status=0 [ 2206.137221] Exit: dla_submit_operation [ 2206.142306] Enter: dla_dequeue_operation [ 2206.147549] exit SDP as there's no further operation [ 2206.153846] Exit: dla_dequeue_operation [ 2206.159005] Exit: dla_initiate_processors status=0 [ 2206.165112] Enter:dla_handle_events, processor:BDMA [ 2206.171292] Exit:dla_handle_events, ret:0 [ 2206.176594] Enter:dla_handle_events, processor:Convolution [ 2206.183393] Exit:dla_handle_events, ret:0 [ 2206.188700] Enter:dla_handle_events, processor:SDP [ 2206.194790] Handle op complete event, processor SDP group 1 [ 2206.201672] Enter:dla_op_completion processor SDP group1 [ 2206.208292] Completed SDP operation index 0 ROI 0 [ 2206.214291] 1 HWLs done, totally 1 layers [ 2206.219588] Enter: dla_free_op_desc op desc index 0 ROI 0 [ 2206.226297] Exit: dla_free_op_desc [ 2206.230992] Exit:dla_op_completion processor SDP group1 status=0 [ 2206.238319] Exit:dla_handle_events, ret:0 [ 2206.243633] Enter:dla_handle_events, processor:PDP [ 2206.249734] Exit:dla_handle_events, ret:0 [ 2206.255054] Enter:dla_handle_events, processor:CDP [ 2206.261168] Exit:dla_handle_events, ret:0 [ 2206.266494] Enter:dla_handle_events, processor:RUBIK [ 2206.272787] Exit:dla_handle_events, ret:0 [ 2206.278142] reset engine done Shutdown signal received, exiting Test pass = Run CDP/CDP_L0_0_small_fbuf creating new runtime context... Emulator starting submitting tasks... [ 2206.992531] Enter:dla_read_network_config [ 2206.999607] Exit:dla_read_network_config status=0 [ 2207.005635] Enter: dla_initiate_processors [ 2207.011044] Enter: dla_submit_operation [ 2207.016189] Prepare CDP operation index 0 ROI 0 dep_count 0 [ 2207.023096] Enter: dla_prepare_operation [ 2207.028344] processor:CDP group:0, rdma_group:0 available [ 2207.035090] Enter: dla_read_config [ 2207.039828] Exit: dla_read_config [ 2207.044490] Exit: dla_prepare_operation status=0 [ 2207.050480] Enter: dla_program_operation [ 2207.055798] Program CDP operation index 0 ROI 0 Group[0] [ 2207.062547] Enter: dla_cdp_program [ 2207.062550] Enter: processor_cdp_program [ 2207.072897] Exit: processor_cdp_program [ 2207.072899] Exit: dla_cdp_program [ 2207.078202] no desc get due to index==-1 [ 2207.088368] no desc get due to index==-1 [ 2207.093737] no desc get due to index==-1 [ 2207.099082] no desc get due to index==-1 [ 2207.104408] no desc get due to index==-1 [ 2207.109715] no desc get due to index==-1 [ 2207.114999] no desc get due to index==-1 [ 2207.120285] Enter: dla_op_programmed [ 2207.125218] Exit: dla_op_programmed [ 2207.130051] Exit: dla_program_operation status=0 [ 2207.136011] Enter: dla_enable_operation [ 2207.141183] Enable CDP operation index 0 ROI 0 [ 2207.146963] Enter: dla_cdp_enable [ 2207.151594] Exit: dla_cdp_enable [ 2207.156124] Enter: dla_op_enabled [ 2207.160724] Exit: dla_op_enabled [ 2207.165223] Exit: dla_enable_operation status=0 [ 2207.171013] Exit: dla_submit_operation [ 2207.175998] Enter: dla_dequeue_operation [ 2207.181144] exit CDP as there's no further operation [ 2207.187347] Exit: dla_dequeue_operation [ 2207.192408] Exit: dla_initiate_processors status=0 [ 2207.198421] Enter:dla_handle_events, processor:BDMA [ 2207.204503] Exit:dla_handle_events, ret:0 [ 2207.209712] Enter:dla_handle_events, processor:Convolution [ 2207.216411] Exit:dla_handle_events, ret:0 [ 2207.221622] Enter:dla_handle_events, processor:SDP [ 2207.227627] Exit:dla_handle_events, ret:0 [ 2207.232836] Enter:dla_handle_events, processor:PDP [ 2207.238832] Exit:dla_handle_events, ret:0 [ 2207.244039] Enter:dla_handle_events, processor:CDP [ 2207.250044] Handle op complete event, processor CDP group 0 [ 2207.256850] Enter:dla_op_completion processor CDP group0 [ 2207.263402] Completed CDP operation index 0 ROI 0 [ 2207.269347] 1 HWLs done, totally 1 layers [ 2207.274594] Enter: dla_free_op_desc op desc index 0 ROI 0 [ 2207.281245] Exit: dla_free_op_desc [ 2207.285878] Exit:dla_op_completion processor CDP group0 status=0 [ 2207.293139] Exit:dla_handle_events, ret:0 [ 2207.298381] Enter:dla_handle_events, processor:RUBIK [ 2207.304601] Exit:dla_handle_events, ret:0 [ 2207.309883] reset engine done Shutdown signal received, exiting Test pass = Run NN/NN_L0_1_small_fbuf creating new runtime context... Emulator starting submitting tasks... [ 2210.896668] Enter:dla_read_network_config [ 2210.903672] Exit:dla_read_network_config status=0 [ 2210.909643] Enter: dla_initiate_processors [ 2210.915004] Enter: dla_submit_operation [ 2210.920099] Prepare Convolution operation index 0 ROI 0 dep_count 1 [ 2210.927667] Enter: dla_prepare_operation [ 2210.932896] processor:Convolution group:1, rdma_group:0 available [ 2210.940346] Enter: dla_read_config [ 2210.945125] Exit: dla_read_config [ 2210.949831] Exit: dla_prepare_operation status=0 [ 2210.955870] Enter: dla_program_operation [ 2210.961236] Program Convolution operation index 0 ROI 0 Group[1] [ 2210.968764] no desc get due to index==-1 [ 2210.974179] no desc get due to index==-1 [ 2210.979577] no desc get due to index==-1 [ 2210.984954] no desc get due to index==-1 [ 2210.990313] no desc get due to index==-1 [ 2210.995644] Enter: dla_op_programmed [ 2211.000630] Update dependency operation index 2 ROI 0 DEP_COUNT=2 [ 2211.008178] Update dependency operation index 1 ROI 0 DEP_COUNT=1 [ 2211.015723] enable SDP in dla_update_dependency as depdency are resolved [ 2211.023905] Enter: dla_enable_operation [ 2211.029220] exit dla_enable_operation without actual enable due to processor hasn't been programmed [ 2211.039839] Exit: dla_enable_operation status=0 [ 2211.045950] Exit: dla_op_programmed [ 2211.050997] Exit: dla_program_operation status=0 [ 2211.057176] Exit: dla_submit_operation [ 2211.062484] Enter: dla_dequeue_operation [ 2211.067953] Dequeue op from Convolution processor, index=2 ROI=0 [ 2211.075536] Enter: dla_submit_operation [ 2211.080938] Prepare Convolution operation index 2 ROI 0 dep_count 1 [ 2211.088792] Enter: dla_prepare_operation [ 2211.094292] processor:Convolution group:0, rdma_group:0 available [ 2211.101991] Enter: dla_read_config [ 2211.106995] Exit: dla_read_config [ 2211.111900] Exit: dla_prepare_operation status=0 [ 2211.118105] Enter: dla_program_operation [ 2211.123600] Program Convolution operation index 2 ROI 0 Group[0] [ 2211.131226] no desc get due to index==-1 [ 2211.136737] no desc get due to index==-1 [ 2211.142218] no desc get due to index==-1 [ 2211.147668] no desc get due to index==-1 [ 2211.153083] no desc get due to index==-1 [ 2211.158471] Enter: dla_op_programmed [ 2211.163499] Update dependency operation index 6 ROI 0 DEP_COUNT=3 [ 2211.171072] Update dependency operation index 3 ROI 0 DEP_COUNT=2 [ 2211.178616] Exit: dla_op_programmed [ 2211.183552] Exit: dla_program_operation status=0 [ 2211.189625] Exit: dla_submit_operation [ 2211.194807] Exit: dla_dequeue_operation [ 2211.200064] Enter: dla_submit_operation [ 2211.205327] Prepare SDP operation index 1 ROI 0 dep_count 0 [ 2211.212329] Enter: dla_prepare_operation [ 2211.217670] processor:SDP group:0, rdma_group:1 available [ 2211.224484] Enter: dla_read_config [ 2211.229290] Exit: dla_read_config [ 2211.233999] Exit: dla_prepare_operation status=0 [ 2211.240023] Enter: dla_program_operation [ 2211.245346] Program SDP operation index 1 ROI 0 Group[0] [ 2211.252076] no desc get due to index==-1 [ 2211.257384] no desc get due to index==-1 [ 2211.262694] no desc get due to index==-1 [ 2211.268009] no desc get due to index==-1 [ 2211.273323] Enter: dla_op_programmed [ 2211.278305] Update dependency operation index 3 ROI 0 DEP_COUNT=1 [ 2211.285843] enable SDP in dla_update_dependency as depdency are resolved [ 2211.293989] Enter: dla_enable_operation [ 2211.299287] exit dla_enable_operation without actual enable due to processor hasn't been programmed [ 2211.309866] Exit: dla_enable_operation status=0 [ 2211.315901] Exit: dla_op_programmed [ 2211.320850] Exit: dla_program_operation status=0 [ 2211.326893] Enter: dla_enable_operation [ 2211.332148] Enable SDP operation index 1 ROI 0 [ 2211.338022] Enter: dla_op_enabled [ 2211.342757] Update dependency operation index 0 ROI 0 DEP_COUNT=1 [ 2211.350310] enable Convolution in dla_update_dependency as depdency are resolved [ 2211.359193] Enter: dla_enable_operation [ 2211.364502] Enable Convolution operation index 0 ROI 0 [ 2211.371107] Enter: dla_op_enabled [ 2211.375865] Exit: dla_op_enabled [ 2211.380508] Exit: dla_enable_operation status=0 [ 2211.386447] Exit: dla_op_enabled [ 2211.391083] Exit: dla_enable_operation status=0 [ 2211.397023] Exit: dla_submit_operation [ 2211.402167] Enter: dla_dequeue_operation [ 2211.407468] Dequeue op from SDP processor, index=3 ROI=0 [ 2211.414162] Enter: dla_submit_operation [ 2211.419400] Prepare SDP operation index 3 ROI 0 dep_count 0 [ 2211.426425] Enter: dla_prepare_operation [ 2211.431824] processor:SDP group:1, rdma_group:0 available [ 2211.438743] Enter: dla_read_config [ 2211.443667] Exit: dla_read_config [ 2211.448470] Exit: dla_prepare_operation status=0 [ 2211.454603] Enter: dla_program_operation [ 2211.460027] Program SDP operation index 3 ROI 0 Group[1] [ 2211.466866] no desc get due to index==-1 [ 2211.472307] no desc get due to index==-1 [ 2211.477739] no desc get due to index==-1 [ 2211.483155] no desc get due to index==-1 [ 2211.488544] Enter: dla_op_programmed [ 2211.493573] Update dependency operation index 7 ROI 0 DEP_COUNT=2 [ 2211.501145] Exit: dla_op_programmed [ 2211.506089] Exit: dla_program_operation status=0 [ 2211.512179] Enter: dla_enable_operation [ 2211.517476] Enable SDP operation index 3 ROI 0 [ 2211.523382] Enter: dla_op_enabled [ 2211.528145] Update dependency operation index 2 ROI 0 DEP_COUNT=1 [ 2211.535711] enable Convolution in dla_update_dependency as depdency are resolved [ 2211.544641] Enter: dla_enable_operation [ 2211.550034] Enable Convolution operation index 2 ROI 0 [ 2211.556769] Enter: dla_op_enabled [ 2211.561671] Exit: dla_op_enabled [ 2211.566466] Exit: dla_enable_operation status=0 [ 2211.572550] Exit: dla_op_enabled [ 2211.577296] Exit: dla_enable_operation status=0 [ 2211.583307] Exit: dla_submit_operation [ 2211.588484] Exit: dla_dequeue_operation [ 2211.593743] Enter: dla_submit_operation [ 2211.598996] Prepare PDP operation index 5 ROI 0 dep_count 1 [ 2211.606013] Enter: dla_prepare_operation [ 2211.611383] processor:PDP group:1, rdma_group:1 available [ 2211.618250] Enter: dla_read_config [ 2211.623101] Exit: dla_read_config [ 2211.627832] Exit: dla_prepare_operation status=0 [ 2211.633842] Enter: dla_program_operation [ 2211.639146] Program PDP operation index 5 ROI 0 Group[1] [ 2211.645852] group id 1 rdma id 1 [ 2211.650493] no desc get due to index==-1 [ 2211.655813] no desc get due to index==-1 [ 2211.661119] no desc get due to index==-1 [ 2211.666410] no desc get due to index==-1 [ 2211.671689] no desc get due to index==-1 [ 2211.676943] Enter: dla_op_programmed [ 2211.681837] Update dependency operation index 11 ROI 0 DEP_COUNT=2 [ 2211.689363] Exit: dla_op_programmed [ 2211.694182] Exit: dla_program_operation status=0 [ 2211.700123] Exit: dla_submit_operation [ 2211.705173] Enter: dla_dequeue_operation [ 2211.710387] Dequeue op from PDP processor, index=11 ROI=0 [ 2211.717109] Enter: dla_submit_operation [ 2211.722266] Prepare PDP operation index 11 ROI 0 dep_count 1 [ 2211.729268] Enter: dla_prepare_operation [ 2211.734523] processor:PDP group:0, rdma_group:0 available [ 2211.741271] Enter: dla_read_config [ 2211.746045] Exit: dla_read_config [ 2211.750733] Exit: dla_prepare_operation status=0 [ 2211.756763] Enter: dla_program_operation [ 2211.762096] Program PDP operation index 11 ROI 0 Group[0] [ 2211.768910] group id 0 rdma id 0 [ 2211.773538] no desc get due to index==-1 [ 2211.778850] no desc get due to index==-1 [ 2211.784149] no desc get due to index==-1 [ 2211.789427] no desc get due to index==-1 [ 2211.794682] no desc get due to index==-1 [ 2211.799906] Enter: dla_op_programmed [ 2211.804748] Update dependency operation index 22 ROI 0 DEP_COUNT=2 [ 2211.812207] Exit: dla_op_programmed [ 2211.816961] Exit: dla_program_operation status=0 [ 2211.822846] Exit: dla_submit_operation [ 2211.827860] Exit: dla_dequeue_operation [ 2211.832958] Enter: dla_submit_operation [ 2211.838041] Prepare CDP operation index 4 ROI 0 dep_count 2 [ 2211.844894] Enter: dla_prepare_operation [ 2211.850106] processor:CDP group:1, rdma_group:1 available [ 2211.856828] Enter: dla_read_config [ 2211.861562] Exit: dla_read_config [ 2211.866189] Exit: dla_prepare_operation status=0 [ 2211.872116] Enter: dla_program_operation [ 2211.877340] Program CDP operation index 4 ROI 0 Group[1] [ 2211.883957] Enter: dla_cdp_program [ 2211.883960] Enter: processor_cdp_program [ 2211.893984] Exit: processor_cdp_program [ 2211.893985] Exit: dla_cdp_program [ 2211.899117] no desc get due to index==-1 [ 2211.908890] no desc get due to index==-1 [ 2211.914053] no desc get due to index==-1 [ 2211.919231] no desc get due to index==-1 [ 2211.924404] no desc get due to index==-1 [ 2211.929580] Enter: dla_op_programmed [ 2211.934423] Update dependency operation index 10 ROI 0 DEP_COUNT=3 [ 2211.941907] Exit: dla_op_programmed [ 2211.946681] Exit: dla_program_operation status=0 [ 2211.952573] Exit: dla_submit_operation [ 2211.957589] Enter: dla_dequeue_operation [ 2211.962782] Dequeue op from CDP processor, index=10 ROI=0 [ 2211.969474] Enter: dla_submit_operation [ 2211.974592] Prepare CDP operation index 10 ROI 0 dep_count 2 [ 2211.981545] Enter: dla_prepare_operation [ 2211.986748] processor:CDP group:0, rdma_group:0 available [ 2211.993436] Enter: dla_read_config [ 2211.998119] Exit: dla_read_config [ 2212.002687] Exit: dla_prepare_operation status=0 [ 2212.008579] Exit: dla_submit_operation [ 2212.013591] Exit: dla_dequeue_operation [ 2212.018685] Exit: dla_initiate_processors status=0 [ 2212.024728] Enter:dla_handle_events, processor:BDMA [ 2212.030851] Exit:dla_handle_events, ret:0 [ 2212.036104] Enter:dla_handle_events, processor:Convolution [ 2212.042876] Exit:dla_handle_events, ret:0 [ 2212.048188] Enter:dla_handle_events, processor:SDP [ 2212.054317] Exit:dla_handle_events, ret:0 [ 2212.059688] Enter:dla_handle_events, processor:PDP [ 2212.065855] Exit:dla_handle_events, ret:0 [ 2212.071217] Enter:dla_handle_events, processor:CDP [ 2212.077357] Exit:dla_handle_events, ret:0 [ 2212.082717] Enter:dla_handle_events, processor:RUBIK [ 2212.089054] Exit:dla_handle_events, ret:0 [ 2212.094454] Enter:dla_handle_events, processor:BDMA [ 2212.100733] Exit:dla_handle_events, ret:0 [ 2212.106130] Enter:dla_handle_events, processor:Convolution [ 2212.113015] Exit:dla_handle_events, ret:0 [ 2212.118409] Enter:dla_handle_events, processor:SDP [ 2212.124584] Exit:dla_handle_events, ret:0 [ 2212.129985] Enter:dla_handle_events, processor:PDP [ 2212.136181] Exit:dla_handle_events, ret:0 [ 2212.141594] Enter:dla_handle_events, processor:CDP [ 2212.147788] Exit:dla_handle_events, ret:0 [ 2212.153183] Enter:dla_handle_events, processor:RUBIK [ 2212.159561] Exit:dla_handle_events, ret:0 一直停在这是什么原因，退也退不出来，怎么办。您有推荐的解决办法吗？

算是出过类似的，主要修改了硬件工程那边的clock gating，其他不记得了。

VVViy commented 5 years ago

Which modules? :sweat: Sorry, that's long time. In terms of "how", i have descriped in detail in previous blog about vivado project.

VVViy commented 5 years ago

造成无法运行的原因可能很多，可能手动修改出错，可能配置出错，无法具体说是哪一块出了问题。假定你所作修改全部正确，我不确定的地方只有一处，关闭clock gating中的SYNTHESIS宏是我后来添加的，之前建工程时并没加它，但后来分析源码时，发现不加它会跑很多仿真测试，而且我记得之前有人提过要加这个宏，但是我并没测试过，所以是我唯一不确定的地方。

VVViy commented 5 years ago

@peterzh2018888

@VVViy 造成无法运行的原因可能很多，可能手动修改出错，可能配置出错，无法具体说是哪一块出了问题。假定你所作修改全部正确，我不确定的地方只有一处，关闭clock gating中的SYNTHESIS宏是我后来添加的，之前建工程时并没加它，但后来分析源码时，发现不加它会跑很多仿真测试，而且我记得之前有人提过要加这个宏，但是我并没测试过，所以是我唯一不确定的地方。 @VVViy

VLIB_BYPASS_POWER_CG NV_FPGA_FIFOGEN FIFOGEN_MASTER_CLK_GATING_DISABLED FPGA SYNTHESIS

这几个我都加了，现在。

我不确定是不是SYNTHESIS宏导致的问题，你再检查一下吧，如果没有其他问题，就把这个宏去掉试试。

VVViy commented 5 years ago

@peterzh2018888

@peterzh2018888

@VVViy 造成无法运行的原因可能很多，可能手动修改出错，可能配置出错，无法具体说是哪一块出了问题。假定你所作修改全部正确，我不确定的地方只有一处，关闭clock gating中的SYNTHESIS宏是我后来添加的，之前建工程时并没加它，但后来分析源码时，发现不加它会跑很多仿真测试，而且我记得之前有人提过要加这个宏，但是我并没测试过，所以是我唯一不确定的地方。 @VVViy

VLIB_BYPASS_POWER_CG NV_FPGA_FIFOGEN FIFOGEN_MASTER_CLK_GATING_DISABLED FPGA SYNTHESIS 这几个我都加了，现在。

我不确定是不是SYNTHESIS宏导致的问题，你再检查一下吧，如果没有其他问题，就把这个宏去掉试试。

SYNTHESIS宏应该是要加的吧，不加好多是仿真。我头文件的宏定义总共有下面几个是不是太多了。

`define CC_ATOMC_DIV_ATOMK_EQUAL_1

`define XSDB_SLV_DIS

`define FPGA

`define SYNTHESIS

`define DESIGNWARE_NOEXIST

`define VLIB_BYPASS_POWER_CG

`define NV_FPGA_SYSTEM

`define NV_FPGA_FIFOGEN

`define NV_FPGA_UNIT

我没用到这么多，我就用到了前一篇博客中提到的那些，就是与interrupt和资源类型相关的宏。另外define DESIGNWARE_NOEXIST这个宏定义在你头文件中就是这么定义的？如果是的话，显然在编译工程时有问题，如果没使用dc designware的话，正确形式应该是define DESIGNWARE_NOEXIST 1

honorpeter commented 5 years ago

define DESIGNWARE_NOEXIST 1 是在vivado里的verilog的头文件里加吗，我加上现象一样。

root@aidla:~# insmod /lib/modules/4.14.0-xilinx-v2018.2/extra/opendla.ko [ 57.071181] reset engine done [ 57.074477] [drm] Initialized nvdla 0.0.0 20171017 for a0000000.nvdla_small on minor 1 root@aidla:~# mount /dev/mmcblk0p3 /media/card mount: /media/card: mount point does not exist. root@aidla:~# cd /media/card -sh: cd: /media/card: No such file or directory root@aidla:~# cd /media root@aidla:/media# ls root@aidla:/media# mkdir card root@aidla:/media# l -sh: l: command not found root@aidla:/media# ls card root@aidla:/media# mount /dev/mmcblk0p3 /media/card root@aidla:/media# cd /media/card/umd root@aidla:/media/card/umd# export TOP=/media/card/umd root@aidla:/media/card/umd# make TOOLCHAIN_PREFIX=/usr/bin/ for dir in core/runtime tests/runtime; do \ make -C $dir; \ done make[1]: Entering directory '/media/card/umd/core/runtime' MODULE_CC /usr/bin/gcc make[1]: Nothing to be done for 'all'. make[1]: Leaving directory '/media/card/umd/core/runtime' make[1]: Entering directory '/media/card/umd/tests/runtime' MODULE_CC /usr/bin/gcc make[1]: Nothing to be done for 'all'. make[1]: Leaving directory '/media/card/umd/tests/runtime' root@aidla:/media/card/umd# export LD_LIBRARY_PATH=/media/card/umd/out/runtime/libnvdla_runtime/ root@aidla:/media/card/umd# dmesg -n 1 root@aidla:/media/card/umd# cd .. root@aidla:/media/card# dmesg -n 1 root@aidla:/media/card# su root ./run_test.sh == Tests for nv_small ==

= Run PDP/PDP_L0_0_small_fbuf creating new runtime context... Emulator starting submitting tasks... Shutdown signal received, exiting Test pass = Run CONV/CONV_D_L0_0_small_fbuf creating new runtime context... Emulator starting submitting tasks... Shutdown signal received, exiting Test pass = Run SDP/SDP_X1_L0_0_small_fbuf creating new runtime context... Emulator starting submitting tasks... Shutdown signal received, exiting Test pass = Run CDP/CDP_L0_0_small_fbuf creating new runtime context... Emulator starting submitting tasks... Shutdown signal received, exiting Test pass = Run NN/NN_L0_1_small_fbuf creating new runtime context... Emulator starting submitting tasks...

can't continue to run.

是不是我工程配置的地址还是不对？

Cell Slave Interface Base Name Offset Address Range High Address  
zynq_ultra_ps_e_0  
Data (40 address bits : 0x00A0000000 [ 256M ] ,0x0400000000 [ 4G ] ,0x1000000000 [ 224G ] ,0x00B0000000 [ 256M ] ,0x0500000000 [ 4G ] ,0x4800000000 [ 224G ])  
axi_gpio_0 S_AXI Reg 0x00_B000_0000 4K 0x00_B000_0FFF  
nvdla_small_0 apb_1 Reg 0x00_A000_0000 64K 0x00_A000_FFFF  
nvdla_small_0  
M_AXI (64 address bits : 16E)  
**zynq_ultra_ps_e_0 S_AXI_HP0_FPD HP0_DDR_LOW 0x0000_0000_4000_0000 1G 0x0000_0000_7FFF_FFFF **
zynq_ultra_ps_e_0 S_AXI_HP0_FPD HP0_LPS_OCM 0x0000_0000_FF00_0000 16M 0x0000_0000_FFFF_FFFF  
zynq_ultra_ps_e_0 S_AXI_HP0_FPD HP0_PCIE_LOW 0x0000_0000_E000_0000 256M 0x0000_0000_EFFF_FFFF  
zynq_ultra_ps_e_0 S_AXI_HP0_FPD HP0_QSPI 0x0000_0000_C000_0000 512M 0x0000_0000_DFFF_FFFF  

或者IP里的数据格式不对？因为我多了一个：`define CC_ATOMC_DIV_ATOMK_EQUAL_1 是不是数据只能是八位？和NN_L0_1_small_fbuf不匹配？

您邮箱多少？我可以把我的整个工程含IP代码发给您，能帮我看下吗？

VVViy commented 5 years ago

前一篇blog你看了吗？第一步不就是在linux下先编译vmod工程吗，designware宏就是在那里自动处理的。 别发我了，我没时间啊。看过这两篇blog之后跑通工程的，我知道的至少两个，所以按照流程自检即可。

honorpeter commented 5 years ago

@VVViy 是的，我是这样做的，但是有一些代码是从sifive里面的block里加进来的，可能有问题。所以我不加designware编译会出问题。耽误您了实在不好意思，我这几天一直在试NN不能继续往下运行这个问题，非常着急，因此发您的有点多，打扰了。

VVViy commented 5 years ago

@peterzh2018 @VVViy 是的，我是这样做的，但是有一些代码是从sifive里面的block里加进来的，可能有问题。所以我不加designware编译会出问题。耽误您了实在不好意思，我这几天一直在试NN不能继续往下运行这个问题，非常着急，因此发您的有点多，打扰了。

打扰谈不上，但给我整个工程做review，这活太大，没那么多时间。如果你整合了sifive rv mcu+nvdla工程内容，那你应该去那边的issue求助，你这种合并的与我这里自编译工程的肯定有差异。

honorpeter commented 5 years ago

@VVViy

@peterzh2018 @VVViy 是的，我是这样做的，但是有一些代码是从sifive里面的block里加进来的，可能有问题。所以我不加designware编译会出问题。耽误您了实在不好意思，我这几天一直在试NN不能继续往下运行这个问题，非常着急，因此发您的有点多，打扰了。

打扰谈不上，但给我整个工程做review，这活太大，没那么多时间。如果你整合了sifive rv mcu+nvdla工程内容，那你应该去那边的issue求助，你这种合并的与我这里自编译工程的肯定有差异。

好的，谢谢 。

honorpeter commented 5 years ago

@VVViy Thank you for your kind help. I have suceed. But cant input real image with NN/NN_L0_1_small_fbuf. And I see this issue https://github.com/nvdla/sw/issues/110#issuecomment-465823181. Is there any other way?

VVViy commented 5 years ago

@honorpeter @VVViy Thank you for your kind help. I have suceed. But cant input real image with NN/NN_L0_1_small_fbuf. And I see this issue https://github.com/nvdla/sw/issues/110#issuecomment-465823181. Is there any other way?

Cool. No, i did't get alternative.

gitosu67 commented 4 years ago

@VVViy Hi, In Step-4, point-3,I appended to my Makefile in nvdla/sw/kmd/ the following:

_opendla-m := opendla.o

append all of sources

opendla-objs := nvdla_core_callbacks.o nvdla_gem.o scheduler.o engine.o bdma.o conv.o sdp.o cdp.o pdp.o rubik.o cache.o common.o engine_data.o engine_isr.o enginedebug.o ########################### ...

So, now it looks this: Screenshot from 2019-09-16 21-44-36

Is this Makefile Correct?

In Step-5, which file do I need to modify: _nvdla/sw/kmd/port/linux/nvdla_corecallbacks.c OR nvdla/sw/kmd/Documentation/devicetree/bindings/nvdla/nvdla.txt? Also, my pl.dtsi file in _/component/plnxworkspace/device-tree/pl.dtsi looks like this: Screenshot from 2019-09-16 22-02-26

I don't know why it shows NVDLA_apb2csb. How to fix this? Thank you so much for your time and help!

VVViy commented 4 years ago

@Okaymaddy @VVViy Hi, In Step-4, point-3,I appended to my Makefile in nvdla/sw/kmd/ the following:

_opendla-m := opendla.o

append all of sources

opendla-objs := nvdla_core_callbacks.o nvdla_gem.o scheduler.o engine.o bdma.o conv.o sdp.o cdp.o pdp.o rubik.o cache.o common.o engine_data.o engine_isr.o enginedebug.o ########################### ...

So, now it looks this: Screenshot from 2019-09-16 21-44-36

Is this Makefile Correct?

In Step-5, which file do I need to modify: _nvdla/sw/kmd/port/linux/nvdla_corecallbacks.c OR nvdla/sw/kmd/Documentation/devicetree/bindings/nvdla/nvdla.txt? Also, my pl.dtsi file in _/component/plnxworkspace/device-tree/pl.dtsi looks like this: Screenshot from 2019-09-16 22-02-26

I don't know why it shows NVDLA_apb2csb. How to fix this? Thank you so much for your time and help!

  1. Actually, you should put kmd compile instructs to <path-to-petalinux-prj>/project-spec/meta-user/recipes-modules/opendla/files/makefile, but not nvdla/sw/kmd.
  2. For device tree, you should modify the <path-to-petalinux-prj>/project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi
  3. Why it shows NVDLA_apb2csb? I have no idea
gitosu67 commented 4 years ago

@VVViy How does your <path-to-petalinux-prj>/project-spec/meta-user/recipes-modules/opendla/files/makefile look like? I am getting some issues while building the project.

VVViy commented 4 years ago

@Okaymaddy @VVViy How does your <path-to-petalinux-prj>/project-spec/meta-user/recipes-modules/opendla/files/makefile look like? I am getting some issues while building the project.

Actually, this is my old project, some details i forgot. Maybe you can post your makefile, it will give me some help.

gitosu67 commented 4 years ago

@VVViy My makefile in <path-to-petalinux-prj>/project-spec/meta-user/recipes-modules/opendla/files/makefile looks like this:

Screenshot from 2019-09-25 15-10-06 Is this correct? Also, do I have to manually edit the opendla.bb file or will it update automatically when I build the project?

I am getting several errors while building the project after adding the opendla module. Such as these: ezgif com-crop Any suggestions on why this is happening will be very helpful.

VVViy commented 4 years ago

@Okaymaddy @VVViy My makefile in <path-to-petalinux-prj>/project-spec/meta-user/recipes-modules/opendla/files/makefile looks like this:

Screenshot from 2019-09-25 15-10-06 Is this correct? Also, do I have to manually edit the opendla.bb file or will it update automatically when I build the project?

I am getting several errors while building the project after adding the opendla module. Such as these: ezgif com-crop Any suggestions on why this is happening will be very helpful.

Sorry, pal, I did not recall whether this file is correct. I just remember put above compile instructs in makefile beginning, but not ending. You must manually rewrite opendla.bb, 'automatically' is impossible!

gitosu67 commented 4 years ago

@VVViy Thank you for your answers. I have a dumb question. How to view the boot messages from the board after I connect it to my computer(set it to boot mode as well) and turn it on?

VVViy commented 4 years ago

@Okaymaddy @VVViy Thank you for your answers. I have a dumb question. How to view the boot messages from the board after I connect it to my computer(set it to boot mode as well) and turn it on?

Boot messages? When you connect your board to computer(board boot mode is set SD card) you will see. The boot mode of your board, you need check your board documents or Google it.

gitosu67 commented 4 years ago

@VVViy In Step-8, for the command su root ./run_test.sh, how are you getting run_test.sh? Also, where can I find the libnvdla_runtime.o ?

VVViy commented 4 years ago

@Okaymaddy @VVViy In Step-8, for the command su root ./run_test.sh, how are you getting run_test.sh? Also, where can I find the libnvdla_runtime.o ?

run_test.sh is my own test script, you can also run tests in nvdla/sw/regression/flatbus/kmd one by one. Where find libnvdla_runtime.o? The easy way is that use 'find' command!

gitosu67 commented 4 years ago

@VVViy Is it okay to build the kmd without creating a module in the Petalinux project? I build the kmd separately (as given here: https://github.com/nvdla/sw) to generate the opendla.ko. Now when the FPGA boots up I'll load this module using insmod. I am not sure if this is the correct way. I get errors while building the petalinux project if I create a module. So I want to know if I can do this without creating a module.

VVViy commented 4 years ago

@Okaymaddy @VVViy Is it okay to build the kmd without creating a module in the Petalinux project? I build the kmd separately (as given here: https://github.com/nvdla/sw) to generate the opendla.ko. Now when the FPGA boots up I'll load this module using insmod. I am not sure if this is the correct way. I get errors while building the petalinux project if I create a module. So I want to know if I can do this without creating a module.

Maybe it works.

gitosu67 commented 4 years ago

Hi @VVViy , Is your pl.dtsi similar to below (specially the reg address)?

/ {
    amba_pl: amba_pl@0 {
        #address-cells = <2>;
        #size-cells = <2>;
        compatible = "simple-bus";
        ranges ;
        FPI_NVDLA_wrapper_0: FPI_NVDLA_wrapper@a0000000 {
            compatible = "xlnx,FPI-NVDLA-wrapper-1.0";
            interrupt-parent = <&gic>;
            interrupts = <0 89 4>;
            reg = <0x0 0xa0000000 0x0 0x1000>;
        };
        psu_ctrl_ipi: PERIPHERAL@ff380000 {
            compatible = "xlnx,PERIPHERAL-1.0";
            reg = <0x0 0xff380000 0x0 0x80000>;
        };
        psu_message_buffers: PERIPHERAL@ff990000 {
            compatible = "xlnx,PERIPHERAL-1.0";
            reg = <0x0 0xff990000 0x0 0x10000>;
        };
    };
};
VVViy commented 4 years ago

@Okaymaddy Hi @VVViy , Is your pl.dtsi similar to below (specially the reg address)?

/ {
  amba_pl: amba_pl@0 {
      #address-cells = <2>;
      #size-cells = <2>;
      compatible = "simple-bus";
      ranges ;
      FPI_NVDLA_wrapper_0: FPI_NVDLA_wrapper@a0000000 {
          compatible = "xlnx,FPI-NVDLA-wrapper-1.0";
          interrupt-parent = <&gic>;
          interrupts = <0 89 4>;
          reg = <0x0 0xa0000000 0x0 0x1000>;
      };
      psu_ctrl_ipi: PERIPHERAL@ff380000 {
          compatible = "xlnx,PERIPHERAL-1.0";
          reg = <0x0 0xff380000 0x0 0x80000>;
      };
      psu_message_buffers: PERIPHERAL@ff990000 {
          compatible = "xlnx,PERIPHERAL-1.0";
          reg = <0x0 0xff990000 0x0 0x10000>;
      };
  };
};
VVViy commented 4 years ago

Sorry, it was too long time, i did not remember such detail.

gitosu67 commented 4 years ago

@VVViy Can you please tell me how you solved this issue that you faced before: https://github.com/nvdla/hw/issues/110#issuecomment-419362935? I am having the same problem. Thanks!

EDIT: I am using the following macros mentioned in your blog: VLIB_BYPASS_POWER_CG NV_FPGA_FIFOGEN FIFOGEN_MASTER_CLK_GATING_DISABLED FPGA SYNTHESIS

VVViy commented 4 years ago

@Okaymaddy @VVViy Can you please tell me how you solved this issue that you faced before: https://github.com/nvdla/hw/issues/110#issuecomment-419362935? I am having the same problem. Thanks!

EDIT: I am using the following macros mentioned in your blog: VLIB_BYPASS_POWER_CG NV_FPGA_FIFOGEN FIFOGEN_MASTER_CLK_GATING_DISABLED FPGA SYNTHESIS

In my case, that is MACRO problem. Maybe you can refer to the thread nvdla/hw#183 (comment)

gitosu67 commented 4 years ago

@VVViy Thank you. I have included all the macros as you have mentioned in your blog. The flatbuff test for CDP is passing but for CONV, SDP its getting stuck on completion.

VVViy commented 4 years ago

@Okaymaddy @VVViy Thank you. I have included all the macros as you have mentioned in your blog. The flatbuff test for CDP is passing but for CONV, SDP its getting stuck on completion.

Ooh, pal, you might check your project, or post question on issue. Good luck!

LeiWang1999 commented 3 years ago

Thank you, But why we need sync after sudo dd if=rootfs.ext4 of=/dev/sdX2 ？

VVViy commented 3 years ago

@LeiWang1999 Thank you, But why we need sync after sudo dd if=rootfs.ext4 of=/dev/sdX2 ？

因为cp到SD卡上的文件块比较大，系统可能会对dd做缓冲优化，sync能够马上将缓冲中数据写入物理设备中，类似于c++写大文件时，close之前加一个flush，道理差不多。（非linux kernel专家，知之有限）

LeiWang1999 commented 3 years ago

了解，我用的是ZYNQ7045期间、发现32位的处理器似乎不能工作，本来kmd里存在一个64位除法，我解决了之后，在umd的make阶段又发现他链接的libprotobuf也是64位的，没法链接，现在已经放弃了😂。

VVViy commented 3 years ago

@LeiWang1999 了解，我用的是ZYNQ7045期间、发现32位的处理器似乎不能工作，本来kmd里存在一个64位除法，我解决了之后，在umd的make阶段又发现他链接的libprotobuf也是64位的，没法链接，现在已经放弃了😂。

上面写的东西只能做参考了，是旧版本代码的映射，官网后续更新了不少代码，编译器放出来时，想再玩一下了，奈何手上没有合适的板子了。

dovanhuong commented 3 years ago

Dear Expert, Thank for your sharing! I'm working to re-produce your result in Vivado 2020.1. However, when I go to step of porting petalinux 2020.1 it have some error like this: tony@soc-solution:~/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux$ petalinux-build INFO: sourcing build tools [INFO] building project [INFO] sourcing build environment [INFO] generating user layers [INFO] generating workspace directory INFO: bitbake petalinux-image-minimal Loading cache: 100% |########################################################################################################################################################################| Time: 0:00:00 Loaded 4230 entries from dependency cache. Parsing recipes: 100% |######################################################################################################################################################################| Time: 0:00:03 Parsing of 2962 .bb files complete (2960 cached, 2 parsed). 4231 targets, 168 skipped, 0 masked, 0 errors. NOTE: Resolving any missing task queue dependencies Initialising tasks: 100% |###################################################################################################################################################################| Time: 0:00:02 Checking sstate mirror object availability: 100% |###########################################################################################################################################| Time: 0:00:03 Sstate summary: Wanted 168 Found 9 Missed 159 Current 860 (5% match, 84% complete) NOTE: Executing Tasks NOTE: Setscene tasks completed ERROR: opendla-1.0-r0 do_compile: oe_runmake failed ERROR: opendla-1.0-r0 do_compile: Execution of '/home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work/zynqmp_generic-xilinx-linux/opendla/1.0-r0/temp/run.do_compile.21451' failed with exit code 1: make: *** No targets. Stop. WARNING: /home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work/zynqmp_generic-xilinx-linux/opendla/1.0-r0/temp/run.do_compile.21451:1 exit 1 from 'exit 1'

ERROR: Logfile of failure stored in: /home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work/zynqmp_generic-xilinx-linux/opendla/1.0-r0/temp/log.do_compile.21451 Log data follows: DEBUG: Executing shell function do_compile NOTE: make -j 32 KERNEL_SRC=/home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work-shared/zynqmp-generic/kernel-source KERNEL_PATH=/home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work-shared/zynqmp-generic/kernel-source KERNEL_VERSION=5.4.0-xilinx-v2020.1 CC=aarch64-xilinx-linux-gcc -fuse-ld=bfd -fmacro-prefix-map=/home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work/zynqmp_generic-xilinx-linux/opendla/1.0-r0=/usr/src/debug/opendla/1.0-r0 -fdebug-prefix-map=/home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work/zynqmp_generic-xilinx-linux/opendla/1.0-r0=/usr/src/debug/opendla/1.0-r0 -fdebug-prefix-map=/home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work/zynqmp_generic-xilinx-linux/opendla/1.0-r0/recipe-sysroot= -fdebug-prefix-map=/home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work/zynqmp_generic-xilinx-linux/opendla/1.0-r0/recipe-sysroot-native= -fdebug-prefix-map=/home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work-shared/zynqmp-generic/kernel-source=/usr/src/kernel LD=aarch64-xilinx-linux-ld.bfd AR=aarch64-xilinx-linux-ar O=/home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work-shared/zynqmp-generic/kernel-build-artifacts KBUILD_EXTRA_SYMBOLS= make: *** No targets. Stop. ERROR: oe_runmake failed WARNING: /home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work/zynqmp_generic-xilinx-linux/opendla/1.0-r0/temp/run.do_compile.21451:1 exit 1 from 'exit 1' ERROR: Execution of '/home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work/zynqmp_generic-xilinx-linux/opendla/1.0-r0/temp/run.do_compile.21451' failed with exit code 1: make: *** No targets. Stop. WARNING: /home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/build/tmp/work/zynqmp_generic-xilinx-linux/opendla/1.0-r0/temp/run.do_compile.21451:1 exit 1 from 'exit 1'

ERROR: Task (/home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/project-spec/meta-user/recipes-modules/opendla/opendla.bb:do_compile) failed with exit code '1' NOTE: Tasks Summary: Attempted 3429 tasks of which 3392 didn't need to be rerun and 1 failed.

Summary: 1 task failed: /home/tony/nvdla_vivado_2020_1_20210510/petalinux_nvdla_fpga/nvdla_petalinux/project-spec/meta-user/recipes-modules/opendla/opendla.bb:do_compile Summary: There were 2 ERROR messages shown, returning a non-zero exit code. ERROR: Failed to build project

Could you please advice me? I'm very appreciate with your knowledge and sharing. Warmly thank you so much