CSL-KU / firesim-nvdla

FireSim-NVDLA: NVIDIA Deep Learning Accelerator (NVDLA) Integrated with RISC-V Rocket Chip SoC Running on the Amazon FPGA Cloud
Other
161 stars 31 forks source link

Cannot build the Quadcore with NVDLALarge #10

Open franktaTian opened 5 years ago

franktaTian commented 5 years ago

Hi, I am trying to rebuild some HW configs on AMI 1.60 which supports Vivado 2018.03.op (Not the 2018.02) Here is the how to : I download the firesim and firesim-nvdla . Setup up the firesim environment successfully

Under the firesim-nvdla/sim export DESIGN=FireSimNoNIC export TARGET_CONFIG=FireSimRocketChipSingleCoreConfig_WithNVDLALarge export PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig75MHz make replace-rtl

Copy the generated directory to firesim as follows : cp -r /home/centos/src/project_data/firesim-nvdla/platforms/f1/aws-fpga/hdk/cl/developer_designs/cl_FireSimNoNIC-FireSimRocketChipSingleCoreConfig_WithNVDLALarge-FireSimDDR3FRFCFSLLC4MBConfig75MHz /home/centos/src/project_data/firesim/platforms/f1/aws-fpga/hdk/cl/developer_designs/

Then build this design under firesim : pass . Create the load the AFI :passed. make f1 : passed

And run simulation successfully! sudo ./FireSim-f1 +permissive $(sed ':a;N;$!ba;s/\n/ /g' runtime.conf) +macaddr0=00:12:6D:00:00:02 +slotid=0 +niclog0=niclog +trace-start0=0 +trace-end0=-1 +linklatency0=350 +netbw0=100 +profile-interval=-1 +zero-out-dram +shmemportname0=default +permissive-off +prog0=darknet-nvdla-bin +blkdev0=darknet-nvdla.img

But when I try to build another HW configuration , the vivado(2018.03.op) can not route successfully.

export DESIGN=FireSimNoNIC export TARGET_CONFIG=FireSimRocketChipQuadCoreConfig_WithNVDLALarge export PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig75MHz make replace-rtl

And when I try to build this design, vivado cannot route it successfully: >> Phase 9 Depositing Routes Phase 9 Depositing Routes | Checksum: 1d4c635ab

Time (s): cpu = 21:24:12 ; elapsed = 09:48:13 . Memory (MB): peak = 25375.250 ; gain = 3578.977 ; free physical = 4768 ; free virtual = 31398 INFO: [Route 35-77] Router completed with failures. Please check the log file for Critical Warnings and run report_route_status for a summary of routing status. ERROR: [Constraints 18-1000] Routing results verification failed due to partially-conflicted nets (Up to first 10 of violated nets): WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_core/u_c/c_int_5/pipe_p1/p1_skid_pipe_data[0] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_core/u_c/c_int_5/pipe_p1/p1_skid_pipe_data[12] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_core/u_c/c_int_5/pipe_p1/p1_skid_pipe_data[32] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_o/u_NV_NVDLA_cvif/u_read/u_eg/lat_fifo1/rq_rd_adr[0] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_o/u_NV_NVDLA_cvif/u_read/u_eg/lat_fifo1/rq_rd_adr[1] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_rdma/u_erdma/u_eg/u_alu/pipe_p1/beat_cnt_reg[0]_rep_2 WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_rdma/u_erdma/u_eg/u_alu/pipe_p1/pipe_skid_out_pd0 WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_core/u_c/c_int_2/pipe_p2/p2_pipe_data_reg_n0[12] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_core/u_c/c_int_2/pipe_p2/p2_pipe_data_reg_n0[35] WRAPPER_INST/CL/firesim_top/top/sim/target/nvdla/u_nvdla/nvdla_top/u_partition_p/u_NV_NVDLA_sdp/u_core/u_c/c_int_2/pipe_p2/p3_skid_data[15]_i_15__13_n_0

...

Routing Is Done. INFO: [Common 17-83] Releasing license: Implementation 183 Infos, 12 Warnings, 7 Critical Warnings and 1 Errors encountered. route_design completed successfully route_design: Time (s): cpu = 21:36:47 ; elapsed = 09:57:01 . Memory (MB): peak = 25399.258 ; gain = 3602.984 ; free physical = 7102 ; free virtual = 33733 ERROR: route_design failed. Writing route_design checkpoint for debug /home/centos/src/project_data/firesim/platforms/f1/aws-fpga/hdk/cl/developer_designs/cl_FireSimNoNIC-FireSimRocketChipQuadCoreConfig_WithNVDLALarge-FireSimDDR3FRFCFSLLC4MBConfig75MHz/build/checkpoints/19_11_02-152028.post_route_design_error.dcp Netlist sorting complete. Time (s): cpu = 00:00:00.25 ; elapsed = 00:00:00.25 . Memory (MB): peak = 25399.258 ; gain = 0.000 ; free physical = 7052 ; free virtual = 33687 Writing placer database... INFO: [Timing 38-480] Writing timing data to binary archive. Netlist sorting complete. Time (s): cpu = 00:00:01 ; elapsed = 00:00:00.40 . Memory (MB): peak = 25399.258 ; gain = 0.000 ; free physical = 5914 ; free virtual = 33197 Writing XDEF routing. Writing XDEF routing logical nets. Writing XDEF routing special nets. Write XDEF Complete: Time (s): cpu = 00:03:43 ; elapsed = 00:01:48 . Memory (MB): peak = 25399.258 ; gain = 0.000 ; free physical = 3905 ; free virtual = 33276 INFO: [Common 17-1381] The checkpoint '/home/centos/src/project_data/firesim/platforms/f1/aws-fpga/hdk/cl/developer_designs/cl_FireSimNoNIC-FireSimRocketChipQuadCoreConfig_WithNVDLALarge-FireSimDDR3FRFCFSLLC4MBConfig75MHz/build/checkpoints/19_11_02-152028.post_route_design_error.dcp' has been generated. write_checkpoint: Time (s): cpu = 00:10:58 ; elapsed = 00:11:30 . Memory (MB): peak = 25399.258 ; gain = 0.000 ; free physical = 6292 ; free virtual = 33654

ERROR: route_design failed. while executing "error $errMsg" invoked from within "if {[catch $impl_step $errMsg]} { if {[string match $phase "route_design"]} { puts "\tERROR: $phase failed. Writing $phase checkpoint f..." (procedure "impl_step" line 92) invoked from within "impl_step route_design $TOP $route_options $route_directive $route_preHookTcl $route_postHookTcl" invoked from within "if {$route} { puts "\nAWS FPGA: ([clock format [clock seconds] -format %T]) - Routing design"; impl_step route_design $TOP $route_options ..." invoked from within "if {$implement} {

####################### \ # Link Design ######################## if {$link} { \ ####Create in-memory prjoect and set..." (file "create_dcp_from_cl.tcl" line 225) INFO: [Common 17-206] Exiting Vivado at Sun Nov 3 07:31:39 2019...

I find you published a AFI as : { "UpdateTime": "2019-03-31T12:38:35.000Z", "Name": "firesim-quadcore-no-nic-nvdla-ddr3-llc4mb", "Tags": [], "PciId": { "SubsystemVendorId": "0xfedd", "VendorId": "0x1d0f", "DeviceId": "0xf000", "SubsystemId": "0x1d51" }, "FpgaImageGlobalId": "agfi-0ceed51cae44c4e59", "Public": true, "State": { "Code": "available" }, "ShellVersion": "0x04261818", "OwnerId": "785173868334", "FpgaImageId": "afi-0f3d175134d69618e", "CreateTime": "2019-03-31T11:36:21.000Z", "Description": "firesim-buildtriplet:FireSimNoNIC-FireSimRocketChipQuadCoreConfig_WithNVDLALarge-FireSimDDR3FRFCFSLLC4MBConfig75MHz,firesim-deploytriplet:FireSimNoNIC-FireSimRocketChipQuadCoreConfig_WithNVDLALarge-FireSimDDR3FRFCFSLLC4MBConfig75MHz,firesim-commit:2e8b749c031da794b6c768df187119fbf3e10170-dirty" }, It work. How did you build it successfully using vivado?

### BTW: 1、 I can also build a HW config and can simulate it( to login ): FireSimNoNIC-FireSimRocketChipQuadCoreConfig_WithNVDLASmall-FireSimDDR3FRFCFSLLC4MBConfig75MHz

but it hang up when run ./solo.sh .

And 2、 when I tried to build a Rocket Siglecore with NVDLALarge and NIC as follows: export DESIGN=FireSim export TARGET_CONFIG=FireSimRocketChipSingleCoreConfig_WithNVDLALarge export PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig75MHz

when I try to simulate it: sudo ./FireSim-f1 +permissive $(sed ':a;N;$!ba;s/\n/ /g' runtime.conf) +macaddr0=00:12:6D:00:00:02 +slotid=0 +niclog0=niclog +trace-start0=0 +trace-end0=-1 +linklatency0=350 +netbw0=100 +profile-interval=-1 +zero-out-dram +shmemportname0=default +permissive-off +prog0=darknet-nvdla-bin +blkdev0=darknet-nvdla.img

It hung on the Commencing simulation. sudo ./FireSim-f1 +permissive $(sed ':a;N;$!ba;s/\n/ /g' runtime.conf) +macaddr0=00:12:6D:00:00:02 +slotid=0 +niclog0=niclog +trace-start0=0 +trace-end0=-1 +linklatency0=6405 +netbw0=200 +profile-interval=-1 +zero-out-dram +shmemportname0=default +permissive-off +prog0=br-base-bin +blkdev0=br-base.img AFI PCI Vendor ID: 0x1d0f, Device ID 0xf000 Using xdma write queue: /dev/xdma0_h2c_0 Using xdma read queue: /dev/xdma0_c2h_0 UART0 is here (stdin/stdout). command line for program 0. argc=45: +permissive +mm_relaxFunctionalModel=0 +mm_openPagePolicy=1 +mm_backendLatency=2 +mm_schedulerWindowSize=8 +mm_transactionQueueDepth=8 +mm_dramTimings_tAL=0 +mm_dramTimings_tCAS=14 +mm_dramTimings_tCMD=1 +mm_dramTimings_tCWD=10 +mm_dramTimings_tCCD=4 +mm_dramTimings_tFAW=25 +mm_dramTimings_tRAS=33 +mm_dramTimings_tREFI=7800 +mm_dramTimings_tRC=47 +mm_dramTimings_tRCD=14 +mm_dramTimings_tRFC=160 +mm_dramTimings_tRRD=8 +mm_dramTimings_tRP=14 +mm_dramTimings_tRTP=8 +mm_dramTimings_tRTRS=2 +mm_dramTimings_tWR=15 +mm_dramTimings_tWTR=8 +mm_rowAddr_offset=18 +mm_rowAddr_mask=65535 +mm_rankAddr_offset=16 +mm_rankAddr_mask=3 +mm_bankAddr_offset=13 +mm_bankAddr_mask=7 +mm_llc_wayBits=3 +mm_llc_setBits=9 +mm_llc_blockBits=6 +macaddr0=00:12:6D:00:00:02 +slotid=0 +niclog0=niclog +trace-start0=0 +trace-end0=-1 +linklatency0=6405 +netbw0=200 +profile-interval=-1 +zero-out-dram +shmemportname0=default +permissive-off br-base-bin +blkdev0=br-base.img using link latency: 6405 cycles using netbw: 200 using netburst: 8 Using non-slot-id associated shmemportname: opening/creating shmem region /port_ntsdefault_0 Using non-slot-id associated shmemportname: opening/creating shmem region /port_stndefault_0 Using non-slot-id associated shmemportname: opening/creating shmem region /port_ntsdefault_1 Using non-slot-id associated shmemportname: opening/creating shmem region /port_stndefault_1 random min: 0x0, random max: 0xffffffffffffffff On init, 915 token slots available on input. Zeroing out FPGA DRAM. This will take a few seconds... Commencing simulation.

Any help will be appreciated. Thanks

Frankta Tian

farzadfch commented 5 years ago

1、 I can also build a HW config and can simulate it( to login ):

The software for small configuration is not stable yet and is planned to be supported in the future releases.

And 2、 when I tried to build a Rocket Siglecore with NVDLALarge and NIC as follows:

When you build a target with NIC you need to specify a topology with network support in config_runtime.ini. Setting topology to example_1config should fix this issue.

As for your original question about building the hardware, I just kicked off a build run. I get back to you on this.

farzadfch commented 5 years ago

By default, the build is launched on AMI 1.5.0. Is there any particular reason that you changed it to AMI 1.6.0?

franktaTian commented 5 years ago

@farzadfch Thanks for reply. I am trying to merge NVDLA into the Firesim master branch.It seems just support 1.6.0 and up now?

franktaTian commented 5 years ago

@farzadfch Thanks again. I can boot the design with NIC(single core + nvdla small) now.Yes ,use the topology=example_1config