search

ReconfigureIO / reco-sdaccel

0 stars 1 forks source link

give priority to vendor directory over built-in sdaccel #226

Closed CampGareth closed 6 years ago

CampGareth commented 6 years ago

If the user doesn't provide a vendored sdaccel we symlink ours in, however if they provide one we currently prioritise our own for host-side code: https://github.com/ReconfigureIO/reco-sdaccel/blob/5e6ca06c3bb75d4f78789fb0686d1341e92df550/sdaccel-builder.mk#L116

Here's the output from my local testing with the current line, prioritising our sdaccel in /opt/. OpenCL errors are expected:

$ LIBRARY_PATH=/runtime/lib/x86_64/:/usr/lib/x86_64-linux-gnu: CGO_CFLAGS=-I/runtime/include/1_2/ GOPATH=/opt/sdaccel-builder/go:"/tmp/.reco-work/vendor" /opt/sdaccel-builder/go-root/bin/go build -tags opencl -o "/tmp/.reco-work/sdaccel/dist"/test-addition /tmp/cmd/test-addition/main.go
# github.com/ReconfigureIO/sdaccel/xcl
In file included from /opt/sdaccel-builder/go/src/github.com/ReconfigureIO/sdaccel/xcl/xcl.go:8:0:
./xcl.h:33:23: fatal error: CL/opencl.h: No such file or directory
 #include <CL/opencl.h>
                       ^
compilation terminated.

Proposed, prioritising the user's sdaccel:

$ LIBRARY_PATH=/runtime/lib/x86_64/:/usr/lib/x86_64-linux-gnu: CGO_CFLAGS=-I/runtime/include/1_2/ GOPATH="/tmp/.reco-work/vendor":/opt/sdaccel-builder/go /opt/sdaccel-builder/go-root/bin/go build -tags opencl -o "/tmp/.reco-work/sdaccel/dist"/test-addition /tmp/cmd/test-addition/main.go
# github.com/ReconfigureIO/sdaccel/xcl
In file included from .reco-work/vendor/src/github.com/ReconfigureIO/sdaccel/xcl/xcl.go:8:0:
./xcl.h:33:23: fatal error: CL/opencl.h: No such file or directory
 #include <CL/opencl.h>
                       ^
compilation terminated.
CampGareth commented 6 years ago

Josh asked me to test this with a project that has a bunch of stuff but no sdaccel, so here we go, testing web-md5 on staging with the following content in vendor directory:

vendor/
├── github.com
│   ├── gin-contrib
│   │   └── sse
│   ├── gin-gonic
│   │   └── gin
│   ├── golang
│   │   └── protobuf
│   ├── json-iterator
│   │   └── go
│   ├── mattn
│   │   └── go-isatty
│   ├── ReconfigureIO
│   │   └── crypto
│   └── ugorji
│       └── go
├── golang.org
│   └── x
│       └── sys
└── gopkg.in
    ├── go-playground
    │   └── validator.v8
    └── yaml.v2
CampGareth commented 6 years ago

well that simulation timed out but everything compiled successfully so I'm calling it a win, logs attached:

running simulation

status: QUEUED
Waiting for Batch job to start
status: STARTED
mkdir -p ""/mnt/.reco-work/sdaccel/dist""
cd "/mnt/.reco-work/sdaccel/dist" && XCL_EMULATION_MODE=hw_emu emconfigutil --xdevice xilinx:aws-vu9p-f1:4ddr-xpr-2pr:4.0 --nd 1
****** configutil v2017.1_sdx (64-bit)
  **** SW Build 1933108 on Fri Jul 14 11:54:19 MDT 2017
    ** Copyright 1986-2017 Xilinx, Inc. All Rights Reserved.
INFO: [ConfigUtil 60-895]    Target platform: /opt/Xilinx/SDx/2017.1.op/platforms/xilinx_aws-vu9p-f1_4ddr-xpr-2pr_4_0/xilinx_aws-vu9p-f1_4ddr-xpr-2pr_4_0.xpfm
emulation configuration file `emconfig.json` is created in ./ directory 
mkdir -p /mnt/vendor/github.com/ReconfigureIO
ln -sf "/opt/sdaccel-builder/go/src/github.com/ReconfigureIO/sdaccel" /mnt/vendor/github.com/ReconfigureIO/sdaccel
mkdir -p ""/mnt/.reco-work/vendor""
ln -sf "/mnt/vendor" ""/mnt/.reco-work/vendor"/src"
/opt/sdaccel-builder/go/bin/reco-fix .
vendor/github.com/ReconfigureIO/crypto/examples/md5/cmd/test-md5/main.go: fixed sdaccel
vendor/github.com/ReconfigureIO/crypto/examples/md5/main.go: fixed sdaccel
LIBRARY_PATH=/opt/Xilinx/SDx/2017.1.op/runtime/lib/x86_64/:/usr/lib/x86_64-linux-gnu:/opt/Xilinx/SDx/2017.1.op/SDK/lib/lnx64.o CGO_CFLAGS=-I/opt/Xilinx/SDx/2017.1.op/runtime/include/1_2/ GOPATH="/mnt/.reco-work/vendor":/opt/sdaccel-builder/go go build -tags opencl -o "/mnt/.reco-work/sdaccel/dist"/download-and-hash /mnt/cmd/download-and-hash/main.go
LIBRARY_PATH=/opt/Xilinx/SDx/2017.1.op/runtime/lib/x86_64/:/usr/lib/x86_64-linux-gnu:/opt/Xilinx/SDx/2017.1.op/SDK/lib/lnx64.o CGO_CFLAGS=-I/opt/Xilinx/SDx/2017.1.op/runtime/include/1_2/ GOPATH="/mnt/.reco-work/vendor":/opt/sdaccel-builder/go go build -tags opencl -o "/mnt/.reco-work/sdaccel/dist"/run-webserver /mnt/cmd/run-webserver/main.go
mkdir -p "/tmp/workspace/.reco-work/sdaccel/build"
mkdir -p "/mnt/.reco-work/sdaccel/verilog"
mkdir -p "/mnt/.reco-work/sdaccel/verilog"/includes
if [ -d "/mnt/includes/" ]; then cp /mnt/includes/* "/mnt/.reco-work/sdaccel/verilog"/includes; fi
cd /opt/sdaccel-builder/eTeak && PATH=/opt/sdaccel-builder/eTeak/bin:/opt/sdaccel-builder/smi/bin:/opt/sdaccel-builder/bin:/opt/sdaccel-builder/go-root/bin:/opt/Xilinx/SDx/2017.1.op/bin:/opt/sdaccel-builder:/opt/Xilinx/SDx/2017.1.op/Vivado/bin:/opt/Xilinx/SDx/2017.1.op/bin:/opt/Xilinx/SDx/2017.1.op/Vivado/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin GOPATH="/mnt/.reco-work/vendor" /usr/bin/time -ao /mnt/times.out -f "verilog,%e,%M" ./go-teak-sdaccel build --full-imports -O -p100  /mnt/main.go -o "/mnt/.reco-work/sdaccel/verilog"/main.v
mkdir -p "/mnt/.reco-work/sdaccel/verilog"/library
mkdir -p ""/mnt/.reco-work/sdaccel/logs""
cd "/tmp/workspace/.reco-work/sdaccel/build" && /usr/bin/time -ao /mnt/times.out -f "xo,%e,%M" vivado -notrace -mode batch \
    -source "/opt/sdaccel-builder/go-teak/src/sdaccel/scripts/sda_kernel_build.tcl" -tclargs \
    -action_source_file ""/mnt/.reco-work/sdaccel/verilog"/main.v" -include_source_dir ""/mnt/.reco-work/sdaccel/verilog"/includes" \
    -library_source_dir ""/mnt/.reco-work/sdaccel/verilog"/library" -param_args_file ""/mnt/.reco-work/sdaccel/verilog"/main.v.xmldef" \
    -vendor reconfigure.io -library sdaccel-builder -name stub -version 0.1 -part xcvu9p-flgb2104-2-i \
    -part_family virtexuplus -axi_data_width 64 > "/mnt/.reco-work/sdaccel/logs"/synthesis_log.txt
mkdir -p ""/mnt/.reco-work/sdaccel/dist/xclbin""
cd "/tmp/workspace/.reco-work/sdaccel/build" && /usr/bin/time -ao /mnt/times.out -f "xclbin,%e,%M" xocc -j4 -O3 -t "hw_emu"  --xdevice xilinx:aws-vu9p-f1:4ddr-xpr-2pr:4.0 -l "/tmp/workspace/.reco-work/sdaccel/build"/"reconfigure_io_sdaccel_builder_stub_0_1.xo" -o "/mnt/.reco-work/sdaccel/dist/xclbin"/"kernel_test".hw_emu.xilinx_aws-vu9p-f1_4ddr-xpr-2pr_4_0.xclbin -r system
****** xocc v2017.1_sdx (64-bit)
  **** SW Build 1933108 on Fri Jul 14 11:54:19 MDT 2017
    ** Copyright 1986-2017 Xilinx, Inc. All Rights Reserved.
INFO: [XOCC 60-899] Reading --xp value from platform: param:compiler.lockFlowCritSlackThreshold=0
Attempting to get a license: ap_opencl
INFO: [XOCC 17-1223] The version limit for your license is '2018.03' and will expire in -137 days. A version limit expiration means that, although you may be able to continue to use the current version of tools or IP with this license, you will not be eligible for any updates or new releases.
Feature available: ap_opencl
INFO: [XOCC 60-629] Linking for hardware emulation target
INFO: [XOCC 60-895]    Target platform: /opt/Xilinx/SDx/2017.1.op/platforms/xilinx_aws-vu9p-f1_4ddr-xpr-2pr_4_0/xilinx_aws-vu9p-f1_4ddr-xpr-2pr_4_0.xpfm
INFO: [XOCC 60-423]   Target device: xilinx:aws-vu9p-f1:4ddr-xpr-2pr:4.0
INFO: [XOCC 60-251]   Hardware accelerator integration...
INFO: [XOCC 60-244] Generating system estimate report...
INFO: [XOCC 60-677] Generated system_estimate.xtxt
INFO: [XOCC 60-586] Created /mnt/.reco-work/sdaccel/dist/xclbin/kernel_test.hw_emu.xilinx_aws-vu9p-f1_4ddr-xpr-2pr_4_0.xclbin
INFO: [XOCC 60-791] Total elapsed time: 0h 9m 4s
mkdir -p "/tmp/workspace/.reco-work/sdaccel/build"
mkdir -p "/tmp/workspace/.reco-work/sdaccel/build"/reports
parse_times times.out > "/tmp/workspace/.reco-work/sdaccel/build"/reports/timing.json
mkdir -p ""/mnt/.reco-work/sdaccel/reports""
merge_reports "/tmp/workspace/.reco-work/sdaccel/build"/reports/reconfigure_io_sdaccel_builder_stub_0_1_util.json "/tmp/workspace/.reco-work/sdaccel/build"/reports/timing.json > "/mnt/.reco-work/sdaccel/reports"/utilization.json
INFO: [SDx-EM 01] Hardware emulation runs detailed simulation underneath. It may take long time for large data set. Please use a small dataset for faster execution. You can still get performance trend for your kernel with smaller dataset.
Simulation ID: 56e8b859-d42a-457c-934b-0b4a379d903e Status: Timed-Out
bohde commented 6 years ago

@reconfig-bot r+

reconfig-bot commented 6 years ago

:pushpin: Commit 887086e has been approved by joshbohde

reconfig-bot commented 6 years ago

:hourglass: Testing commit 887086e with merge a164027...

reconfig-bot commented 6 years ago

:broken_heart: Test failed - status-jenkins

CampGareth commented 6 years ago

:( some tests failed due to the underlying EC2 instance terminating

CampGareth commented 6 years ago

@reconfig-bot retry

reconfig-bot commented 6 years ago

:hourglass: Testing commit 887086e with merge 4126c34...

reconfig-bot commented 6 years ago

:broken_heart: Test failed - status-jenkins

CampGareth commented 6 years ago 
Job status
Status FAILED
Status reason Host EC2 (instance i-04775320dff43b7ae) terminated.

and again.

CampGareth commented 6 years ago

@reconfig-bot retry

reconfig-bot commented 6 years ago

:hourglass: Testing commit 887086e with merge 88ade6f...

reconfig-bot commented 6 years ago

:sunny: Test successful - status-jenkins Approved by: joshbohde Pushing 88ade6f7a4361597db0aca019c7e144bdc5f0029 to master...