eine
commented
6 years ago @kraigher , I've checked bsub and, although similar, it is overkill. Because it is meant for HPC with multiple physical devices, an additional orchestration service (LSF) must be run. My proposal is more straighforward because all the containers are expected to be executed in the same machine, so no additional service is required.
Well, docker is the service, somehow, as docker exec works for both local containers and services in a swarm cluster. But I'll focus on the local case, which is easier to follow. Four functionally equivalent examples are shown below:
- All the commands at the same time with local tools.
- All the commands at the same time in a temporal container (same as #323, but without VUnit).
- All the commands at the same time in a named container running in the background.
- Commands one-by-one in a named container running in the background.
I want to achieve 2, 3 or 4 with VUnit. I will focus on 4, as I think it is the less invasive approach from VUnit's perspective:
- Create the container and keep it running in the background, before calling VUnit or as a pre-task hook.
- Intercept any
ghdl <GHDL_ARGS>call from VUnit and decorate asdocker exec -t -w <PATH> <CONTAINER_NAME> bash -c 'ghdl <GHDL_ARGS>'. - Stop and remove the container when VUnit exits or as a post-task hook.
That's quite easy to achieve, as long as all the sources and products are kept in the single folder that is shared between VUnit and the container with GHDL. However, VUnit requires additional external sources to be built. E.g., for example/vhdl/array_axis_vcs more than 100 files from usr/local/lib/python3.6/site-packages/vunit_hdl-3.0.3rc0-py3.6.egg/vunit/vhdl/ are compiled. There are two solutions:
- Share two folders between VUnit and the GHDL container: the project folder and the installation path of VUnit sources.
- Share only the project folder and make VUnit copy the required sources there.
Unless you tell me not to do so, I'd go with the first option, because copying the same sources to every project makes little sense to me.
As a result, the command 'decoration' is required to not only prepend and append some args, but also to rewrite the two paths that might appear in <GHDL_ARGS>.
Interception/decoration script
Shall all be done without any modification to VUnit, this can be a local prototype of the interception script:
$ mv /usr/local/bin/ghdl /usr/local/bin/ghdlbin
$ vim /usr/local/bin/ghdl
#!/bin/sh
/usr/local/bin/ghdlback $@
:wqIn order to see the exact commands that are executed:
#!/bin/sh
if [ "$1" != "--version" ]; then
echo "$@"
fi
/usr/local/bin/ghdlback $@and we get:
-a --workdir=/work/vunit_out/ghdl/libraries/vunit_lib --work=vunit_lib --std=08
-P/work/vunit_out/ghdl/libraries/vunit_lib -P/work/vunit_out/ghdl/libraries/osvv
m -P/work/vunit_out/ghdl/libraries/lib /usr/local/lib/python3.6/site-packages/vu
nit_hdl-3.0.3rc0-py3.6.egg/vunit/vhdl/check/src/check.vhd
Compiling into vunit_lib: ../usr/local/lib/python3.6/site-packages/vunit_hdl-3.0
.3rc0-py3.6.egg/vunit/vhdl/array/src/array_pkg.vhd
passed
-a --workdir=/work/vunit_out/ghdl/libraries/vunit_lib --work=vunit_lib --std=08
-P/work/vunit_out/ghdl/libraries/vunit_lib -P/work/vunit_out/ghdl/libraries/osvv
m -P/work/vunit_out/ghdl/libraries/lib /usr/local/lib/python3.6/site-packages/vu
nit_hdl-3.0.3rc0-py3.6.egg/vunit/vhdl/array/src/array_pkg.vhd
Compiling into lib: src/test/tb_axis_loop.vhd
passed
--elab-run --std=08 --work=lib --workdir=/work/vunit_out/ghdl/libraries/lib -P/w
ork/vunit_out/ghdl/libraries/vunit_lib -P/work/vunit_out/ghdl/libraries/osvvm -P
/work/vunit_out/ghdl/libraries/lib tb_axis_loop tb -gtb_path=/work/src/test/ -gr
unner_cfg=active python runner : true,enabled_test_cases : test,output path : /w
ork/vunit_out/test_output/lib.tb_axis_loop.test_260bb5c8c675e898eca5dc9024a4420e
de12c0bc/,tb path : /work/src/test/,use_color : true --assert-level=errorthat we can rewrite as:
-a --workdir=${WORKDIR}/vunit_out/ghdl/libraries/vunit_lib --work=vunit_lib --std=08
-P${WORKDIR}/vunit_out/vunit_lib -P${WORKDIR}/vunit_out/ghdl/libraries/osvv
m -P${WORKDIR}/vunit_out/lib ${VUDIR}/vhdl/check/src/check.vhd
Compiling into vunit_lib: ..${VUDIR}/vhdl/array/src/array_pkg.vhd
passed
-a --workdir=${WORKDIR}/vunit_out/ghdl/libraries/vunit_lib --work=vunit_lib --std=08
-P${WORKDIR}/vunit_out/ghdl/libraries/vunit_lib -P${WORKDIR}/vunit_out/ghdl/libraries/osvv
m -P${WORKDIR}/vunit_out/ghdl/libraries/lib ${VUDIR}/vhdl/array/src/array_pkg.vhd
Compiling into lib: src/test/tb_axis_loop.vhd
passed
--elab-run --std=08 --work=lib --workdir=${WORKDIR}/vunit_out/ghdl/libraries/lib -P${WORKDIR}/vunit_out/ghdl/libraries/vunit_lib -P${WORKDIR}/vunit_out/ghdl/libraries/osvvm -P
${WORKDIR}/vunit_out/lib tb_axis_loop tb -gtb_path=${WORKDIR}/src/test/ -gr
unner_cfg=active python runner : true,enabled_test_cases : test,output path : ${WORKDIR}/vunit_out/test_output/lib.tb_axis_loop.test_260bb5c8c675e898eca5dc9024a4420e
de12c0bc/,tb path : ${WORKDIR}/src/test/,use_color : true --assert-level=errorwhere
WORKDIR="/work"
VUDIR="/usr/local/lib/python3.6/site-packages/vunit_hdl-3.0.3rc0-py3.6.egg/vunit"So we can either hardcode or use regexps to replace those with MYWORKDIR and MYVUDIR, that correspond to the paths inside the container. But it would be cleaner if we could instruct VUnit to use MYWORKDIR and MYVUDIR only for external calls. Note that it should keep using WORKDIR and VUDIR internally.
If that was supported, examples 2, 3 and 4 could be easily achieved replacing /usr/local/bin/ghdlback $@ with docker exec -t -w /src ghdl-sim bash -c "ghdl $@".
I might also have overlooked the docs. If this is already supported, please let me know.
Examples: from local to docker run to docker exec
git clone https://github.com/1138-4EB/hwd-ide
cd hwd-ide/examples/full_adder
#---
#--- All the commands at the same time with local tools
#---
mkdir -p output
cd output
ghdl -a ../hdl/*.vhd
ghdl -r adder_tb --vcd=adder.vcd
cd ..
ls -la output
rm -rf output
#---
#--- All the commands at the same time in a temporal container
#---
docker run --rm -t \
-v /$(pwd)://src \
-w //src \
ghdl/ghdl:stretch-mcode \
bash -c '\
mkdir -p output && \
cd output && \
ghdl -a ../hdl/*.vhd && \
ghdl -r adder_tb --vcd=adder.vcd\
'
ls -la output
rm -rf output
#---
#--- All the commands at the same time in a named container running in the background
#---
# First, create the container and keep it running in the background
docker run --name ghdl-sim -d \
-v /$(pwd)://src \
ghdl/ghdl:stretch-mcode \
tail -f /dev/null
# Then, exec commands inside
docker exec -t \
-w //src \
ghdl-sim \
bash -c '\
mkdir -p output && \
cd output && \
ghdl -a ../hdl/*.vhd && \
ghdl -r adder_tb --vcd=adder.vcd\
'
# Last, stop and remove the container
docker rm -f ghdl-sim
ls -la output
rm -rf output
#---
#--- Commands one-by-one in a named container running in the background
#---
# First, create the container and keep it running in the background
docker run --name ghdl-sim -d \
-v /$(pwd)://src \
ghdl/ghdl:stretch-mcode \
tail -f /dev/null
# Then, run each command in its own exec
docker exec -t -w //src ghdl-sim bash -c '\
mkdir -p output \
'
docker exec -t -w //src/output ghdl-sim bash -c '\
ghdl -a ../hdl/*.vhd \
'
docker exec -t -w //src/output ghdl-sim bash -c '\
ghdl -r adder_tb --vcd=adder.vcd \
'
# Last, stop and remove the container
docker rm -f ghdl-sim
ls -la output
rm -rf output
kraigher
Because VUnit requires the simulator to be available in the path, and executable names (vsim, vcom, etc.) are 'hardcoded', VUnit must be installed in the same docker image as the simulator. In a use case, I have two images, one with GHDL and a different one with QuestaSim, and VUnit is installed in both of them. Indeed, because I have versions with and without VUnit, there are four images. Each time VUnit is updated, two of them have to be rebuilt.
My proposal is to make VUnit execute simulators which are inside a container, from outside:
In the latter, I would have three images instead of four, and I would have to rebuild a single one. Note how this would scale with more simulators. Furthermore, if the VUnit docker image was generated in the vunit travis-ci flow, I would consume it as an alternative to
ghdl/ext:vunit.A workaround would be to 'hack' it at system level. That is, write some man-in-the-middle shell scripts to get VUnit's calls and forward them to containers. However, it is not clean, because it is not straighforward to handle the lifecycle of tests as VUnit does.