RISCY_FPU parameter of tb_pulp reads 0 in Questasim

[omerfirmak]

Hi

First of all, great work! Thanks to all contributors.

I have managed to get all the components (sdk, simulator etc) up and running but although default value of RISCY_FPU parameter of testbench being 1, it reads 0 in questasim and core throws an exception when it encounters a floating point instruction. Is there a switch that I have to set in the build process to enable FPU?

Thanks in advance

[haugoug]

There is a runner which generates the right vsim command depending on the selected configuration. This runner is overwriting the default core configuration that you see in the testbench. You can see it when you do make run, you should see something like that:

# 10.6b

# source tcl_files/config/run_and_exit.tcl
# source tcl_files/run.tcl
# vsim -c -quiet vopt_tb -L models_lib -L vip_lib -t ps "+nowarnTRAN" "+nowarnTSCALE" "+nowarnTFMPC" "+TB_PATH=" "+UVM_NO_RELNOTES" "+preload_file=efuse_preload.data" -dpicpppath /usr/bin/g++ -warning 3197,3748 -sv_lib pkg/sdk/dev/install/ws/lib/libpulpdpi "+VSIM_PATH=" -gUSE_SDVT_SPI=0 -gUSE_SDVT_CPI=0 -gRISCY_FPU=1 -gENABLE_DEV_DPI=0 -gLOAD_L2=JTAG -gUSE_SDVT_I2S=0 -gCORE_TYPE=0

You can see that the runner is giving the real values for RISCY_FPU and CORE_TYPE. To get the core with the FPU you have to source this file when you configure the SDK:

source configs/pulpissimo.sh

In case your test was already compiled for a different configuration, first reconfigure the test with:

make conf

And then compile and run, and check that you get the right parameters on the generated vsim command.

[omerfirmak]

Sorry for reopening this. But configurations seems to be fine.

pulp-run --platform=rtl --dir=/mnt/HDD/pulp-rt-examples/knn/build/pulpissimo --config-file=/mnt/HDD/pulp-rt-examples/knn/build/pulpissimo/config.json --binary=knn/knn 
VSIM_PATH is correctly defined, using following RTL platform: /mnt/HDD/pulpissimo/sim
Launching VSIM with command:
export VSIM_DESIGN_MODEL=sverilog;  export VSIM_RUNNER_FLAGS="-gCORE_TYPE=0 -gRISCY_FPU=1 +preload_file=efuse_preload.data -gLOAD_L2=JTAG -warning 3197,3748" && vsim -64 -c -do 'source /mnt/HDD/pulpissimo/sim/tcl_files/config/run_and_exit.tcl' -do 'source /mnt/HDD/pulpissimo/sim/tcl_files/run.tcl; run_and_exit;'
Reading pref.tcl

# 10.3d

# source /mnt/HDD/pulpissimo/sim/tcl_files/config/run_and_exit.tcl
# source /mnt/HDD/pulpissimo/sim/tcl_files/run.tcl
# vsim -c -do "source /mnt/HDD/pulpissimo/sim/tcl_files/config/run_and_exit.tcl" -do "source /mnt/HDD/pulpissimo/sim/tcl_files/run.tcl; run_and_exit;" 

VSIM_RUNNER_FLAGS is generated correctly but actual call to vsim doesnt include those and I cant see that variable in environment

[haugoug]

Have you modified anything in ./sim/tcl_files ?

[omerfirmak]

Nope, not a single line. I manually exported the flags below, but it didnt help.

export VSIM_DESIGN_MODEL=sverilog; export VSIM_RUNNER_FLAGS="-gCORE_TYPE=0 -gRISCY_FPU=1 +preload_file=efuse_preload.data -gLOAD_L2=JTAG -warning 3197,3748"

[haugoug]

Can also show the rest of the terminal output ?

[omerfirmak]

pulp-run --platform=rtl --dir=/mnt/HDD/pulp-rt-examples/knn/build/pulpissimo --config-file=/mnt/HDD/pulp-rt-examples/knn/build/pulpissimo/config.json --binary=knn/knn 
VSIM_PATH is correctly defined, using following RTL platform: /mnt/HDD/pulpissimo/sim
Launching VSIM with command:
export VSIM_DESIGN_MODEL=sverilog;  export VSIM_RUNNER_FLAGS="-gCORE_TYPE=0 -gRISCY_FPU=1 +preload_file=efuse_preload.data -gLOAD_L2=JTAG -warning 3197,3748" && vsim -64 -c -do 'source /mnt/HDD/pulpissimo/sim/tcl_files/config/run_and_exit.tcl' -do 'source /mnt/HDD/pulpissimo/sim/tcl_files/run.tcl; run_and_exit;'
Reading pref.tcl

# 10.3d

# source /mnt/HDD/pulpissimo/sim/tcl_files/config/run_and_exit.tcl
# source /mnt/HDD/pulpissimo/sim/tcl_files/run.tcl
# vsim -c -do "source /mnt/HDD/pulpissimo/sim/tcl_files/config/run_and_exit.tcl" -do "source /mnt/HDD/pulpissimo/sim/tcl_files/run.tcl; run_and_exit;" 
# Start time: 21:07:41 on Feb 27,2018
# //  Questa Sim-64
# //  Version 10.3d linux_x86_64 Oct  6 2014
# //
# //  Copyright 1991-2014 Mentor Graphics Corporation
# //  All Rights Reserved.
# //
# //  THIS WORK CONTAINS TRADE SECRET AND PROPRIETARY INFORMATION
# //  WHICH IS THE PROPERTY OF MENTOR GRAPHICS CORPORATION OR ITS
# //  LICENSORS AND IS SUBJECT TO LICENSE TERMS.
# //
# ** Warning: (vsim-3015) /mnt/HDD/pulpissimo/sim/../ips/pulp_soc/rtl/pulp_soc/soc_interconnect_wrap.sv(126): [PCDPC] - Port size (32) does not match connection size (64) for port 'AXI_Master_w_data_o'. The port definition is at: /mnt/HDD/pulpissimo/sim/../ips/pulp_soc/rtl/pulp_soc/soc_interconnect.sv(198).
#         Region: /tb_pulp/i_dut/soc_domain_i/pulp_soc_i/i_soc_interconnect_wrap/i_soc_interconnect
# ** Warning: (vsim-3015) /mnt/HDD/pulpissimo/sim/../ips/pulp_soc/rtl/pulp_soc/soc_interconnect_wrap.sv(126): [PCDPC] - Port size (4) does not match connection size (8) for port 'AXI_Master_w_strb_o'. The port definition is at: /mnt/HDD/pulpissimo/sim/../ips/pulp_soc/rtl/pulp_soc/soc_interconnect.sv(199).
#         Region: /tb_pulp/i_dut/soc_domain_i/pulp_soc_i/i_soc_interconnect_wrap/i_soc_interconnect
# ** Warning: (vsim-3015) /mnt/HDD/pulpissimo/sim/../ips/pulp_soc/rtl/pulp_soc/soc_interconnect_wrap.sv(126): [PCDPC] - Port size (32) does not match connection size (64) for port 'AXI_Master_r_data_i'. The port definition is at: /mnt/HDD/pulpissimo/sim/../ips/pulp_soc/rtl/pulp_soc/soc_interconnect.sv(229).
#         Region: /tb_pulp/i_dut/soc_domain_i/pulp_soc_i/i_soc_interconnect_wrap/i_soc_interconnect
# ** Warning: (vsim-8683) Uninitialized out port /tb_pulp/i_dut/soc_domain_i/pulp_soc_i/i_clk_rst_gen/i_fll_soc/i_FLL_digital/TQ has no driver.
# This port will contribute value (U) to the signal network.
# ** Warning: (vsim-8683) Uninitialized out port /tb_pulp/i_dut/soc_domain_i/pulp_soc_i/i_clk_rst_gen/i_fll_soc/i_FLL_digital/JTQ has no driver.
# This port will contribute value (U) to the signal network.
# ** Warning: (vsim-8683) Uninitialized out port /tb_pulp/i_dut/soc_domain_i/pulp_soc_i/i_clk_rst_gen/i_fll_per/i_FLL_digital/TQ has no driver.
# This port will contribute value (U) to the signal network.
# ** Warning: (vsim-8683) Uninitialized out port /tb_pulp/i_dut/soc_domain_i/pulp_soc_i/i_clk_rst_gen/i_fll_per/i_FLL_digital/JTQ has no driver.
# This port will contribute value (U) to the signal network.
# ** Warning: (vsim-8683) Uninitialized out port /tb_pulp/i_dut/soc_domain_i/pulp_soc_i/i_clk_rst_gen/i_fll_cluster/i_FLL_digital/TQ has no driver.
# This port will contribute value (U) to the signal network.
# ** Warning: (vsim-8683) Uninitialized out port /tb_pulp/i_dut/soc_domain_i/pulp_soc_i/i_clk_rst_gen/i_fll_cluster/i_FLL_digital/JTQ has no driver.
# This port will contribute value (U) to the signal network.
# ** Warning: (vsim-3040) Command line generic/parameter "USE_SDVT_I2S" not found in design.
# ** Warning: (vsim-3040) Command line generic/parameter "USE_SDVT_SPI" not found in design.
#  run_and_exit
# START_ADDR[0] = 0x1a100000; END_ADDR[0] = 0X1a200000
# START_ADDR[1] = 0x10000000; END_ADDR[1] = 0X10400000
# [TB]                    0 - Using FLL
# [TB]                    0 - Not using CAM SDVT
# [TB]                    0 - Asserting hard reset
# ** Warning: (vsim-3533) [FOFIW] - Failed to open file "fs/file_0_0.txt" for writing.
# No such file or directory. (errno = ENOENT)    : /mnt/HDD/pulpissimo/sim/../rtl/tb/tb_fs_handler.sv(269)
#    Time: 0 ps  Iteration: 0  Instance: /tb_pulp/i_fs_handler/FS_PRINTF_CLUSTER
# ** Warning: (vsim-3533) [FOFIW] - Failed to open file "fs/file_31_0.txt" for writing.
# No such file or directory. (errno = ENOENT)    : /mnt/HDD/pulpissimo/sim/../rtl/tb/tb_fs_handler.sv(269)
#    Time: 0 ps  Iteration: 0  Instance: /tb_pulp/i_fs_handler/FS_PRINTF_FC
# ** Warning: (vsim-8315) /mnt/HDD/pulpissimo/sim/../ips/pulp_soc/rtl/fc/fc_demux.sv(97): No condition is true in the unique/priority if/case statement.
# ** Warning: (vsim-8315) /mnt/HDD/pulpissimo/sim/../ips/pulp_soc/rtl/fc/fc_demux.sv(97): No condition is true in the unique/priority if/case statement.
# [TB]              5701000 - Virtually removing ROM as JTAG boot was selected in testbench
# [test_mode_if]              7101000 - Init
# [TB]              7101000 - Enabling clock out via jtag
# [test_mode_if]              9801000 - Setting confreg to value 000800.
# [TB]             59801000 - Releasing hard reset
# [SOC_CLK]            409873188 - SOC_CLK Ready
# [TB]            411273188 - Loading L2
# [dbg_if_soc] WRITE32 burst @1a104008 for           4 bytes.
# [dbg_if_soc] WRITE32 burst @1a104004 for           4 bytes.
# [TB]            739073188 - Loading L2
# [JTAG] Loading L2 with jtag interface
# [dbg_if_soc] WRITE32 burst @1c000000 for        1024 bytes.
# [dbg_if_soc] WRITE32 burst @1c000400 for          72 bytes.
# [dbg_if_soc] WRITE32 burst @1c000c50 for        1024 bytes.
# [dbg_if_soc] WRITE32 burst @1c001050 for        1024 bytes.
# [dbg_if_soc] WRITE32 burst @1c001450 for        1024 bytes.
# [dbg_if_soc] WRITE32 burst @1c001850 for        1024 bytes.
# [dbg_if_soc] WRITE32 burst @1c001c50 for         152 bytes.
# [dbg_if_soc] WRITE32 burst @1c008000 for        1024 bytes.
# [dbg_if_soc] WRITE32 burst @1c008400 for        1024 bytes.
# [dbg_if_soc] WRITE32 burst @1c008800 for        1024 bytes.
# [dbg_if_soc] WRITE32 burst @1c008c00 for        1024 bytes.
# [dbg_if_soc] WRITE32 burst @1c009000 for        1024 bytes.
# [dbg_if_soc] WRITE32 burst @1c009400 for        1024 bytes.
# [dbg_if_soc] WRITE32 burst @1c009800 for        1024 bytes.
# [dbg_if_soc] WRITE32 burst @1c009c00 for        1024 bytes.
# [dbg_if_soc] WRITE32 burst @1c00a000 for         864 bytes.
# [TB]          12431673188 - Triggering fetch enable
# [TRACER] Output filename is: trace_core_1f_0.log
# [dbg_if_soc] WRITE32 burst @1a104008 for           4 bytes.
# [TB]          12445573188 - Waiting for end of computation
#          13199499827: Illegal instruction (core 0) at PC 0x1c008e86:
# [STDOUT-CL31_PE0] [RT(31,0)] WARNING: Reached illegal instruction (PC: 0x1c008e86, opcode: 0xd20000d3
# [TB]          13432873188 - Received status core: 0x7fffffff
# ** Note: $stop    : /mnt/HDD/pulpissimo/sim/../rtl/tb/tb_pulp.sv(604)
#    Time: 13432873188 ps  Iteration: 0  Instance: /tb_pulp
# Break at /mnt/HDD/pulpissimo/sim/../rtl/tb/tb_pulp.sv line 604
# Stopped at /mnt/HDD/pulpissimo/sim/../rtl/tb/tb_pulp.sv line 604
# End time: 21:08:07 on Feb 27,2018, Elapsed time: 0:00:26
# Errors: 0, Warnings: 15
VSIM reported an error, leaving
make: *** [/mnt/HDD/pulp-rt-examples/knn/build/pulpissimo/__rules.mk:132: run] Error 1

Looks like vsim.tcl actually sees the VSIM_RUNNER_FLAGS and appends it to the related variable. But dont know what happens beyond that.

[haugoug]

Can you show which instruction you have at address 0x1c008e86 ? If you don't want to deal with objdump flags you can do: make dis

[omerfirmak]

1c008e86: d20000d3 fcvt.d.w ft1,zero

vsim_cmd variable in vsim.tcl actually has the expected flags.

vsim -c -quiet vopt_tb -L models_lib -L vip_lib -t ps +nowarnTRAN +nowarnTSCALE +nowarnTFMPC +TB_PATH=/mnt/HDD/pulpissimo/sim +UVM_NO_RELNOTES -gCORE_TYPE=0 -gRISCY_FPU=1 +preload_file=efuse_preload.data -gLOAD_L2=JTAG -warning 3197,3748 -gENABLE_DEV_DPI=0 -gUSE_SDVT_CPI=0 -gUSE_SDVT_I2S=0 -gUSE_SDVT_SPI=0 +VSIM_PATH=/mnt/HDD/pulpissimo/sim

[haugoug]

Ok this is a compiler issue. The compiler is not working with only 32 bits floats active so we temporary activated the 64 bits float for the compilation. You get an illegal instruction because the compiler has inserted a 64 bit instruction that our core does not support. Are you using doubles in your code ? We will investigate how to fix that and let you know. By the way you can also see the executed instructions in this file: build/pulpissimo/trace_core_1f_0.log You may see some FPU instructions which has been successfully executed before this one.

[omerfirmak]

I see, they are all floats btw.

So, RISCY_FPU parameter shows as 1 in qeusta. But it looks like trace log doesnt contain any FPU instructions, which is probably normal since 1c008e86 is 3rd instruction in to the main function.

1c008e82 <main>:
1c008e82:   7159                    addi    sp,sp,-112
1c008e84:   cece                    sw  s3,92(sp)
1c008e86:   d20000d3            fcvt.d.w    ft1,zero
1c008e8a:   1c0029b7            lui s3,0x1c002
1c008e8e:   b2898993            addi    s3,s3,-1240 # 1c001b28 <test>

[omerfirmak]

I had a call to sqrt , which works on doubles. So apparently I did use doubles, my bad. Changed that to sqrtf and compiler no longer emits double precision instructions., which solves my problem for now. Thank you!

[haugoug]

Ok good we will still try to remove this limitation so that software emulation can be used for doubles.

[haugoug]

By the way I forgot to tell you that the latest versions of the toolchain now support doubles. However this will use a mixture of HW instructions plus SW emulation so it is always better to use floats instead of doubles.