Koheron / koheron-sdk

SDK for FPGA / Linux Instruments
https://www.koheron.com/software-development-kit/
Other
100 stars 40 forks source link

Red pitaya crash #418

Closed CHV123 closed 7 years ago

CHV123 commented 7 years ago

Hi,

I'm having a problem where the red pitaya disconnects and stops responding once I try to read the status registers or write to the control registers and bram. I can't figure out how to diagnose this. Do you know what might be happening or have any suggestions for diagnosis?

The instrument builds and is written to the red pitaya without errors and no system errors seem to appear in the red pitaya logs.

Thanks for any help, Charlie

jeanminet commented 7 years ago

Hi Charlie,

My first guess would be that you have a clock problem. If your bram / status / control register is not clocked, linux will hang when trying to access mapped memory.

For diagnosis, you can ssh to your red pitaya and run the command:

journalctl -u koheron-server.service -f

This outputs the log of the server. https://www.koheron.com/software-development-kit/documentation/linux-driver-cpp/debugging

Jean

CHV123 commented 7 years ago

I will give that a go, using some log commands in the driver. However, after just a quick test it seems that journalctl is cleared after re-boot and I lose ssh once the test program is run. This means that I can get no more information other than that the test program has indeed started the Koheron server. I am also connecting to the Pitaya using the COM port, and the same thing happens as it does for ssh.

Also a look at my block diagram seems to show that my memory (bram/status/control) is clocked (from the PS using FCLK_CLK0). If they weren't clocked properly would this not show up in verification or the synthesis/implementation?

Thanks, Charlie

jeanminet commented 7 years ago

You can remove the line tmpfs /var/log tmpfs size=1M,noatime 0 0 in /etc/fstab to make the log persistent. But I don't think you would get more information...

Can you give the output of cat /sys/devices/soc0/amba/f8007000.devcfg/fclk/fclk0/set_rate ?

CHV123 commented 7 years ago

Output of that command is 199999998

Charlie

jeanminet commented 7 years ago

So FCLK_CLK0 is running at 200 MHz.

Did you specify:

parameters:
    fclk0: 200000000

in the config.yml ?

CHV123 commented 7 years ago

Yes, that has been specified in the config file for this design. I noticed that for the spectrum example two clocks are used from the PS, one for the fifo's and registers and another for the bram. Is this due to the larger design, as my design is also pretty big?

jeanminet commented 7 years ago

In the spectrum example, I use the 2 available general purpose AXI master ports: M_AXI_GP0 (connected to FCLK_CLK0) and M_AXI_GP1 (connected to FCLK_CLK1). It can be useful when you have too many slaves connected to a single master. Moreover, you can use different clock speeds to achieve timing closure.

Does Vivado complain about timing in your design ?

CHV123 commented 7 years ago

Yes, it looks like the timing is definitely the problem as my total negative slack is quite high. I will try and solve this and see if it works.

jeanminet commented 7 years ago

That would explain the problem. Maybe you can start by specifying

parameters:
    fclk0: 125000000

in your config.yml

CHV123 commented 7 years ago

Thanks, I'll try that. Is it possible to change the implementation strategy from the make command?

jeanminet commented 7 years ago

It is not really from the make command but you can change the implementation strategy like this: c867086d98

CHV123 commented 7 years ago

Thanks for all the help Jean. I close the issue seeing as it's my timing that's the problem.

Charlie

CHV123 commented 7 years ago

Hi Jean,

I've sorted the timing and added some commands in the c driver to aid with debug but the red pitaya is still exhibiting the same problems. Do you have any more suggestions? The statements I added to the driver are not showing up in the journalctl -u koheron-server.service -f command.

Thanks, Charlie

jeanminet commented 7 years ago

Hi Charlie,

It is hard to tell without more information. Maybe I could try to help if you can send me your project.

Jean