pavel-demin / red-pitaya-notes

Notes on the Red Pitaya Open Source Instrument
http://pavel-demin.github.io/red-pitaya-notes/
MIT License
337 stars 209 forks source link

Bus error or non-responding system with project "adc_test" #528

Closed jonschumacher closed 7 years ago

jonschumacher commented 7 years ago

I tried to run the adc_test project but if I start the server first, loading the image on the FPGA leads to a non-responding system. Trying the other way around with first loading the image and then starting the server yields a "Bus error". According to the post in this thread [1] it is necessary to reduce the memory available to Linux and I guess not doing so might be related to the error. The post refers to [2], but this post is about using the Xilinx SDK and I am unable to adapt this to the Red Pitaya ecosystem. I would really appreciate any help on this subject.

[1] https://forum.redpitaya.com/viewtopic.php?f=14&t=551 [2] http://blog.fakultaet-technik.de/development-with-zynq-part-3-build-boot-files/

pavel-demin commented 7 years ago

Thanks for testing the adc_test project.

It's been awhile since I tested it myself. Looks like I introduced a couple of typos when recently updated all my projects. I've just fixed them with this commit. Now, adc-test-server.c can be compiled without error messages.

After this fix, it works without problem with my Debian based images. I think that to reproduce your problem, I'll need more information about your setup (SD card image, how do you build adc_test.bit and adc-test-server, how do you configure FPGA and how do you run adc-test-server).

Anyway, I can think of at least two solutions:

BTW. The ideas from [2] are implemented in the following patches:

jonschumacher commented 7 years ago

Hi Pavel,

I was actually naive enough to simply use the official RP image. I patched the server in the same way you did and set the version of the Clock Wizard to 5.4 since I use Vivado 2017.2. It compiled/synthesized fine, but the described problems occurred.

Now I downloaded the image you prepared [1] and ran the makefile. I then simply compiled the server with the command given in its head comment and ran it with ./adc-test-server. Since I don't have gcc installed on Windows, I used Putty to have a quick look at the output and then wrote a Python client to dump the received data into a file. I did not yet write a plotting ability but rather tried to modify the blockdesign in order to to replace the real ADC input by a constant zero on both channels to see if I am getting the contents of the correct memory. Well, I did not receive zeros only but rather (in binary) a lot of ones mixed with a few zeros. I am suspecting a mismatch in memory addresses. The writer has its memory range set to the full range from 0x0000_0000 to 0x1FFF_FFFF. Since I do not yet understand how the FPGA and the server work together for the cyclic selection of memory addresses, I am unable to fix this. Also, do you have an idea for testing the code properly? I guess using something like a counter running at 125MHz to check for gaps in the transmitted data would not work due to the CIC filter.

Thanks for your help so far. It would be really great to get this running. I also tried to understand the approach of Nils Roos, but the relevant code is quite dependend on the RP ecosystem as far as I can see it and it's even harder to grasp than your code. Plus, your code is (at least I hope so) easier to integrate into a custom blockdesign (I am trying to synchronize multiple RPs and have them start sending the ADC data via network at a mutual trigger signal. The synchonization part is running, but I need a way to get the data out.).

Cheers Jonas

[1] https://www.dropbox.com/sh/5fy49wae6xwxa8a/AADbxrBWEtqAGqeEZeY-Y-fga/red-pitaya-debian-8.6-armhf-20161219.zip?dl=1

jonschumacher commented 7 years ago

Hi Pavel,

I am now trying to understand the design. As far as I can see, the signal flow from the ADC to the RAM is as follows:

  1. The ADC core combines both channels into an AXI stream with a width of 32
  2. The Subset Converter core selects the upper 16 bits, effectively reducing the transmitted data to that from ADC port B
  3. The CIC compiler decimates the signal by a factor configured in the address range from 0x4000_0000 to 0x4000_0FFF and supplied by the AXI Configuration core
  4. The AXI Data Width Converter combines four 16bit values into one 64bit value
  5. The RAM Writer core buffers the data in a FIFO and then streams it to the HP0 bus. The target address is set by using a base value from cfg_data and add to it the 20bit internal address. (What puzzles me about the assignment to m_axi_awaddr is, that int_addr_reg with a width of ADDR_WIDTH=20 is concatenated with ADDR_WIDTH zeros, which would result in adding a 40bit value to the 32bit value of cfg_data.)
  6. One server thread reads the current address from the status register and waits until 4MB are present in the memory. If this is the case, the data is copied into a buffer which is then send to the client via network by another thread.

I hope my understanding of the design is correct. I think my biggest troubles are in understanding the configuration register and the server code. Why did you choose to read exactly 4MB in a block? And I seem to be too blind to see, where the desired ringbuffer is implemented. Is it just the wrapping of the 20bit internal address register which is incremented with each sample?

Greetings Jonas

pavel-demin commented 7 years ago

Hi Jonas,

Your understanding of the design is correct.

The main purpose of the adc_test project was to test how fast the ADC samples could be transferred to a remote PC. Here is a link to a post on the Red Pitaya forum with the results of those tests:

http://forum.redpitaya.com/viewtopic.php?t=317&p=2118#p2118

Why did you choose to read exactly 4MB in a block?

The idea was to have a combination of

And I seem to be too blind to see, where the desired ringbuffer is implemented. Is it just the wrapping of the 20bit internal address register which is incremented with each sample?

Yes, it is just the wrapping of the 20bit internal address register.

jonschumacher commented 7 years ago

Hi Pavel,

it took me some time, but I finally got it working. It was not working out of the box and I still don't know why but after connecting all resets to the processing system reset and deleting the packetizer IPs it is. Actually, what are the packetizers for? I looked into the Verilog code but I could not figure out their purpose. And it's working without them.

Thanks for your awesome work and for responding so quickly. I am doing the project for a university and hopefully I will be allowed to publish it. Since it does contain parts of your work: Do you have some preferences for acknowledging your work? I know you chose the MIT licence but your cores do not contain a specific comment.

Cheers Jonas

pavel-demin commented 7 years ago

Hi Jonas,

I'm glad that you've found a working combination of the IP cores and resets.

It was not working out of the box and I still don't know why but after connecting all resets to the processing system reset and deleting the packetizer IPs it is.

After posting my first comment to this issue, I've also modified the code in a similar way: https://github.com/pavel-demin/red-pitaya-notes/commit/b225481f869ca5d5856470d563cca812dc0d2c87

Actually, what are the packetizers for?

I use packetizers to form packets or sequences of a fixed number of data frames. The combination of axis_constant, axis_packetizer was how I changed the cic_compiler decimation rate in my old projects. It sent just one single data frame after a reset. Now, I use axis_variable that sends one data frame when the input data changes.

Do you have some preferences for acknowledging your work?

A reference with a link to this GitHub project would be great.

Best regards,

Pavel