Koheron / koheron-sdk

SDK for FPGA / Linux Instruments
https://www.koheron.com/software-development-kit/
Other
101 stars 41 forks source link

Network transfers break after updating instrument (Alpha250, HTTP Bad Request) #586

Closed a-panella closed 11 months ago

a-panella commented 11 months ago

Hi,

SDK Version: 22.04, image version: 16.04, precompiled image

I'm using an Alpha250 and am attempting to run a custom design on one.

I have edited adc-dac-bram example to connect to my Alpha250 (static IP). The Alpha250 is running the 16.04 reference image. I can ping it and ssh to it. So network connectivity itself is fine.

If I try running the example software adc-dac-bram, with only the IP address changed to match my board, then connection is refused, and it seems port 36000 is used as default. requests.exceptions.ConnectionError: Failed to connect to 192.168.111.2:36000 : [Errno 111] Connection refused

Examining nginx config on the board /etc/nginx/sites-enabled/nginx-server.conf shows that, in this image, it listens to port 80 only, not 36000. I modified this file on the image to listen on port 36000 also, after which I can successfully connect and "run" the instrument

It seems that it fails when I try to "connect" to the instrument with test software.

It fails when checking version of server. I.e koheron.py --> check_version. This is due to assert failure on reserved region of packet payload not being zero:

Traceback (most recent call last):
  File "<REDACTED....>.py", line 15, in <module>
    client = connect(host, 'adc-dac-bram', restart=True)
  File "/home/server/.local/lib/python3.10/site-packages/koheron/koheron.py", line 54, in connect
    client = KoheronClient(host)
  File "/home/server/.local/lib/python3.10/site-packages/koheron/koheron.py", line 259, in __init__
    self.check_version()
  File "/home/server/.local/lib/python3.10/site-packages/koheron/koheron.py", line 267, in check_version
    server_version = self.recv_string(check_type=False)
  File "/home/server/.local/lib/python3.10/site-packages/koheron/koheron.py", line 406, in recv_string
    return self.recv_dynamic_payload().decode('utf8')
  File "/home/server/.local/lib/python3.10/site-packages/koheron/koheron.py", line 368, in recv_dynamic_payload
    assert reserved == 0
AssertionError

I have checked the packets in wireshark, they look OK until the Alpha250 reports a HTTP Bad Request, presumably because the payload it received failed the assert reserved == 0 condition.

Commenting out the check just causes the software to hang on the next dynamic payload, which happens to be load_instrument. So simply ignoring this check will not work, as the presumed packet misalignment will stay broken

Can I please get a confirmation on the correct ports to use to run an example? I am running Ubuntu 22.04 on my dev machine (and originally tested this with Koheron SDK branch=22.04, but when I got the above error, I tried Koheron SDK branch=master as well, got same results)

Thanks for your time

a-panella commented 11 months ago

Note: Other issues have listed Ubuntu 22.04 as being broken due to internal python libraries... I will downgrade my docker container to 20.04 and see if this fixes the issue (see here: https://github.com/Koheron/koheron-sdk/issues/542)

tvanderbruggen commented 11 months ago

Hi,

Yes this is probably due to GLIBC incompatibility between Ubuntu 22.04 and previous version which breaks cross-compilation.

If you SSH to the board and run journalctl -u koheron-server -n 200, you should see something like GLIBC_2.33 not found.

You need to cross-compile your instrument from a version of Ubuntu earlier than 22.04 for it to execute on the 16.04 image. As you said if you have a container running 20.04 that should do the trick ...

If you want to run from the branch 22.04, you also need to build an image with Ubuntu 22.04 and flash it on an SD card. To build the image run the next commands from the branch 22.04:

make CONFIG=...../config.yml os
sudo make CONFIG=...../config.yml image
a-panella commented 11 months ago

Thanks for the response. I will go with option A for now, as I am using Vivado 2023.1, I cannot build OS/image due to deprecation of vivado tool hsi in this version of Vivado. I will attempt to use original 16.04 image with a Koheron branch=20.04 build. Will close issue if the python tests can communicate

a-panella commented 11 months ago

I can confirm that the python software can now communicate with the Koheron board. I was using 16.04 image, and 20.04 branch of Koheron SDK. Note: you may need to update koheron using pip install --upgrade koheron if there is a client/server mismatch

Closing this issue.

jefedufeu commented 8 months ago

Hello,

Have you tried anything to fix the hsi? I saw there's a thread about migrating to vivado 2020 but it seems to have stopped...

Thanks

a-panella commented 8 months ago

@jefedufeu My solution ended up being to use an older version of Vivado for the OS build only, and using Vivado 2023.2 for FPGA/bitstream/BD changes. It seems for the initial base design I was using (adc-dac-bram), it built fine in 2023.2. Flashing the OS was something I needed to do only once to fix GLIBC issue mentioned above

jefedufeu commented 8 months ago

Hello,

Thanks but I'm not sure to have well understand. As I understand you made your own OS with make ... os / img command ? From a Ubuntu 18 or 20 ? Am I right ? And then you use a Ubuntu 22 and Vivado 2023 to build fpga design via : make ... run ? I am very interested but I'm sorry I'm new with this SDK so I'm not mastering everything Thanks again,

Jef

a-panella commented 8 months ago

As I understand you made your own OS with make ... os / img command ? From a Ubuntu 18 or 20 ? Am I right ?

Yes, I used Ubuntu 20.04 + Vivado 2017.2 to make OS with make ... os / img

And then you use a Ubuntu 22 and Vivado 2023 to build fpga design via : make ... run ?

Yes, I think I used Ubuntu 20.04 + Vivado 2023.2 to make OS with make ... run