realtimeradio / souk-firmware

Simons Observatory UK RFSoC Firmware
GNU General Public License v3.0
3 stars 1 forks source link

CPU stops #64

Open sr-cdf opened 6 days ago

sr-cdf commented 6 days ago

Hi @jack-h,

FYI

I can make the CPU stop at will in python by writing to the mixer lo phase increment shared bram using the axil-mmap (in casperfpga.transport_localmem.LocalMemTransport) and then reading from it straight away in the next line:

This code will crash the CPU:

(py3venv) casper@localhost:~$ sudo ~/py3venv/bin/ipython

import souk_mkid_readout
r = souk_mkid_readout.SoukMkidReadout('localhost',configfile='/home/casper/src/souk-firmware/software/control_sw/config/souk-single-pipeline-krm-2G.yaml',local=False)
r.program()
r.initialize()
r_fast = souk_mkid_readout.SoukMkidReadout('localhost',configfile='/home/casper/src/souk-firmware/software/control_sw/config/souk-single-pipeline-krm-2G.yaml',local=True)

i=0
ramname = f'{r.mixer.prefix}tx_lo{i}_phase_inc'  # 'p0_mix_tx_lo0_phase_inc': Sbram:p0_mix_tx_lo0_phase_inc,
phase_addrs_tx = []
phase_addrs_tx += [r_fast.mixer.host.transport._get_device_address(ramname)]

nbytes = r.mixer._n_serial_chans * 4
maxwrite=512
nwrite = (nbytes // maxwrite)

b0=bytes([0]*nbytes) # all 0x00
b1=bytes([255]*nbytes) # all 0xFF

#must write in 512 byte chunks?
for i in range(len(phase_addrs_tx)):
    for j in range(nwrite):
        print(i,j)
        raw1 = b1[j*maxwrite:(j+1)*maxwrite]
        r_fast.mixer.host.transport.axil_mm[phase_addrs_tx[i]+j*maxwrite : phase_addrs_tx[i] +(j+1)*maxwrite] = raw1
        ret1 = r_fast.mixer.host.transport.axil_mm[phase_addrs_tx[i]+j*maxwrite : phase_addrs_tx[i] +(j+1)*maxwrite]
        print(ret1==raw1)

# system prints 0,0 then freezes before the next print

There are no kernel messages in dmesg or journal or syslog, nothing printed to the serial konsole, does not respond over network or serial, just everything stops responding and needs a power cycle.

Note that the system does not crash if i put a time.sleep(0) between the read and the write.

What could be the cause of the crash?

jack-h commented 16 hours ago

@mitchburnett Don't suppose you have any words of wisdom? I don't know if this is fundamental, or some error in the way python is memmapping /dev/mem. I haven't investigated at all yet, but before I start I wondered if you had ever seen this.