Access to tumbleed/uhdl?

[ams]

Do you want access to the Fossil repository? Maybe this could be kept on a branch, of course that is up to you what you prefer! :-)

[nusgart]

I wonder if we might be miscommunication here. I never got things to work on the Arty. Not having a SD card working is a show stopper for that -- I haven't tried doing it with the working MMC controller (or should that be known working MMC card?) though. That might be easy attempt though. But I only have one 2G SD card, and was a bit excited to get SDHC support working since it makes things a bit easier.

Sorry, I wasn't very clear. I forgot to provide some important context -- I was wondering whether those changes are present in the Arty A7 port, and whether they might be one of the factors preventing it from booting. If they are, that could save me a lot of time looking.

Also, I agree that SDHC support would be great.

The way the Lisp Machine boots is to read the microcode from the SD card, it then switches to PROMDISABLE. That will then start loading the LOD partition on the disk. When that is done, it will call LISP-TOP-LEVEL (which is the first Lisp function to be called).

Just out of curiosity, is there a way to set breakpoints on lisp functions in CC? I don't expect there is, but it would be nice.

[ams]

Sorry, I wasn't very clear. I forgot to provide some important context -- I was wondering whether those changes are present in the Arty A7 port, and whether they might be one of the factors preventing it from booting. If they are, that could save me a lot of time looking.

That was sorta what I wanted to check, and nothing of what I could see that would cause things not to boot. I think I will try tomorrow a small experiment where I only use your RAM controller.

Just out of curiosity, is there a way to set breakpoints on lisp functions in CC? I don't expect there is, but it would be nice.

I've never tried it, since you sorta need the definitions of the running system I think. But the (native) CC has support for it.

EDIT: The experiment should at least cancel out issues with the SD card, and other such things on the Arty.

EDIT1: On that note, can you say anything about the state of the RAM controller?

[nusgart]

That was sorta what I wanted to check, and nothing of what I could see that would cause things not to boot. I think I will try tomorrow a small experiment where I only use your RAM controller. EDIT1: On that note, can you say anything about the state of the RAM controller?

The ram controller should work -- it works in simulation and successfully loads the microcode, however, it only works on Xilinx 7-series FPGAs -- I'm not quite sure what you mean.

EDIT: The experiment should at least cancel out issues with the SD card, and other such things on the Arty.

I'm using an SDSC card for my tests, although I bought it off of amazon and it might be sketchy.

Just out of curiosity, is there a way to set breakpoints on lisp functions in CC? I don't expect there is, but it would be nice.

I've never tried it, since you sorta need the definitions of the running system I think. But the (native) CC has support for it.

Fair enough. I'm still using the C version of CC, so I guess that won't work.

[ams]

Great, that was what I wanted to know.

I noticed you moved the video buffer into BRAM, but the size of it seems very small (8912 32-bit words, but the TV buffer needs to be 1MiB) to fit the whole video memory?

[nusgart]

Great, that was what I wanted to know.

I noticed you moved the video buffer into BRAM, but the size of it seems very small (8912 32-bit words, but the TV buffer needs to be 1MiB) to fit the whole video memory?

I didn't change anything there. The video buffer is exactly the right size for a 768x896 bitmap (each pixel is one bit). Also, it's 21504 x 32-bit words.

[nusgart]

..., I never committed those changes since the changes in the vga_display.v weren't playing with my monitor

Oh, sorry, I forgot to mention this, but vga_display.v was that way because I forgot to remove a "temporary" hack. Basically, at one point the only VGA monitor I had was only 720 pixels tall, so I had to change the video mode used. As a side effect, for some reason, that caused the vertical address reset code to mess up, so it would continuously scroll upwards. I didn't bother investigating or fixing that because 1) it only happens when the video mode's resolution is smaller than the CADR's, 2) it was the only way for me to see the other 176 vertical lines, and 3) I knew that I'd eventually get a better monitor, and the port had much larger problems at the time. Unfortunately, when I got a better monitor, I forgot to actually commit the fix. I've done so now.

[ams]

So I've been slowly (life getting in the way) working to get the SDHC stuff working, not yet successful in booting from my SanDisk cards though.

As a change of pace, I tried running the test bench for the X7 RAM controller; but it fails on its tests. Before I dwell on that, do you have any left over changes spelunking somewhere?

EDIT:

Starting VRAM VGA read
vram_vga_read @0064;  2003ns
vram_vga_read @0064 done;  2003ns
vram_vga_read failed 00000000 != 0029cbb8
vram_vga_read @0066;  2323ns
vram_vga_read @0066 done;  2323ns
vram_vga_read failed 0001ca00 != 00492492
vram_vga_read @0068;  2643ns
vram_vga_read @0068 done;  2643ns
vram_vga_read failed 00012400 != 006db6db
Finished VRAM VGA read

[nusgart]

So I've been slowly (life getting in the way) working to get the SDHC stuff working, not yet successful in booting from my SanDisk cards though.

As a change of pace, I tried running the test bench for the X7 RAM controller; but it fails on its tests. Before I dwell on that, do you have any left over changes spelunking somewhere?

No, I've been trying to figure out the vram test failures for a while now. I'm not sure why it's failing, but the VRAM logic is directly copied from ram_controller_lx45.v, so I figured that it would be close enough to working that I could leave that for later. The more important part of that testbench is the SDRAM test, which takes a lot of simulation time — about 129.128 μs — because the MIG needs to perform calibration. The SDRAM tests pass, which is one of the reasons I think that ram_controller_X7.sv works, but it's hard to be sure without a successful boot to lisp.

Also, on another note, I think a recent change broke the SD controller for SDSC cards -- the old bitstream seems to work so I don't think its the card. Could you test and see if it works for you? Edit: I just tested again and it seems to work now??? I'm going to look at this more closely

[ams]

Will report -- when I figure out why I'm not getting strange messages.

[DRC PDRC-34] MMCM_adv_ClkFrequency_div_no_dclk: The computed value 200.000 MHz (CLKIN1_PERIOD, net pll_clk3) for the VCO operating frequency of the MMCME2_ADV site MMCME2_ADV_X1Y1 (cell rc/u_dram_memif/u_dram_memif_mig/u_ddr3_infrastructure/gen_mmcm.mmcm_i) falls outside the operating range of the MMCM VCO frequency for this device (600.000 - 1200.000 MHz). The computed value is (CLKFBOUT_MULT_F * 1000 / (CLKINx_PERIOD * DIVCLK_DIVIDE)). Please run update_timing to update the MMCM settings. If that does not work, adjust either the input period CLKINx_PERIOD (40.000000), multiplication factor CLKFBOUT_MULT_F (8.000000) or the division factor DIVCLK_DIVIDE (1), in order to achieve a VCO frequency within the rated operating range for this device.
[DRC PDRC-43] PLL_adv_ClkFrequency_div_no_dclk: The computed value 400.000 MHz (CLKIN1_PERIOD, net sysclk_IBUF_BUFG) for the VCO operating frequency of the PLLE2_ADV site PLLE2_ADV_X1Y1 (cell rc/u_dram_memif/u_dram_memif_mig/u_ddr3_infrastructure/plle2_i) falls outside the operating range of the PLL VCO frequency for this device (800.000 - 1600.000 MHz). The computed value is (CLKFBOUT_MULT_F * 1000 / (CLKINx_PERIOD * DIVCLK_DIVIDE)). Please adjust either the input period CLKINx_PERIOD (10.000000), multiplication factor CLKFBOUT_MULT_F (4) or the division factor DIVCLK_DIVIDE (1), in order to achieve a VCO frequency within the rated operating range for this device.

EDIT: Was Vivado that change my MIG settings ... sighs.

[ams]

Just for record, could you hook up your board, and then run (c)CC against it? I just want to verify that we are on the same page.

[ams@argon zl-cc]$ ./ccc
/dev/ttyUSB0: No such file or directory
ready
> r
A  = 00000000774 (000001fc)
M  = 00000000774 (000001fc)
PC = 00000000613 (0000018b)                            
MD = 00000000000 (00000000)
VMA= 00000000774 (000001fc)
disk-state 3 (0x0003)
bd-state   20 (0x0014)
mmc-state  32 (0x0020)

Scratch my though about it being DISK-OP-CONTINUED for now. Need to verify.

EDIT:

611 0002024044010310:    ((VMA-START-WRITE) ADD VMA A-1)
612 0200000000240244:    (JUMP-IF-PAGE-FAULT ERROR-PAGE-FAULT)
DISK-OP-CONTINUED:
613 0200000006320447:    (CALL-XCT-NEXT DIV)
614 0002740000450050:    ((M-TEMP-2) A-HEADS-TIMES-BLOCKS)

And stepping a bit,

> s
flags1: 8200 (waiting ssdone ) flags2: 001b (vmaok nop ) PC = 00000000611 (00000189)
> s
flags1: 8200 (waiting ssdone ) flags2: 000b (vmaok ) PC = 00000000612 (0000018a)
> s
flags1: 8200 (waiting ssdone ) flags2: 000b (vmaok ) PC = 00000000613 (0000018b)
> s
flags1: 8200 (waiting ssdone ) flags2: 000d (jcond vmaok ) PC = 00000000614 (0000018c)
> s
flags1: 8200 (waiting ssdone ) flags2: 001b (vmaok nop ) PC = 00000000611 (00000189)
> s
flags1: 8200 (waiting ssdone ) flags2: 000b (vmaok ) PC = 00000000612 (0000018a)
> s
flags1: 8200 (waiting ssdone ) flags2: 000b (vmaok ) PC = 00000000613 (0000018b)
> s
flags1: 8200 (waiting ssdone ) flags2: 000d (jcond vmaok ) PC = 00000000614 (0000018c)
> s
flags1: 8200 (waiting ssdone ) flags2: 001b (vmaok nop ) PC = 00000000611 (00000189)
> 

So seems we are spinning. Resetting the system (via (c)CC using the x command) does get it into a weird state:

PAGE-0-PARITY-FIX:
310 0002024042010310:    ((VMA-START-READ) ADD VMA A-1)
311 0200000000240244:    (JUMP-IF-PAGE-FAULT ERROR-PAGE-FAULT)
312 0000025064010030:    ((MD-START-WRITE) SETM MD)
313 0200000000240244:    (JUMP-IF-PAGE-FAULT ERROR-PAGE-FAULT)
314 0202364003100241:    (JUMP-LESS-THAN VMA A-400 PAGE-0-PARITY-FIX)
315 0002140060010050:    ((MD) A-4)
316 0602201444030011:    ((VMA-START-WRITE) DPB (Byte-field 1 11) M-ONES A-5)

> s
flags1: 8200 (waiting ssdone ) flags2: 001b (vmaok nop ) PC = 00000000310 (000000c8)
> s
flags1: 8200 (waiting ssdone ) flags2: 000f (jcond vmaok ) PC = 00000000311 (000000c9)
> s
flags1: 8200 (waiting ssdone ) flags2: 000b (vmaok ) PC = 00000000312 (000000ca)
> s
flags1: 8200 (waiting ssdone ) flags2: 000b (vmaok ) PC = 00000000313 (000000cb)
> s
flags1: 8200 (waiting ssdone ) flags2: 000b (vmaok ) PC = 00000000314 (000000cc)
> s
flags1: 8200 (waiting ssdone ) flags2: 000d (jcond vmaok ) PC = 00000000315 (000000cd)
> s
flags1: 8200 (waiting ssdone ) flags2: 001b (vmaok nop ) PC = 00000000310 (000000c8)
> 

[nusgart]

My fpga successfully loads the microcode (I'm using an SDSC card). It used to get stuck in FINDCORE, but right it's getting stuck at PC=21646 (DISK-RECALIBRATE-WAIT).

Edit: just pushed my current code -- there's an important VGA fix that allows 1080p to work (1080p lines are 2200px long but the horizontal counter was 11 bits).

[ams]

Great, then I got the same issue.

flags1: 1200 (promdisable ssdone ) flags2: 000f (jcond vmaok ) PC = 00000020540 (00002160)
> s
flags1: 1200 (promdisable ssdone ) flags2: 000b (vmaok ) PC = 00000020541 (00002161)
> s
flags1: 1200 (promdisable ssdone ) flags2: 000f (jcond vmaok ) PC = 00000020542 (00002162)
> s
flags1: 1200 (promdisable ssdone ) flags2: 000b (vmaok ) PC = 00000020543 (00002163)
> s
flags1: 1200 (promdisable ssdone ) flags2: 000f (jcond vmaok ) PC = 00000020544 (00002164)
> s    
flags1: 1200 (promdisable ssdone ) flags2: 000f (jcond vmaok ) PC = 00000020545 (00002165)
> s
flags1: 1200 (promdisable ssdone ) flags2: 000b (vmaok ) PC = 00000020546 (00002166)

EDIT: But that is the FINDCORE loop, in my case. The PC you gave, is it octal or hexadecimal?

EDIT2:

FINDCORE1:
20537 0213664205520741:  (CALL-GREATER-OR-EQUAL VMA A-V-PHYSICAL-PAGE-DATA-END FINDCORE2)
20540 0200000003730644:  (CALL-IF-PAGE-FAULT ILLOP)
20541 0026443000310310:  ((M-B) ADD M-B 551@A)
20542 0213543205540243:  (JUMP-EQUAL M-B A-FINDCORE-SCAN-POINTER FINDCORE3)
20543 0600125001550740:  ((M-TEM) LDB (Byte-field 20 0) MD A-ZERO)
20544 0016555442510310:  ((M-T VMA-START-READ) ADD M-TEM A-V-PAGE-TABLE-AREA)
20545 0242315605360043:  (JUMP-EQUAL-XCT-NEXT M-TEM 1046@A FINDCORE0)
20546 0007201207210314:  ((A-COUNT-FINDCORE-STEPS) ADD ALU-CARRY-IN-ONE M-ZERO A-COUNT-FINDCORE-STEPS)
20547 0200000003730644:  (CALL-IF-PAGE-FAULT ILLOP)
20550 1400025003600140:  (DISPATCH (Byte-field 3 0) MD D-FINDCORE ILONG)
20551 0016603042010310:  ((VMA-START-READ) ADD M-B A-V-PHYSICAL-PAGE-DATA)
FINDCORE2:
20552 0100100000310050:  (POPJ-AFTER-NEXT (M-B) A-ZERO)
20553 0016603042010310:  ((VMA-START-READ) ADD M-B A-V-PHYSICAL-PAGE-DATA)

[nusgart]

The PC I gave is octal, directly from cc (I use a slightly modified version of the code that was part of usim)

[ams]

What did you change to get it to DISK-RECALIBRATE-WAIT? Right now I just want to get to the exact same point as you are, then we can at least be on the same wave length. Right now I'm using the same tree as you, with the same XDC, but with cores regenerated for Vivado 2020.2, and vga_display.v changes reverted since they do not play with my monitor.

Things getting stuck in DISK-RECALIBRATE don't make much sense to me.

EDIT: While doing some quick verification between the Pipistrello and the A7; I noticed that the run lights are shifed one run light position on the A7 compared to the Pipistrello.

EDIT: This might mean that the video memory is slightly misaligned somehow...

[nusgart]

What did you change to get it to DISK-RECALIBRATE-WAIT? Right now I just want to get to the exact same point as you are, then we can at least be on the same wave length.

I think I must have bumped the SD card (or something) when that was happening, I've gone to test again and it isn't happening any more. I'm back to getting stuck in the usual findcore loop, and I've attached a trace log (it has just over 200 steps) to this comment as trace.log.

Right now I'm using the same tree as you, with the same XDC, but with cores regenerated for Vivado 2020.2, and vga_display.v changes reverted since they do not play with my monitor.

If you have a 1080p monitor, try using the newest vga_display changes (make sure you give it a 148.5 MHz vga_clk). The previous ones didn't work because I forgot to increase the number of bits in the horizontal counter (a 1080p line is 2200 pixels, which causes problems with a 11 bit counter...).

EDIT: While doing some quick verification between the Pipistrello and the A7; I noticed that the run lights are shifed one run light position on the A7 compared to the Pipistrello.

EDIT: This might mean that the video memory is slightly misaligned somehow...

No idea.

[ams]

If you have a 1080p monitor, try using the newest vga_display changes (make sure you give it a 148.5 MHz vga_clk). The previous ones didn't work because I forgot to increase the number of bits in the horizontal counter (a 1080p line is 2200 pixels, which causes problems with a 11 bit counter...).

Alas, I have a old Dell.

[ams]

I think I must have bumped the SD card (or something) when that was happening, I've gone to test again and it isn't happening any more. I'm back to getting stuck in the usual findcore loop, and I've attached a trace log (it has just over 200 steps) to this comment as trace.log.

I guess that is good, we at least have something that behaves the same.

[ams]

Here is a screenshot from the Arty:

And here from the Pipistrello:

[nusgart]

That is kind of odd. I have no idea why it would do that, but it's possible that there is an alignment issue or that some other strange thing is happening.

On another note, do you have any way of checking 1) how much RAM the microcode thinks there is, and 2) how much it thinks is free? I'm fairly certain that these are stored in the SYS-COM area, but my attempts to implement a main-memory read function have failed (actually, I think anything to do with the MD or VMA registers seems not to work with CC -- will test more)

[ams]

1) how much RAM the microcode thinks there is

That would be SYS:%SYS-COM-MEMORY-SIZE (divide by 1024. to get in in KW). Reading A-V-SYSTEM-COMMUNICATION-AREA + the index of %SYS-COM-MEMORY-SIZE in the SYSTEM-COMMUNICATION-AREA-QS table (in A memory) should give you that. I'll try it later.

2) how much it thinks is free?

That is slightly more complicated since it needs to account for various areas:

;Returns the number of words of free space                                                                                                                    
(DEFUN GET-FREE-SPACE-SIZE ()
  (* (LOOP FOR I FROM (// (+ (REGION-ORIGIN INIT-LIST-AREA) (REGION-LENGTH INIT-LIST-AREA))
                          %ADDRESS-SPACE-QUANTUM-SIZE)
                 BELOW (// VIRTUAL-MEMORY-SIZE %ADDRESS-SPACE-QUANTUM-SIZE)
           COUNT (ZEROP (AREF #'ADDRESS-SPACE-MAP I)))
     %ADDRESS-SPACE-QUANTUM-SIZE))

I think anything to do with the MD or VMA registers seems not to work with CC -- will test more)

Is the machine stopped?

EDIT: If things are misaligned somehow, it would explain why say FINDCORE1 is looping.

[nusgart]

1. how much RAM the microcode thinks there is

That would be SYS:%SYS-COM-MEMORY-SIZE (divide by 1024. to get in in KW). Reading A-V-SYSTEM-COMMUNICATION-AREA + the index of %SYS-COM-MEMORY-SIZE in the SYSTEM-COMMUNICATION-AREA-QS table (in A memory) should give you that. I'll try it later.

1. how much it thinks is free?

That is slightly more complicated since it needs to account for various areas:.....

Ok, that confirms what I thought. I just want to verify that the microcode correctly detects the amount of RAM available - there could be some edge case that causes it to fail to detect the correct amount of ram and then get stuck thrashing (although I would expect it to error out if there wasn't enough for the wired pages instead of getting stuck in an infinite loop).

I think anything to do with the MD or VMA registers seems not to work with CC -- will test more)

Is the machine stopped?

No, I had messed up and forgotten that a spy bus write to VMA wouldn't actually trigger a memory cycle. I have no idea why I forgot that.

EDIT: If things are misaligned somehow, it would explain why say FINDCORE1 is looping.

Possibly, but I'm not sure.

[ams]

I'm slightly stuck with the DDR3 (my understanding of that is very weak); been trying to even get the examples working by getting into trouble with Vivado an unexplainable errors.

[ams]

So managed to get something working, but the code doesn't successfully read back memory. Seems that MIG / Arty A7 are immensely peculiar with their settings. The mig.prj at Digilent isn't working for me so far. The clock is supposed to be 166.667 MHz, according to some. Other say 100 MHz. So confusing.

ddr3.txt

Finally got it (the above file) working (sys_clk_i at 166.6667 MHz, and ref_clk_i at 200 MHz).

[ams]

Question, the sysclk you generated, that has a active high reset signal, but AFAIU the MIG wants a active low on the sys_rst line by default?

EDIT:

And the MIG core does seem to be active-low.

https://github.com/nusgart/uhdl/blob/df7c90bd1f5a04c20a6bcb2a15cb450895b1a957/cadr.srcs/sources_1/ip/dram_memif/mig_a.prj#L26

[nusgart]

Question, the sysclk you generated, that has a active high reset signal, but AFAIU the MIG wants a active low on the sys_rst line by default? ....

You seem to be misunderstanding the reset logic in ram_controller_X7. The reset signal is active low, and it does get driven low after a reset. However, an important point is that it needs to be held low for some minimum amount of time (I don't remember the exact number of cycles), so the reset controller holds it low for 65535 cycles. I know that the reset logic and clock generation work properly because ddr calibration completes successfully (getting it to work took longer than I would like to admit back when I was getting started with FPGAs).

So managed to get something working, but the code doesn't successfully read back memory. Seems that MIG / Arty A7 are immensely peculiar with their settings. The mig.prj at Digilent isn't working for me so far. The clock is supposed to be 166.667 MHz, according to some. Other say 100 MHz. So confusing.

ddr3.txt

Finally got it (the above file) working (sys_clk_i at 166.6667 MHz, and ref_clk_i at 200 MHz).

Use exactly the files I provided -- sys_clk_i (= sysclk) will be at 100 MHz because that's what the Arty A7 provides. I cannot guarantee proper functioning otherwise. Do note that I have the Arty A7-35t not the Arty A7-100T, and my priority is to get this functioning on my board, but everything other than the pin assignments should be the same. Try not to change anything else related to the DDR3 SDRAM if possible.

[ams]

Question, the sysclk you generated, that has a active high reset signal, but AFAIU the MIG wants a active low on the sys_rst line by default? ....

You seem to be misunderstanding the reset logic in ram_controller_X7. The reset signal is active low, and it does get driven low after a reset. However, an important point is that it needs to be held low for some minimum amount of time (I don't remember the exact number of cycles), so the reset controller holds it low for 65535 cycles. I know that the reset logic and clock generation work properly because ddr calibration completes successfully (getting it to work took longer than I would like to admit back when I was getting started with FPGAs).

Roger that, I was just confused. DDR3 is something I've always been struggling with getting it working.

So managed to get something working, but the code doesn't successfully read back memory. Seems that MIG / Arty A7 are immensely peculiar with their settings. The mig.prj at Digilent isn't working for me so far. The clock is supposed to be 166.667 MHz, according to some. Other say 100 MHz. So confusing. ddr3.txt Finally got it (the above file) working (sys_clk_i at 166.6667 MHz, and ref_clk_i at 200 MHz).

Use exactly the files I provided -- sys_clk_i (= sysclk) will be at 100 MHz because that's what the Arty A7 provides. I cannot guarantee proper functioning otherwise. Do note that I have the Arty A7-35t not the Arty A7-100T, and my priority is to get this functioning on my board, but everything other than the pin assignments should be the same. Try not to change anything else related to the DDR3 SDRAM if possible.

Don't worry, this was just for my own sake trying to get a better understanding of how the DDR3 parts works and seeing where to hook up chipscope since the weird way the run-lights are getting shifted around (easier in a trivial example than big one, and synth times). The boards should be pin compatible, yes (I checked the XDC files for the two, and they are exactly the same at least for non-DDR3 pins).

[ams]

@nusgart Hi, been a while ... I've been busy with restoring System 98 -- which I've managed to get working (and also into a state that it can get recompiled); I'm also in s state where I have a System 99 "cold load" working (its the image that loads the fully system) and currently working towards getting the rest of System 99 working.

Have you done anything on uhdl?

[ams]

And now System 99 is fully loaded and spinning; will publish something in the next few days or so.

[nusgart]

I haven't really managed to get much done in the last while. I've been busy with grad school, and I still haven't been able to figure out the problems preventing the arty board from booting. I suspect that the MiSTer branch might actually be easier to get working. My plan was to write an HPS-visible interface on the light-weight h2f bridge so that the hps could act as a FEP to aid in debugging, but I just haven't had time to do much.

[ams]

I've not done anything on the HDL side; mostly concentrating on the Lisp Machine system it self; when I can make new releases from scratch, I'll ping -- at that point I'd like to get back into the HDL mess.

Best of luck with your grad studies!

[ams]

Thought I'd pop in and see if anything new has happened as of late?

System 99 was fully restored, and a new series has been rolled, System 300, with bunch of fixes and some new features. I've not done anything serious on the HDL side (I did attempt porting the schemtics to VHDL as a learning experience, but it has stalled). The simulator now works with SDL2 as well as X11.

Chaosnet on the global Chaosnet networks is very reliable, and some (simulated) Lisp Machines are online. I've been wanting to get back to HDL stuff for a while, but still struggle with the more complicated aspects (DDR3 ... sighs).