Agon locks up when POS and/or VPOS is used with VDU 31

[tonedef71]

I presumed this was an issue with the VDP, but it was suggested that the issue might need to be fixed in BBC BASIC, so I am re-submitting the issue here.

Any of the following lines of code in BASIC or BASIC ADL will cause the Agon to lock-up:

10 VDU 31, POS, VPOS
20 VDU 31, 0, VPOS
30 VDU 31, POS, 0

The VDP should handle the evaluation of POS and VPOS properly so that the proper X and Y coordinates are passed to VDU 31.

[breakintoprogram]

Thanks. I'll take a look at that next week.

[breakintoprogram]

The issue here is that VPOS calls GETSCR, which gets the current cursor coordinates, does a VDU 23 to fetch the details from the VDP. Nested VDU calls are not permitted by the VDP by design.

; GETCSR: return cursor position in x=DE, y=HL
;
GETCSR:         PUSH    IX          ; Get the system vars in IX
            MOSCALL mos_sysvars     ; Reset the semaphore
            RES 0, (IX+sysvar_vpd_pflags)
            VDU 23
            VDU 0
            VDU vdp_cursor
$$:         BIT 0, (IX+sysvar_vpd_pflags)
            JR  Z, $B           ; Wait for the result
            LD  D, 0
            LD  H, D
            LD  E, (IX + sysvar_cursorX)
            LD  L, (IX + sysvar_cursorY)            
            POP IX          
            RET 

So for example VDU 31, POS, 5 ends up being sent to the VDP as VDU 31, 23, 0, &82, 5, which is why it crashes. There may not be a straightforward solution to this.

[tonedef71]

@breakintoprogram Is it a problem with not being able to maintain two different versions of the same state on the stack?

[stevesims]

this issue, fundamentally, is two different things

first of all, @tonedef71 an important thing to understand here is that an Agon is actually two separate computers working together, the eZ80 running MOS and BASIC, and an ESP32 running the VDP (which provides a VDU command interpreter).

the comms protocol between these two computers is fairly simplistic, so it's not possible to interleave VDU commands

as @breakintoprogram points out, right now VDU 31, POS, 5 will end up getting sent to the VDP as VDU 31, 23, 0, &82, 5 - the 23, 0, &85 part of that being an attempt to request the cursor screen position. what the VDP receives is a VDU 31 command with arguments 32,0, and then a character &82 (which it will display on screen) and a VDU 5, which enables the graphics cursor...

meanwhile the GETSCR routine on the eZ80 is sitting there waiting for bit zero of the VDP flags to be set, which would indicate that a response has been received from the VDU 23,0,&85 command it attempted to send. this response will never come, as the VDP has not attempted to process that command, since it didn't receive that command - it got lost

in this example the VDP is never going to send an appropriate response packet, and therefore bit zero of the VDP flags byte is never going to get set. the fix for the lockup that's being seen would be for the GETSCR routine to somehow stop its infinite loop... a simplistic solution therefore is to make the loop more sophisticated and to provide an alternative timeout exit. NB this issue affects all similar routines

writing several iterations of this response has left me with an idea. an imperfect one, for sure, maybe slightly crazy, but it's an idea. :grin:

when a situation like this occurs, the VDP will be left forever waiting for a command from the eZ80, and one will never come.

what the VDP could do then is detect that it's not received any commands for a while and send a "kick" message to MOS. that message could set all of the VDP protocol bits, and thus unlock bits of code that are waiting for a semaphore bit to be set. the kick message could be a variant of the "general poll" message packet

a more sophisticated version of this would have routines like GETSCR detecting that a true response hasn't been received (that it has instead been kicked) and to re-try their VDU command. the VDP at that point should have timed out any commands it may think were pending arguments, so the retry should work.

[tonedef71]

@stevesims Thank you for the detailed explanation of what is happening behind the scenes; something along the lines of a communication protocol deadlock. Your thoughts on circumventing the issue on the VDP side are interesting. Are there other BBC BASIC to VDP scenarios in which this extra level of consideration may be warranted, or is the scenario we are discussing here a rare outlier?

Can the implementation of the BBC BASIC interpreter be enhanced to not do the VDU interleave. I'm thinking about the BBC BASIC interpreter precomputing the POS and VPOS variables separately before invoking the VDU 31 x, y statement; of course, the BBC BASIC interpreter would first need to recognize that the POS and/or VPOS exist within the context of a VDU 31 x,y construct. BBC BASIC already has special handling in its code to preserve the order of operations for unary and binary operations; perhaps a variation of that technique can be coded to circumvent an interleaved call to VDU. I'm not sure about all of the obstacles which need to be surmounted for this approach.

[stevesims]

@tonedef71 an issue here that prevents a full solution is that the BASIC interpreter cannot always prevent the interleave, it can only attempt to mitigate it.

As a way of illustrating this, consider that these two programs will essentially perform identically:

10 VDU 31, 4, 2

10 VDU 31
20 VDU 4
30 VDU 2

each will send the exact same three bytes to the VDP, and be interpreted identically by the VDP as a single command.

BASIC has no understanding, and no way of understanding, what command sequences the VDP understands.

you're right that the interpreter could recognise the usage of POS or VPOS and try to circumvent interleaved calls. indeed in an earlier draft of my answer from yesterday I had included that as a suggestion :grin: but this can only ever be a partial solution.

if we take the above example programs and replace in the use of POS, we get the following:

10 VDU 31, POS, 2

a modified BASIC interpreter could ensure on seeing that statement that the byte stream to the VDP will be 23,0,&82,31,<result-of-POS>,2

but the same wouldn't be possible with the second example...

10 VDU 31
20 VDU POS
30 VDU 2

as in this example the POS isn't part of a single VDU statement, the interpreter simply can't know that the POS value needs to be fetched before line 10. (bear in mind that other lines of code could be present between lines 10 and 20)

this second example would still result in the current deadlock situation even with a smarter BASIC interpreter.

so yes - we should try to prevent the interleaving of commands as much as practical, and "hoisting" the command to get values of POS or VPOS to occur before statements that use them as an argument would help

my other idea is about preventing the deadlock, allowing the system to recover

this issue doesn't seem to affect the use of POS or VPOS with some other statements that are, essentially, VDU command wrappers such as MOVE, DRAW and PLOT

[tonedef71]

but the same wouldn't be possible with the second example...

10 VDU 31
20 VDU POS
30 VDU 2

as in this example the POS isn't part of a single VDU statement, the interpreter simply can't know that the POS value needs to be fetched before line 10. (bear in mind that other lines of code could be present between lines 10 and 20)

To accommodate multiline VDU 31 sequences without interleave, I guess the "waiting game" logic moves from the VDP to the BBC BASIC interpreter. When the interpreter encounters a 31 inside of a VDU statement (and it is not currently in a wait state for another multibyte VDU sequence), a new multi-byte wait state begins. The 31 signals the wait for the very next two VDU bytes (the X coordinate and the Y screen coordinate). The VDU 31 command has to wait for the next two bytes sent over VDP. If the two expected VDU bytes are not subsequently accounted for, then BBC BASIC can abort sending that VDU 31 sequence to the VDP altogether. A VDU POS and/or a VDU VPOS would legitimately satisfy the pending X and Y byte coordinates that VDU 31 is waiting for, and the VDU 31 wait state gets cleared.

this issue doesn't seem to affect the use of POS or VPOS with some other statements that are, essentially, VDU command wrappers such as MOVE, DRAW and PLOT

Interesting. It would be interesting to understand how BBC BASIC is handling VPOS and POS inside of the MOVE, DRAW and PLOT commands.

[breakintoprogram]

Okay I think there's a way around this. I think I can preprocess the VDU command and write the results to the VDP at the end.

Interesting. It would be interesting to understand how BBC BASIC is handling VPOS and POS inside of the MOVE, DRAW and PLOT commands.

It's not an issue as the values are evaluated before they get written out to the VDP.

[breakintoprogram]

The solution was to write out the VDU stream first to a buffer in BBC BASIC, then write that buffer out to the VDP at the end of evaluation.

[stevesims]

@breakintoprogram nice work :grin:

as noted in my earlier comments, the solution of writing the VDU stream to a buffer will only work for BASIC VDU statements that contain complete VDU commands. it is still possible to have a program that will cause the same errors as originally reported simply by splitting the VDU command across multiple lines. your current solution is great tho, and will work with the majority of examples

the true solution to this is to send VDU commands for things such as VPOS and POS via a separate command stream and have them handled by a separate VDU command processor on the VDP. of course at present this isn't possible with the current MOS/VDP setup, as we only have a single command stream.

work is however being done by @turbovega on a new bi-directional protocol between MOS and the VDP. part of that work should allow for separate command streams. once that is in place we can look to modify BASIC to use a separate stream for info commands such as POS/VPOS, thus providing a more complete solution. this should be able to be implemented in a backwards-compatible manner

[breakintoprogram]

@stevesims thanks. It is now only making a single call to MOS to output the data stream using RST 18h, so was worth doing anyway.

[stevesims]

cool - yeah - more use of RST 18h is definitely good :grin:

there's definitely some interesting enhancements and optimisations that can potentially be made in several corners of the system. more use of RST 18h is definitely one of them - and indeed a more efficient RST18h implementation has been experimented with. that may well end up being part of the bi-directional packet protocol.