search

joaotavora / sly

Sylvester the Cat's Common Lisp IDE
1.27k stars 145 forks source link

Mac OSX/SBCL: Delayed output on standard output, possibly issue with buffer not getting cleared #426

Open mustiboost opened 3 years ago

mustiboost commented 3 years ago

I just wanted to share some simple input output code with you that I am trying to run on sly/emacs27.1/sbcl on Mac to test clearing of input/output buffers. Unfortunately, it's not working as it should. Here is the code:

(defun ask-input (x)
  (format t x))

(defun square(x)
  (format t "The square of your number is: ~d~%" (* x x)))

(defun test()
  (ask-input "enter a number to square: ")
  (finish-output)
  (square (read)))

When I run (test) I get the prompt "enter a number to square: ", but after entering a number, the repl in sly just returns nil instead of "The square of your number is....". This problem only occurs in sly but not in sbcl that I run from the terminal. It also does not occur in slime. I tried all kinds of (finish-output) and (finish-output nil) in all kinds of places with no effect. Only when I run (test) again does the output from the previous (test) appear right before the new prompt appears. This tells me that some kind of buffer is not cleared...but not sure which buffer.

Here is a sample run: 

CL-USER> (test)
enter a number to square: 45
NIL
CL-USER> (test)
The square of your number is: 2025
enter a number to square:

This issue seems similar or related to issue #347 from what I can tell.

joaotavora commented 3 years ago

This shouldn't happen by default, but have a look at https://joaotavora.github.io/sly/#REPL-output, and try to turn to turn off the dedicated output stream, to see it it makes a difference.

joaotavora commented 3 years ago

Indeed #347 is related, and this seems to be mac specific. I'll see later if we should merge the two issues,

mustiboost commented 3 years ago

This shouldn't happen by default, but have a look at https://joaotavora.github.io/sly/#REPL-output, and try to turn to turn off the dedicated output stream, to see it it makes a difference.

Ok, I turned off the dedicated output stream in ~/slynkrc and the problem went away, but what is the disadvantage of doing that? What am I giving up besides some efficiency? Is it even going to be noticeable? (i.e. maybe slower?) Also, does slime also utilize a dedicated output stream?

joaotavora commented 3 years ago

You're not giving up anything besides efficiency, I think, and i don't know how much, to be honest. It's entirely possible this was created in a time when event processing in Emacs and Lisps wasn't so stable and this would provide a stabler alternative. The dedicated stream is on by default with M-x sly but not M-x sly-connect because when connecting to other hosts you might not have the necessary port readily open.

Is it even going to be noticeable? (i.e. maybe slower?)

You tell me! If you're transferring large amounts of text to your REPL and at the same time trying to evaluate expression and stuff, you might notice it, I guess.

Also, does slime also utilize a dedicated output stream?

Yes, I think so, and under the same conditions. Maybe buffering is tweaked differently on Macs there. I'll have a look later.

mustiboost commented 3 years ago

Got it, thanks! Yes, taking a look at the slime way of dealing with this is definitely the easiest path forward when it comes to fixing this bug.

joaotavora commented 3 years ago

Yes, this is a bug, and I will look into it, though I'm kind of considering just changing the default to "don't make a separate dedicated socket", since this option seems to be more trouble than it's worth. Emacs Lisp, much like JS, runs on a single thread model, so it's not like the Lisp output is coming in truly in parallel. More likely this mechanism was designed for much more ancient versions of Emacs where process communication was not stable.

As I'm not using SLY daily for the moment, it would be nice to have your feedback as to how not making the separate stream performs in general.

Furthermore, with SLY supporting multiple REPLs, the number of sockets being doubled can make the process list become crowded and confusing.

mustiboost commented 3 years ago

Ok, no problem. So far I have not noticed any issues, but I'll report back if I notice any slowdown or weird behavior anywhere.

fermata101 commented 3 years ago

I just ran into this issue on Arch Linux, so I don't think it is macOS-specific. I believe it started happening after I recently did a system update, which included an update of Emacs to the latest git version. Disabling the dedicated output stream does seem to fix the issue for me as well.

SLY version: commit 41f4d650485217aa1f2afa7c159418f103a09231 Emacs version: 28.0.50.149180-1 (2021-07-26) Linux kernel version: 5.13.5

joaotavora commented 3 years ago

@fermata101 the name and version of the Lisp implementation that you're using is useful.

fermata101 commented 3 years ago

Sorry, SBCL 2.1.1.

dieggsy commented 3 years ago

For what it's worth, I think I'm seeing similar odd buffered behavior, even whith slynk:*use-dedicated-output-stream* set to nil.

Sly commit 540a8c5b Allegro 10.1 Enterprise Edition, latest patches updated today Emacs: 27.1 MacOS Catalina 10.15.7

joaotavora commented 3 years ago

similar odd buffered behavior,

Thank you, but this might be too vague. Post a small recipe. Even if not a Emacs -Q from scratch recipe (preffered), an example interaction in the SLY REPL would be very useful. I have access to Allegro 10.1.

dieggsy commented 3 years ago

For example:

CL-USER> (defun this () (princ "that") (sleep 4) (princ "this"))
THIS
CL-USER> (this)

I need to wait the full four seconds before seeing any output.

joaotavora commented 3 years ago

I need to wait the full four seconds before seeing any output.

That's exactly what the sbcl REPL does, too. I dont' think princ is guaranteed in the spec to flush *standard-out* immediately. (but if you can correct me, probably you should post a bug to SBCL, too)

The original problem, OTOH, has to do with (read).

dieggsy commented 3 years ago

Hm, but it's not what the Allegro REPL does - it prints this first, then sleeps, then prints that. Slime reflects Allegro's REPL behavior as well, for whatever that's worth.

If this is indeed unrelated, my bad - I'm happy to file a new issue if needed.

joaotavora commented 3 years ago

Hm, but it's not what the Allegro REPL does - it prints this first, then sleeps, then prints that.

I know. I'm not saying Allegro REPL's is "wrong".

Slime reflects Allegro's REPL behavior as well, for whatever that's worth.

Yes, I know. SLIME has a stream flushing polling loop thread which I think is slightly fishy and looks bad in the thread listing. I removed it in SLY. You could add it back, preferably as a contrib.

If this is indeed unrelated, my bad - I'm happy to file a new issue if needed.

I don't think I'll do much about it, other than re-suggest the contrib. I'm more interested in reproducing and getting to the bottom of the (read) bug, but I have little time for that as well.

joaotavora commented 3 years ago

Wondering if @fermata101 isn't mixing up the two issues, too... @fermata101 what is your recipe?

dieggsy commented 3 years ago

No worries, I appreciate your responsiveness here. Perhaps I'll look into a contrib. Thanks!

dieggsy commented 3 years ago

@joaotavora Sorry to continue adding to this issue, but since we discussed here - after reading https://franz.com/support/documentation/9.0/doc/streams.htm#force-finish-output-2 , it looks like Allegro does a force output for certain functions if interactive-stream-p, which is setfable, returns true.

Doing a (setf (interactive-stream-p *standard-output*) t) seemed to produce the output behavior I want/expect. Not sure if this could be added to sly itself, but it might make sense to at least add to the docs?

joaotavora commented 3 years ago

@joaotavora Sorry to continue adding to this issue, but since we discussed here -

No problem at all.

Very interesting discovery. You can try both: a PR for adding to the docs in some appropriate section, or trying a global setting in SLY itself. Of course for the latter you'd have to test with more implementations. Is interactive-stream-p a CLHS thing, or just Allegro?

dieggsy commented 3 years ago

It appears to be Allegro only. I have yet to find an SBCL equivalent (it looks likely there isn't one). I'm also not sure of the best way to permanently set this on the user side, as it seems the *standard-output* variable is changed after .slynk.lisp is loaded.

EDIT: I'll add that on a regular Allegro terminal REPL, (interactive-stream-p *standard-output*) defaults to t, so it might make sense to add this even if just as an allegro-specific enhancement.

EDIT 2: It does appear that SBCL has an interactive-stream-p, but it doesn't produce the same behavior / isn't setfable.

fermata101 commented 3 years ago

Wondering if @fermata101 isn't mixing up the two issues, too... @fermata101 what is your recipe?

My recipe is just like the original bug report:

CL-USER> (defun test ()
  (format t "Enter num: ")
  (finish-output)
  (let ((x (read)))
    (format t "Square: ~A~%" (* x x))
    (finish-output)))
TEST
CL-USER> (test)
Enter num: 8
NIL
CL-USER> (test)
Square: 64
Enter num:

After disabling dedicated output stream:

CL-USER> (test)
Enter num: 8
Square: 64
NIL
CL-USER>