Closed greghendershott closed 2 years ago
Forward and back expression navigation are meant to be potentially different than just navigating matches. Shrubbery support does something different, for example. So, yes, using that when supplied and falling back to match-based navigation is the intent.
Since drracket:grouping-position
is new, we can change the protocol if that helps resolve a mismatch with Emacs.
Thanks for the quick reply.
EDIT: See below instead.
~I'll need to work on it more to be sure, but:~
~Maybe an optional fail
argument (-> natural? any)
that defaults to (lambda _ #f)
would preserve the current behavior of just returning #f
, while allowing the failing position to be reported in some way the caller wants.~
~However that would change the return contract from (or/c #t #f natural?)
to effectively any
, which isn't ideal.~
~So I could also imagine redesigning this to return something like:~
(or/c
(cons/c 'none natural?) ;the old #f enhanced to be a failure position
'use-s-expression ;the old #t renamed to be more self-evident
natural?) ;a success position
However I need to research the under-documented meaning of "the failure position" wrt Emacs forward-sexp
. I'm still reloading my brain with many details from a year ago. I'll report back.
Forward and back expression navigation are meant to be potentially different than just navigating matches. Shrubbery support does something different, for example.
If a lang lexer has produced tokens that open or close expressions/blocks, then it sure seems like forward/backward matching using those tokens would give the positions of the expressions/blocks.
So when you have time, could you please say more about what group-positions could say differently than matching, and how/why?
Maybe there's a simple example that would make me (and potentially other doc readers) say, "Oh, of course."
Is it something like, the lang uses off-side rule indenting, but doesn't produce open/close tokens?
I think some off-side rule langs tokenize indent as open/close curly braces (maybe Haskell?), which they also allow. But maybe some langs don't want to, or can't, and grouping-positions is for them?
I think I figured out (at least part of) the rationale. I was confused because a year ago I had been thinking of a design where a lang lexer could produce open and close tokens for multi-character keyword lexemes like "begin" and "end", as well as single-char delimiters like parens. However that's not the design here; drracket:paren-matches
and color-lexer
's pairs
argument are all (listof (list/c symbol? symbol?))
.
So I guess the story for such langs (using say "begin" and "end") is that forward/backward-match definitely can't/won't do anything interesting; the idea is to use drracket:grouping-position
. And I guess same story for an off-side rule lang.
To follow up on what grouping-position
should do when it can't move forward/backward:
Above I suggested having it return, not just #f
, but also the failure position. However:
forward-sexp
(or a forward-sexp-function
) is supposed to signal the position that blocked advancing, and the position after it:If unable to move over a sexp, signal `scan-error' with three arguments: a message, the start of the obstacle (usually a parenthesis or list marker of some kind), and end of the obstacle.
What this seems to be saying (setting aside the message, which is N/A), "please give the bounds of the token where it stopped". So it's two positions.
[[[This is then used by Emacs functions like up-list
that call forward-sexp
. That's probably N/A as an end user "move up list" command b/c I can use grouping-position
up for that. But it might help third party Emacs packages continue to work well.]]]
shrubbery-grouping-position
forward/backward doesn't actually return #f
when it can't move --- it returns the same position passed in. That's not what's documented, but, I think it's fine -- maybe even better?Therefore: Combining these two, my wrapper of drracket:grouping-position
can check for it not moving, give the position to get-token-range
to get both positions, and return those to the Emacs front end.
So I think the only open action for this issue, is to clarify the "can't move" return value --- is it supposed to be #f
(as documented) or the position (as shrubbery-grouping-position
seems to do)?
I think I'd prefer the latter. Since it's a brand new spec maybe it would be good to pick/enforce just one.
Otherwise I guess I could defensively code for (or #f (= old new))
.
I left a comment then deleted it quickly; you may have gotten an email. I confused myself for a second, about a "count" argument that's not actually present in drracket:grouping-position
. Please ignore.
Here is what I'm doing to support a forward-sexp-function
in Emacs. It behaves well with shrubbery-grouping-position
. It handles both the documented #f
and the actual "unchanged position" values.
;;; Something for Emacs forward-sexp-function etc.
(define/public (grouping gen pos dir limit count)
(cond
[(<= count 0) pos]
[else
(block-until-updated-thru gen
(case dir
[(up backward) min-position]
[(down forward) max-position]))
(let loop ([pos (sub1 pos)] ;1.. -> 0..
[count count])
(define (failure-result) ;can't/didn't move
(call-with-values (λ () (get-token-range pos)) list))
(match (grouping-position this pos limit dir)
[#f (failure-result)]
[#t 'use-default-s-expression]
[(? number? new-pos)
(cond [(< 1 count) (loop new-pos (sub1 count))]
[(= new-pos pos) (failure-result)]
[else (add1 new-pos)])]))])) ;0.. -> 1..
So if you're fine with users like me doing the defensive thing (I'm fine with that) you could just close this issue (maybe update a phrase in the docs).
So far, it makes sense to me that shrubbery-grouping-position
should change to return #f
when it doesn't move. Could it make sense in some situation to claim success, even when the position doesn't change? Or is the idea of using the same position as "no move possible" better because it avoids that possible distinction and so usefully narrows the space of behavior?
TL;DR: I think the latter, although it's not a strong opinion.
Although I don't want to over-think this: With APIs sometimes I try to think of mistakes people might make, and/or unnecessary "ceremony".
From that point of view:
I think it's better to return the original position -- especially if the text% "primitives" sort of lead you to do that (?) -- instead of asking grouping-position
authors to notice and carefully return #f
instead.
The indentation
and range-indentation
functions return #f
to mean "use default s-expression behavior". So it is a little "weird" that grouping-position
instead uses #t
for this. The "ternary" result is easier to mess up.
So I lean toward having it return the position (changed or not), or #f
to mean "s-expression", and that's it.
Having said all that, on the other hand:
If the number of people writing such things can be counted on one hand, and it's easy to get help, maybe it doesn't matter.
Bugs with navigation in tools are fairly blatant (and in my experience highly likely to be reported :smile:) so it's unlikely mistakes here would subtly fester for long.
On the tool/consuming end, I will probably do something defensive, regardless.
@greghendershott Thanks for the explanation!
I was convinced... but then looked at how the newly extracted Racket-navigation functions will end up providing the implementation of methods like find-up-sexp
and get-backward-sexp
. I think some of those implementations can return the input position as opposed to #f
, at least in some cases. I'm not sure, but if that does happen, then it seems like there's also chance that something relies on the current specific behavior. So, I'm inclined to leave this alone, after all.
I've pushed changes to syntax-color
and gui
to help sort this out.
The syntax-color
package now provides a color-textoid<%>
interface that specifies the subset of text:color<%>
methods that indentation and navigation functions can use in an environment without framework
or racket/gui
. The syntax-color
package also provides the default Racket S-expression indentation and navigation functions that were formerly built into text:racket-mixin
. The text:racket-mixin
implementation calls the ones in syntax-color
; for example, the find-up-sexp
method is implemented using the racket-up-sexp
function that uses only methods in color-textoid<%>
.
The default indentation implementation uses a table of rules that, in general, can be configured as a preference in DrRacket. The syntax-color
layer exports a default configuration and a conversion from a serializable configuration to a function that can be passed to the S-expression indenter from syntax-color
. So, a configuration could be loaded and parsed without using framework
.
Finally, expeditor
now uses the Racket indenter from syntax-color
by default.
I think all those changes might close this issue, but I'm not sure.
Thank you!!
I caught up with these changes this morning, as well as applying some more of your recent commits in expeditor/object.rkt to my similar class. So I have commit 4854c32 and tests against snapshot pass.
I'm still planning to move that to syntax-color-lib
as we discussed. But for the next N days I'd like to dog-food and exercise this more, while it's still in the same repo as Racket Mode's .el and .rkt code that uses it.
Although I'd like to follow up on https://github.com/racket/expeditor/issues/10 I think we can close this one!! :tada:
In expeditor @mflatt defined a
like-text%
object that implements somecolor:text<%>
methods including{forward backward}-match
, IIUC to supply todrracket:indentation
and/ordrracket:range-indentation
functions.I'm trying to understand the relation of these to the
'forward
and'backward
flavors of the newdrracket:grouping-position
.Are they equivalent, or is there some subtle difference?
In some scenarios would a lang implement one using the other?
For a tool providing expression navigation commands, if a lang supplies
drracket:grouping-position
, should a tool prefer that (and{forward backward}-match
are just a get-info default)?Is
grouping-position
for forward/backward merely a preference/convenience --- or, could it be meaningfully different for some lang and a tool should definitely avoid using{forward backward}-match
?Maybe the answers are in the DrRacket and/or expeditor source code and eventually I'll go look, but I wanted to ask the question here since it goes to intent. (Also this might become a FR or PR to add a little explanation/discussion to the docs.)
More context for why I ask:
Emacs has the concept of a
forward-sexp-function
-- a variable the value of which is a function. Theforward-sexp
function uses that if non-nil. So ~= a parameter.(Note:
forward-sexp
is actually forward/backward in one function; there's acount
arg that may be negative. So this is about both directions.)One wrinkle is that if scanning fails, e.g. you can't move forward anymore,
forward-sexp
signals an exception with that failed position. And various Emacs commands built usingforward-sexp
, rely on that -- they might catch that signal and use the position. So this is behavior that aforward-sexp-function
needs to emulate.So I can do the failed-position behavior with something implemented much like
{forward backward}-match
, scanning tokens --- but could that possibly be wrong if a lang suppliesdrracket:grouping-position
?