Dealing with name clashes for ?? introduced in the latest version of Racket

[emina]

Racket v6.90.0.24 exports ?? as a special identifier (PR 1803).

Unfortunately, the Rosette language that's built on Racket also has a ?? form, so Rosette no longer works on the HEAD version of Racket (issue 97, latest build). For example, these tests no longer compile.

If we change the affected parts of the Rosette implementation to use a custom language that explicitly excludes ?? from Racket, then Rosette will be broken on every earlier version of Racket.

If we change the Rosette language to rename ?? to something else, then all Rosette code will be broken.

Is there any way to have our cake and eat it too in this case?

Thanks!

[rfindler]

Sounds to me like we should not export ??.

Robby

On Thu, Apr 5, 2018 at 12:45 PM, Emina Torlak notifications@github.com wrote:

Racket v6.90.0.24 exports ?? as a special identifier (PR 1803 https://github.com/racket/racket/pull/1803).

Unfortunately, the Rosette language that's built on Racket also has a ?? form, so Rosette no longer works on the HEAD version of Racket (issue 97 https://github.com/emina/rosette/issues/97, latest build https://travis-ci.org/emina/rosette/jobs/360707476). For example, these tests https://github.com/emina/rosette/blob/master/test/query/synthax.rkt no longer compile.

If we change the affected parts of the Rosette implementation to use a custom language that explicitly excludes ?? from Racket, then Rosette will be broken on every earlier version of Racket.

If we change the Rosette language to rename ?? to something else, then all Rosette code will be broken.

Is there any way to have our cake and eat it too in this case?

Thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/racket/racket/issues/2031, or mute the thread https://github.com/notifications/unsubscribe-auth/AAYWsOeTs0A33lnStfFeiq_rSlYVHAh1ks5tllg1gaJpZM4TI4ba .

[lexi-lambda]

I’m not sure if this is good enough to solve your problem, but for what it’s worth ?? can be escaped the same way as ellipses. That is, it can be escaped by writing (... ??), or more generally, (... template) where template may contain ??.

[lexi-lambda]

Oh, nevermind, I think I misunderstood the issue—it’s not just an issue with Rosette itself but also an issue with its clients? I guess Rosette could use a compile-time version-case and exclude ?? conditionally, but that’s not very pleasant (and it seems like that also might not completely solve the problem).

[bennn]

If we change the affected parts of the Rosette implementation to use a custom language that explicitly excludes ?? from Racket, then Rosette will be broken on every earlier version of Racket.

One work-around: push the current Rosette to a branch & create a version exception on pkgs.racket-lang.org to point earlier versions of Racket to that branch.

[rfindler]

Folks: I don't think that we should break backwards-compatibility in this way. We know that adding new exports can create backwards compatibility issues and I think that various subsets of us have reached consensus on taking back new exports when we discover serious repercussions.

[emina]

@lexi-lambda Yes, you are right: the issue is both with Rosette and clients.

@bennn We could do that that but this sounds like it might create problems down the line, with having to keep the branch up to date with the master ...

[jackfirth]

Would not exporting ?? from #lang racket or from (require racket) but exporting it from (require syntax/parse) work?

[lexi-lambda]

Folks: I don't think that we should break backwards-compatibility in this way.

I am generally more willing to break backwards compatibility than most, but I think I agree here that this seems like too much, given how fundamental syntax is to Racket. It’s just a bit of a shame that there doesn’t seem to be a fantastic alternative in this case, so we seem stuck with the worse version of syntax forever (excluding something like #lang racket2).

Would not exporting ?? from #lang racket or from (require racket) but exporting it from (require syntax/parse) work?

I don’t think so, since it would still break any code that requires syntax/parse but already used ?? for something else, and I imagine (though I don’t know for sure) that Rosette uses syntax/parse. As you point out, though syntax can continue to recognize ?? specially, it just has to be exported from somewhere else. It could come from a new syntax/parse/template module or something like that, but exporting it from any existing module has the potential for breakage.

[emina]

Given that Racket has many more users than Rosette, maybe the globally optimal tradeoff is to change ?? in Rosette to something else, and sacrifice the backward compatibility there.

We'll do a quick analysis of the client code we know about and see how bad this would be.

[lexi-lambda]

As @rfindler mentioned to me offline, it might still make sense to move ?? into syntax/parse, since it would at least significantly reduce the probability of conflict. It wouldn’t solve the problem entirely, but it would make it much less likely.

One more small comment: it should be possible to have an import that hides ?? from racket/base that works on all versions of Racket by writing it this way:

(require racket/require
         (subtract-in racket/base rosette/lib/synthax)
         rosette/lib/synthax)

I don’t think this is an ideal solution, but I think this at least addresses the “If we change the affected parts of the Rosette implementation to use a custom language that explicitly excludes ?? from Racket, then Rosette will be broken on every earlier version of Racket,” point in @emina’s original issue.

[emina]

@lexi-lambda I may not have explained the first point well. But my understanding is that we'd have hide Racket's ?? in the main module as well as the module that defines ??. And this latter module can't refer to rosette/lib/synthax because that would introduce a circular dependency.

[emina]

A quick note: we looked at some of the bigger client code bases for Rosette, and unfortunately, ?? does appear in those, and we've also found cases where both ?? and syntax-parse are used.

[rfindler]

Another possibility (probably obvious) is that Racket's ?? could be called something else.

[lexi-lambda]

I may not have explained the first point well. But my understanding is that we'd have hide Racket's ?? in the main module as well as the module that defines ??. And this latter module can't refer to rosette/lib/synthax because that would introduce a circular dependency.

Ah, I see, yes. That would make it harder. You could probably use filtered-in to exclude the ?? in a way that works with older Rackets, but that starts to get a bit silly.

[emina]

Since it seems to be a good idea anyway to move ?? into syntax/parse, maybe we can try that first, and see if the clients that use both the Rosette ?? and syntax-parse continue to work?

If so, that would be a good-enough solution for us, and Rosette docs for ?? could include a warning that using both may cause problems.

[rmculpepper]

I agree with Robby that breaking compatibility is bad, and it makes more sense to amend the recent changes to syntax than to change Rosette.

I'd rather not move the new exports to a separate library. A main motivation for the merge was to have the new features available by default, rather than having to piece together multiple libraries. Even exporting them from syntax/parse instead would be a step back; ?? is only really useful with syntax-parse, but ?@ is quite useful even with define-syntax-rule.

My preference would be to rename ?? and ?@, probably to ~? and ~@ since there's precedent for syntaxy things marked with a leading ~. (If anyone knows of a conflict with these names, please speak up!) Alexis asked a while back (and sorry for not answering then... deadlines) about using longer names with words instead of punctuation, but I do think of these as punctuation, like #', ..., #,@ etc; it helps me visually distinguish the forms controlling the template from the contents of the template.

[emina]

The names ~? and ~@ would work for us (no known uses).

[rfindler]

Is there an easy way to check the pkg server to see if any (non-private?) modules export anything with those names?

[stamourv]

There's the usual catalog-archive + grep + inspect method. Starting a catalog-archive now.

[LeifAndersen]

I'm going to set aside the conversation of backwards compatibility for a moment and address Emina's original question:

Is there any way to have our cake and eat it too in this case?

As it turns out there is. Although its not as nice as it could be.

The problem is that except-in will error if an identifier is not provided. What we need is some sort of except-in/quiet, that will work like except-in when the required module provides the given identiier, but if it doesn't, simply doesn't error.

IMO, something like this should be provided by Racket, but its currently not. So we need to construct it ourselves with a require transformer. This transformer needs to:

1. Resolve the required path to a module
2. Get the exports from that module
3. Intersect the except-in excluded identifiers with the module's exported ones.
4. Pass the results into an expand-import call, returning the resulting values.

Here is a toy version that works when the module is racket/base, which I believe will work for this specific problem. In general I suspect you need to a much better job resolving the module path:

#lang racket

(require (for-syntax racket/require-transform
                     racket/list
                     racket/set
                     racket/dict
                     syntax/parse))

(define-syntax except-in/quiet
  (make-require-transformer
   (syntax-parser
     [(_ modname:id exception-names:id ...)
      (define mod (syntax->datum #'modname))
      (define exceptions (map syntax-e (attribute exception-names)))
      (define-values (objs macros) (module->exports mod))
      (define excluded
        (set-intersect exceptions
                       (set-union (map first (dict-ref objs 0 '()))
                                  (map first (dict-ref macros 0 '())))))
      (expand-import #`(except-in modname #,@excluded))])))

Now you can do something like:

(require (except-in/quiet racket/base
                          list
                          flarglblarg))

And racket/base is required, excluding list, but not flarlblarg, because that is not provided.

Obviously this particular implementation is not as robust as it should be, but it 'should' be good enough for Rosette's case....hopefully. :)

~Leif Andersen

On Thu, Apr 5, 2018 at 4:40 PM, Robby Findler notifications@github.com wrote:

Is there an easy way to check the pkg server to see if any (non-private?) modules export anything with those names?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/racket/racket/issues/2031#issuecomment-379069540, or mute the thread https://github.com/notifications/unsubscribe-auth/AARc5oDyY_buu1mx3odgGxOSAF_YlIqQks5tloEogaJpZM4TI4ba .

[stamourv]

~? is used internally by the lazy language. I didn't find any other uses. I did not find any uses of ~@.

[rmculpepper]

@stamourv thanks!

[lexi-lambda]

My preference would be to rename ?? and ?@, probably to ~? and ~@ since there's precedent for syntaxy things marked with a leading ~. (If anyone knows of a conflict with these names, please speak up!) Alexis asked a while back (and sorry for not answering then... deadlines) about using longer names with words instead of punctuation, but I do think of these as punctuation, like #', ..., #,@ etc; it helps me visually distinguish the forms controlling the template from the contents of the template.

I’m a bit late to this, but here’s my POV:

1. syntax/parse patterns already use words for pattern control, not symbols, and they use them prefixed with ~. To visually distinguish these directives from pattern bindings, I’ve long surrounded the ~-prefixed identifiers in curly braces (e.g. {~seq x y z}, {~optional #:foo}), but that’s stylistic choice.

2. It’s elegant to have patterns that have the same syntax as their corresponding value constructors. For example, racket/match allows us to match against a list using (list a b c), and we can rebuild that same list on the RHS using the same syntax, (list a b c). We already have this correspondence between syntax-case and syntax via the ellipses, but we do not have any such correspondence between syntax-parse and syntax / template.

3. Given the above, it would be aesthetically pleasing to use ~seq instead of ?@ / ~@, since it has the same splicing meaning in the pattern language as ?@ has in the value language. Similarly, ~optional or ~or could be used in the template language to match the corresponding concept in the value language. I already use curly braces around ?@ and ??, and I will continue to do so if they are changed.

4. Sadly, I think using ~seq, ~optional, or ~or directly would be a very poor idea in practice, since any macros that currently generate syntax/parse patterns would be broken by the change, and I’m sure quite a lot of those macros exist. Therefore, it would be safer to change the prefix for backwards-compatibility’s sake, using something like ?seq, ?optional, or ?or.

5. I even think it would be nice if syntax/parse were to export a form that uses the ~-prefixed versions instead of the ?-prefixed ones to regain the elegant syntax / syntax-case correspondence, but picking a name for such forms would be tricky, since repurposing template would be backwards-incompatible with anyone currently using syntax/parse/experimental/template (which I understand is in fairly widespread use, despite the experimental in the name).

I think going with ~? and ~@ over the other names is disappointing, not because I have any real issue with the symbolic names (I write too much Haskell to have any problem with symbols), but because it seems to miss out on an opportunity for some much-needed consistency in Racket’s museum of syntax parsing/construction forms. That said, I don’t care that much (I probably wouldn’t have brought this up at all were it left unmentioned), and this certainly isn’t a hill I’m going to die on. But at least I can feel like I made my case. :)

[ChengCat]

I tend to be a perfectionist, and I think @lexi-lambda is right that ~seq ~optional ~or would be more consistent.

Using a slightly different symbol like ?seq has the advantage that escaping ~seq would not be needed for macros generating syntax/parse patterns, but that would be inconsistent with the situation of generating ....

[emina]

Thank you all!

This fixes the ?? clash.