Support 'without' for Anonymous Records

[cartermp]

I propose we support expressions of the following:

let a = {| X = 1; Y = 2; Z = 3|}
let a' = {| a without Y |} // {| X = 1; Z = 3 |}

That is, being able to construct a new anonymous record that is a subset of another one.

The existing way of approaching this problem in F# is to manually construct a':

let a = {| X = 1; Y = 2; Z = 3|}
let a' = {| X = a.X; Z = a.Z |} // {| X = 1; Z = 3 |}

Pros and Cons

The advantages of making this adjustment to F# are:

• Less code involved to create a subset
• This sort of "drop a row"-style programming would be familiar to those using Python and Pandas for data science-y tasks

The disadvantages of making this adjustment to F# are :

• Anonymous Records can do even more than normal records, making the "smooth path to nominalization" less of a goal for these types.

Extra information

Here's what the RFC says about this:

Supporting "smooth nominalization" means we need to carefully consider whether features such as these allowed:

• removing fields from anonymous records { x without A }
• adding fields to anonymous records { x with A = 1 }
• unioning anonymous records { include x; include y }

These should be included if and only if they are also implemented for nominal record types. Further, their use makes the cost of nominalization higher, because F# nominal record types do not support the above features - even { x with A=1 } is restricted to create objects of the same type as the original x, and thus multiple nominal types will be needed where this construct is used.

However, Anonymous Records already support {| x with SomethingElse = foo |} to construct a new AR that has more fields than the one it was constructed from. This means that the middle point is already sort of violated, since you cannot reproduce this with record types.

Estimated cost (XS, S, M, L, XL, XXL): S-M

Affidavit (please submit!)

Please tick this by placing a cross in the box:

• [x] This is not a question (e.g. like one you might ask on stackoverflow) and I have searched stackoverflow for discussions of this issue
• [x] I have searched both open and closed suggestions on this site and believe this is not a duplicate
• [x] This is not something which has obviously "already been decided" in previous versions of F#. If you're questioning a fundamental design decision that has obviously already been taken (e.g. "Make F# untyped") then please don't submit it.

Please tick all that apply:

• [x] This is not a breaking change to the F# language design
• [x] I or my company would be willing to help implement and/or test this

[jwosty]

Would you be able to chain multiple withouts, as in something akin to this?

{| a without Foo without Bar |}

[cartermp]

I would expect a semicolon-delimited list of labels, sort of an inverse to what you can do today:

let a = {| X = 1; Y = 2; Z = 3|}
let b = {| a with A = 3; B = 4; C = 12 |}

So it would probably look like:

let _ = {| a without Foo; Bar |}

[jwosty]

What are your thoughts on allowing combining with and without?

{| a without Foo; Bar
     with Baz = ... |}

[7sharp9]

It might be worth noting that in Elm:

Pretty much no one ever used field addition or deletion. In the few cases where people did use it, it got pretty crazy pretty quickly.

[7sharp9]

RE: https://elm-lang.org/blog/compilers-as-assistants#simplified-records

The syntax in Elm was actually really nice and intuitive.

[cartermp]

@jwosty I think combining would probably be out of scope - works for simple stuff, but there'd have to be some rule for determining what subsequent with or withouts applied to - just the base a, or the result of a without ... or a with ...?

[dsyme]

I think I'm basically OK with this - it is true that it is no worse than {| x with A = 1 |} and aligned with it.

Note this would make without a reserved keyword. I'm ok with that if under a /langversion switch but we should be aware of it, and it would be the first such new keyword we've introduced via /langversion

[cartermp]

Just to clarify, for backwards compat, would we still compile something like this?

let without x = ()

without 12

[yatli]

to the extreme:

let y = {| x without without |}

without as a keyword would be highly contextual, so I guess it would not interfere with most parts?

[cartermp]

Correct, I can't think of a way for it to collide unless we explicitly took a breaking change here.

[dsyme]

Don't be shy :)

let without = {| with="or"; without="you" |}
let with = {| without without without |}

[Tarmil]

I was going to propose with out in 2 words, but then I realized that out isn't actually a keyword in F#, so it's no better than without 😄

More seriously, we don't have contextual keywords yet, do we? This feels like too minor a feature to introduce such a major concept.

[EBrown8534]

@cartermp @dsyme Is there any reason we couldn't {| thing not with whatever |}? This would avoid the keyword issue and, quite possibly, make just as much sense.

[yatli]

perhaps because not is a function? If thing is a functor then it's ambiguous.

[dsyme]

I guess with- would work, as a new keyword (in the language with is never followed by - today)

{| a with- Foo; Bar
     with Baz = ... |}

[cartermp]

Would there be an issue with a contexual keyword like this? I don't think it's too crazy. Generally I don't think I'd like with-; we don't really have a precedent for tagging - or + onto letters or words (like Scala or OCaml's co/contravariance)

[7sharp9]

I think with- looks confusing, you could end up with it on a pattern match to inverse the matches for consistency.

[dsyme]

Would there be an issue with a contexual keyword like this? I don't think it's too crazy. Generally I don't think I'd like with-; we don't really have a precedent for tagging - or + onto letters or words (like Scala or OCaml's co/contravariance)

It feels hard TBH

[cartermp]

I'm little with- simmer a bit and it doesn't seem too bad.

[cartermp]

https://github.com/fsharp/fslang-design/pull/370

[dsyme]

perhaps this would fly, though it's a bit weird too:

{| a with not Foo; not Bar; Baz = ... |}

or

{| a with -Foo; -Bar; Baz = ... |}

or for the unix heads (joke)

{| a with rm Foo; rm Bar; Baz = ... |}

[matthid]

Can someone describe why this is needed? What are the applications or existing use-cases / scenarios?

[dsyme]

Can someone describe why this is needed? What are the applications or existing use-cases / scenarios?

I think the OP explains why - basically, if you can create a new anonymous with extra fields, it seems reasonable to also support removing a field.

I don't think it's crucial at all, it's just a reasonable completion of the anon record feature within the criteria of the original RFC.

[matthid]

Yes I understand what the feature should do. I'm asking because you could also "solve" this another way: Why not allow Anonymous Records at places where fewer fields are required?

let f {| X = x |} = x
f {| X = 1; Y = 2 |} // 1

This obviously needs to solve how we can represent this in IL and also some other corner cases, but it would remove the need to explicitly remove fields.

However, for this to be useful you need to know what the feature use-cases are (which hasn't been answered)

[7sharp9]

Im all for more record row polymorphism type features, it is really curious that the feature was almost never used in Elm though.

[cartermp]

@matthid This was filed as a suggestion after speaking with @rickasaurus and his team at an F# meetup. The "drop a column" use case is applicable to any scenario where you'd do the same with Pandas/Python. It is also a dual to the "add a column" functionality you can do now with no apparent downside aside from just being another thing you can learn.

Your comment - effectively structural subtyping - was also brought up as highly relevant to that space, but it's explicitly called out as a non-goal here: https://github.com/fsharp/fslang-design/blob/master/FSharp-4.6/FS-1030-anonymous-records.md#design-principle-no-structural-subtyping

So the issues listed in the RFC would have to be resolved for that to progress.

Though the two suggestions are sort of related, but the ability to "drop a column and move on" seems distinct enough from structural subtyping to warrant its own issue.

[dsyme]

Though the two suggestions are sort of related, but the ability to "drop a column and move on" seems distinct enough from structural subtyping to warrant its own issue.

Yes, drop-a-column may well get used in places where there is no strongly-typed consumer with the new set of columns made explicit.

@cartermp One criticism of these features might be that these operations have no corresponding things in the type algebra, e.g. given type R = {| X:int |} you can't write {| R with Y:int |} nor {| R with rm X |} (equivalent to {| X:int Y:int |} and {| |} respectively). But I don't think that really matters, just mentioning it for completeness

[matthid]

Though the two suggestions are sort of related, but the ability to "drop a column and move on" seems distinct enough from structural subtyping to warrant its own issue.

Not necessarily. One could think of a implementation where structural subtyping is implemented by the compiler initializing a new instance. In this scenario there is no "need" for "remove" as they would be gone for reflection as well. Note that I'm not saying that I like that particular suggestion of mine. Just tried to understand the reasoning behind this original suggestion better. Thanks.

[cartermp]

@dsyme as I was writing the RFC I was actually wondering if it was worth exploring that kind of type algebra for normal records 🙂. Decided not to write it out.

@matthid

Not necessarily. One could think of a implementation where structural subtyping is implemented by the compiler initializing a new instance. In this scenario there is no "need" for "remove" as they would be gone for reflection as well.

Sure, the compiler could employ whatever tricks it needs, but that doesn't guarantee that the programmer also understands that the compiler is doing this for them. It also requires type annotations wherever you use it to make it clear that you're working with a subset. A "drop a column" approach still ties you to the nominal representation, but it's extremely clear that you've got a subset after constructing one.

[EBrown8534]

@matthid FWIW to a newbie like myself, the "drop a column" approach makes the most sense.

In my opinion, there is absolutely no situation where I should accept a {| X : int |} and be able to pass a {| X : int; Y : int |} strictly because the X is no longer the same. In the context of Y it has a new meaning. I should have to do {| thing with -Y |} or {| X = thing.X |}. It means I decided how to do the conversion.

I think the suggestion of structural subtyping falls too close to implicit conversions -- F# doesn't have implicit conversions, and in my eyes these two are one-and-the-same.

[drk-mtr]

(Another newbie comment)

Personally I'm not into the with-, however I guess an upside might be that you could move to with+ for additions. That avoids the scenario where someone accidentally adds a field due to a typo. I realise a change of syntax for additions isn't part of this proposal though, so probably a moot point!

[yatli]

does x and z have the same type:

let x = {...}
let y = {| x with foo = 1 |}
let z = {| y without foo |}

let _ = (x = z)

[yatli]

what is the precedence of:

{| x with foo = 1 without foo |}

[cartermp]

@yatli Please refer to the RFC here: https://github.com/fsharp/fslang-design/pull/370

[abelbraaksma]

@yatli, x and y are the same types, but y has a different value for foo. Variable z is a new type, without the field foo.

[yatli]

@cartermp thanks, I think it answers my latter question. Still I’d like to know, in the “1+1-1” case, does it have the same type as the original.

@abelbraaksma that’s a gotcha, we’ve got different assumptions here—whether the type of x has foo already or not.

[Tarmil]

@cartermp thanks, I think it answers my latter question. Still I’d like to know, in the “1+1-1” case, does it have the same type as the original.

If I'm not mistaken, the identity of an anonymous record is determined by its defining assembly and its field names and types. So given your example:

let x = {...}
let y = {| x with foo = 1 |}
let z = {| y without foo |}

• if x was defined in another assembly, then all 3 variables have different types.
• if x was defined in this assembly:
  • if x had a field foo: int, then x and y have the same type.
  • if x had a field foo: SomeOtherType, then all 3 variables have different types.
  • if x didn't have a field foo, then x and z have the same type.

[yatli]

thanks @Tarmil, imho this kind of detail should be clarified, and written into the RFC.

I was assuming that every mutated anonymous record will produce a different type, and two mutations have different types even if the type signature appears to be the same:

• if x didn't have a field foo, x and z have different types.

@cartermp do you expect the behavior as @Tarmil described?

[cartermp]

@yatli Note that this is already covered in the existing anonymous records RFC. If you feel specific sections need clarification, please comment on the PR - it's easier to address that way.

[realvictorprm]

I'm super hyped for that feature because it allows me to do the same out of the box what this library in the Scala World called Chimney is able to achieve only with marcos. https://github.com/scalalandio/chimney

type DomainEventA = {| A: int; B : float; CreatedAt: System.DateTime |}
type DomainEventB = {| A: int; CreatedAt: string |}

module DomainEventA =
    let toDomainEventB(a: DomainEventA ) : DomainEventB =
        {| {| a without B |} with CreatedAt = a.CreatedAt |> string |} |}

How is the syntax intented to be used with mixing with & without? That feature is particually usefull for mapping between invidual domain events which share a large amount of similar fields but sometimes don't have some fields! :3

[cartermp]

@realvictorprm Please refer to the RFC pull request here: https://github.com/fsharp/fslang-design/pull/370

I believe it addresses your question

[realvictorprm]

You're right. I'm very pleased with the RFC, maybe I should try creating a draft implementation for that 🤔

[jbeeko]

Can someone describe why this is needed? What are the applications or existing use-cases / scenarios?

We would use this it tailor an API response object based on permissions available to the requestor. For example the response pipeline might have a function hideSensitiveInformation.

[gusty]

We would use this it tailor an API response object based on permissions available to the requestor. For example the response pipeline might have a function hideSensitiveInformation.

Then this feature won't help you as you won't know it at compile-time.

[jbeeko]

We would use this it tailor an API response object based on permissions available to the requestor. For example the response pipeline might have a function hideSensitiveInformation.

Then this feature won't help you as you won't know it at compile-time.

I was thinking of something like this:

generateJson dataRecord authorities =
    (match authority with
    | Admin -> {| dataRecord |}
    | User -> {| dataRecord without SensitiveInfo |}) |> serialize

[jcmrva]

I'm looking forward to this feature but I'm not a fan of with-. It's awkward to type, goes against precedent (as mentioned), and could be mistaken for with at a glance. How do you pronounce it (on a Twitch stream, perhaps), "with minus"?

What about an less-common antonym like sans? Same length, but more visually/audibly distinct.

Side note, I'm not sure why this wasn't done much in Elm but I find the syntax unclear at a glance. { z - x } looks like an expression.

[rmunn]

Just saw in the discussion of my syntax proposal in #492 (where I said "if #762 introduces the without keyword, then we should definitely use that here instead of not with) that with- was being considered as the keyword for this feature rather than without. So I want to repeat my reaction here:

Ugh, with- is ugly. That's about the worst choice possible. I understand not wanting to introduce new keywords, as I was bending over backwards not to do so myself, but yikes. Well, if with- is settled on, then it's what should be used here too, but yikes. (Wait, I said "yikes" already. Well, that just goes to show how ugly I think that choice is.)

[baronfel]

I tend to agree with @rmunn. Aesthetically without seems cleaner, and it communicates the intent better than tagging existing keywords with sigils. These sorts of contextual keywords do have use, and if this is the driver for working on compiler to support such things then let's do it. Are there other suggestions where we've changed the desired syntax due to a lack of contextual keywords? If there are enough such cases it might make sense to pick one and do the work.

[Krzysztof-Cieslak]

Yeah, with- looks terrible. It's just saving 2 (TWO!) characters from without and it introduces something that looks terrible, and confusing to users.

[auduchinok]

From the top of my head: what if we move in another direction and specify fields to include, something like this?

let r1 = {| A = 1; B = ""; C = true |}

let r2 = {| r1 with A, B |}

// or
let r3 = {| r1 with A; B |}

It won't require adding another keyword or using the with- syntax, but probably wouldn't look as pretty in bigger records. Do people tend to use lots of fields in anonymous records?