Lists produced by multiplying a list by a number can be aliased

[jonathansharman]

x = 2 * [2 * [0]];
x[0][0] = 1;
print(x);

Expected output: [[1, 0], [0, 0]]

Actual output: [[1, 0], [1, 0]]

[jonathansharman]

This may be working as intended due to Crayon's aliasing rules, but I found it quite confusing, and it makes the above pattern (which I think is the most obvious use for the feature) unusable. Maybe at least when the multiplied list is a list of primitives, each list could be made distinct?

[blakeohare]

Closing as working as intended/fixed. Multiplying a list is a shallow copy, not a deep copy. Changing this for a specific situation would make the behavior inconsistent. The good news is that there's a Core.newArray2D(dim1, dim2) function in the next version of Crayon to address this specific scenario: https://github.com/blakeohare/crayon/blob/master/Libraries/Core/src/core.cry#L177

[blakeohare]

Python's take for comparison:

You can also use .map(): x = (2 * [0]).map(t => 2 * t)

[jonathansharman]

I feel like Python made the wrong choice here, as evidenced by the fact that all of the top answers to this Stack Overflow question are careful to caveat that the copies are shallow. I can't think of a situation where I'd actually want aliasing when doing this operation, so its usefulness seems limited to 1D lists of primitives.

[blakeohare]

This feature is basically just intended for creating lists with a dynamic starting size since the only other way to make a list is defining it with specific starting values, and so the fact that its usefulness is limited to 1D lists of primitives is by design.

For creating 2D arrays, this is not going to happen with this feature without implementing deep copy, which are an extensive rabbit hole of inconsistent behaviors:

• Object cloning: Cannot create copies of objects without re-running the constructors. This is very C++-y but not a normal thing to happen in most other high level languages, which is where I'd like to keep Crayon.
• Cyclic data structures would have to have some sort of special casing where no solution would be predictable.

For the long term, newArray2D() is going to be the answer for your scenario. But this specific feature will never have deep copy. I'd also like to eventually have some compact syntax for creating multi-dimensional grids (and open to suggestion since newArray2D is a mouthful and only specific to grids) at which point, I'd be happy removing list multiplication altogether in future versions.

[jonathansharman]

You and I have debated this before (can't remember in which thread), but the inclusion of a cloning mechanism is very much not just a C++ thing:

• C++ has copy constructors
• Python has deepcopy()
• C# has ICloneable
• Java has Object.clone()
• Rust has Clone and Copy
• etc.

IMHO the only good reason not to have (opt-in) cloning/deep copying in a language is if that language is purely functional.

[blakeohare]

I haven't used Rust, so I can't comment on that.

In the case of C#, IClonable is a custom interface that the user must implement and isn't something that's inherent to all objects. Java's clone() is inherent to objects, but only certain kinds (for example, if you add a final field, cloning no longer occurs). I wasn't aware of Python's so I tested it out and it seems to also be based on heuristics that aren't necessarily predictable. For example, since methods are treated as simple fields that are copied, a private variable based on a closure within the constructor caused this to happen:

The rules of creating copies of objects in C++ are based on replicating memory as-is and have consistent rules that can be mostly predictable whereas the reference-only languages, while "safer", are restricted to having some degree of unpredictability and lots of caveats that go along with it that are language dependent. Shallow copy as default is as much of an arbitrary choice as deep copy, but it doesn't need situation-specific rules for various types and its behavior is universally understood. If something can't be predicted and consistent, it's something that I'd prefer to leave for the user to implement for their situation but at the same time offering workarounds that help in limited but common situations that can be predicted, especially if the workaround can be leveraged for a partial implementation of what they need.

An additional case against inherent object cloning in Crayon is that objects are often used as wrappers around handles to native data where the internal state of the object works in tandem to the native version. For example, cloning a GameWindow instance would yield unpredictable results when you start calling the methods and the state of the other object is no longer consistent with its clone. Or in cases like some of the IPC classes, cloning an object with native handles create a new surface for security vulnerabilities. While it may seem silly to "want" to clone a GameWindow, introducing the ability to deep copy objects can easily cause this to happen by accident. For example, imagine a user having a Sprite object that they want to duplicate and that Sprite has a reference to its Scene object and the Scene object has a reference to the GameWindow. In my opinion, the dangers of having deep copy outweigh the downside of forcing the user to make their own custom copy implementation.

Back to list multiplication, this is just a means to replicate the ability to declare a 1D array of a certain size like C#/Java's int[] nums = new int[size]; and has little other purpose, as you pointed out. That said, I can see how wanting a multi-dimensional array like C#'s int[,] grid = new int[width, height] is important and I would like to add this as a specific feature in the future, I just don't think changing the behavior of list multiplication in an everything-is-a-reference language is the way. This is my main contrast to C++ where declaring an array of a class will create a list of constructor-invoked instances. But other languages that do support deep copy are based on heuristics and quirks.

Further workarounds for complex data deep copying also exist through serialization. For example, a common trick in JavaScript which lacks any deep copy alternative is to round-trip the data through the JSON serializer/parser which will gracefully handle primitives in addition to nested lists and dictionary-like objects. This trick also works equally well in Crayon.

I'm not necessarily opposed to offering more extensive options for deep copying. Nice features to have would include:

• a built in method to copy nested data structures that don't have object instances like JSON round-tripping, or something that tracks cyclic references and clones them gracefully like Python's copy.deepcopy.
• user-defined clone behavior via an interface like C# and disallowing the cloning of any other object that doesn't implement the interface. This would work for Acrylic which has interfaces, but for Crayon, this would more likely be in the form of field annotations.

My main point I am opposed to is automatic invocation of heuristic based copying. Changing list multiplication cannot be done without addressing this. Additionally, because Crayon, like Python, is an everything-is-a-reference language and creating a copy of a reference is still a reference to the same object, I fundamentally disagree that Python made the wrong call here and so the fact that shallow copy is in itself a heuristic based copy, the heuristic simply ends there. I don't deny that it can be confusing for people that aren't used to pure reference based languages and as mentioned, I'm more willing to create alternate syntax to create arrays of dynamic starting length and then remove list multiplication altogether rather than change the implicit copy behavior from multiplying a list to something that is arguably less expected for a segment of users that come from Pythony/JavaScripty based backgrounds.

[jonathansharman]

The semantics of a deep copy always depends on the type, and all of the example languages I gave either allow or require the definition of a copy to be specified on a per-type basis. In C++ this is done by allowing the compiler to generate a copy ctor or defining it manually. In Rust, by implementing or #deriveing the Clone trait. In C#, by implementing ICloneable. In Python, by writing a __copy__() or __deepcopy__() function (because Python is terrible). What it ought to mean to deepcopy a closure is up for debate, but Python arguably chose the worst option. In C++, you have to explicitly say whether the closure takes captures by reference or value. Rust basically figures it out for you, and the borrow checker ensures you don't get mutable aliasing by accident. Java doesn't allow capturing non-final variables at all, so aliases of captures are always safe.

The rules of creating copies of objects in C++ are based on replicating memory as-is

I would disagree with this characterization of copy semantics in C++. Primitive (and other trivially copyable) types can just be memcpy'd, but anything that manages a resource (including strings, containers, and smart pointers) requires a non-trivial copy ctor, and any type that contains such a type as a field must also have a non-trivial copy ctor (which the compiler will generate by default). This is much the same as the other languages.

Shallow copy as default is as much of an arbitrary choice as deep copy

Implicit deep copying on assignment is the only weird thing about C++ here, and I'm not advocating for that in Crayon. However, consider these examples:

x = [1];
y = x.clone();
y[0] = 2;
print(x[0]); // 1

class Wrapper {
    field value;
    constructor(value) { this.value = value; }
}
x = [new Wrapper(1)];
y = x.clone();
y[0].value = 2;
print(x[0].value); // 2

It would be very nice if the built-in list and dictionary clone() methods at least called clone() recursively, similar to other languages. It'll be a Core.InvalidInvocationException if any of the nested types don't have clone(). (Alternatively, you could provide a default clone() implementation for all objects that just clone()s fields recursively until you get down to primitives, which would be more ergonomic. This could be opt-in via an annotation, similar to Rust's #derive(Clone).)

For example, cloning a GameWindow instance would yield unpredictable results when you start calling the methods and the state of the other object is no longer consistent with its clone.

Just don't implement clone() for GameWindow, and let it be an error if the user tries to clone one, either directly or recursively.

For example, imagine a user having a Sprite object that they want to duplicate and that Sprite has a reference to its Scene object and the Scene object has a reference to the GameWindow.

Every game engine I've worked with in C#, C++, and Rust has distinguished between a resource and a handle to a resource. Cloning a handle shallow-copies the resource. I imagine Crayon's resources are already exposed via handles anyway, so nothing would need to change here.

Lastly, I would still advocate for list multiplication to call clone() implicitly, even if it's less like JS and Python. There are many terrible things in both these languages, some of which you've already decided to change in Crayon. When trying to initialize a list of objects using the multiply syntax, I would rather get an early runtime error than get silent aliasing. However, I would understand not wanting to introduce implicit clone()s, and like you say it'll be moot once you add multidimensional array syntax.