dart-lang / language

Design of the Dart language
Other
2.66k stars 205 forks source link

Access to private members and Dev mode #693

Closed icatalud closed 4 years ago

icatalud commented 4 years ago

I posted #409 some time ago, but it got rapidly closed. This has been a permanent annoying issue for me so I came back to revise the post and the arguments given and I would like to raise this issue again, hoping that this time it remains open for further discussion and/or for future reference for people that might also want this feature, unless there is a conclusive argument that completely closes the door. The arguments I could gather to justify disallowing private member access are the following:

  1. If people accessed private variables, it makes the maintenance of the libraries more difficult, since developers of libraries are expecting to be free to rename or remove private members. Counter-argument: This is true, but it is not conclusive about the best way to deal with the issue. Complete member access disallowance might not be the best way to deal with it, this could be dealt with on a higher abstraction layer than the language itself. Not accessing private members could be for example a requirement to publish packages in pub.dev or it could be possible to access private members if the library on which private members are being accessed had a fixed version in the pubspec file.
  2. A library exposes a public API that has all the methods and values necessary to have the library behave as described, private members are implementation details or values that are unnecessary and it produces confusion to expose them. Counter-argument: There is often a thin line when defining which members to make public or not in an API. When wishing for a certain API it is easy to imagine which methods would define it, however as the implementation starts, new ideas, variations and parameters show up and subjectively some of them are chosen to be made public (for advanced library usage). Not all of them are made public because then the library becomes more complex, it makes it clumsier for the autocomplete to show the API defining methods and harder for people to figure out which method they actually need (because there are too many options). When people use a library for a prolonged period of time, it is more common to become familiar with the internal implementation. Many times there is a desire of extending or modifying certain behavior to adapt the library better to a certain need (I would call this advanced library usage). In these cases it is extremely useful to be able to play around with the internals. The confusion can easily be avoided by making clear private access is an “advanced” or “dev” feature which should be avoided unless it is really understood what is being done and making it clumsy to access it (for example requiring special keyword or enabling a special mode in the pubspec).

In my case, I had to permanently spend time, find work arounds and comprise the best design in order to deal with the impossibility to access private members. At the very least I would have liked to easily access them to code a prototype solution (get excited about it) and then think about how to make it possible without private access. Particularly in Flutter I have permanently desired to access the _child or _dirty field in Element (examples that I can remember right now). I do not think those fields are ever going to be changed, they are hidden mainly because accessing them makes no sense in the common Flutter app, it would only cause confusion and if they get overridden it could break the Framework proper flow.

Proposal

Option 1: Allow accessing private members only through a keyword “private NameOfSuperClass _nameOfPrivateVariable” and explicitly discouraging the use of private unless strictly needed in the language guide or through dart analyzer default warnings (forcing ignore warning comment) to avoid using this. Similarly it should be possible to instantiate and access private classes using private _NameOfPrivateClass.

Option 2: Allow access to all private members from a certain import by adding a special keyword like "import private ..." or "import dev ...". There is still the problem with conflicting private member names of ancestor classes, but then the private member could be accessed by prefixing the name of the class like in Option 1.

Personal philosophy about freedom

I believe there should always be as much freedom as possible, because this allows exploration and creativity to express. Creativity is by definition something new and unexpected, therefore it cannot be predicted, if it could be predicted then it would not be creative. Absolute freedom can of course be harmful, so under what justifications should be freedom be contained (IMO)?

Programming language freedom

In a programming language there is no direct obvious risk to harm anyone with freedom like there could be with a physical object. Playing with code is a harmless sandbox (one of the beautiful things of writing code), so in my opinion a language should allow to play with it within all the imaginable possibilities even if it seems useless or inconsistent. For example I would like that in Java it was possible to access internals which do not seem useful and is not consistent, like reading or setting reference pointers. Some might use their creativity to make something interesting, but not necessarily useful in a direct productive way. For example, art or entertainment are not directly useful-productive, but they probably have an indirect impact on it. To prevent rare features to cause disorganization or confusion, all that is required is to be crystal clear that those are advanced features and/or put barriers to use them, so that only “advanced” users reach them. For example I would enable all kind of weird things in a special "dev mode" that can be activated either in the pubspec or in the dart file. The language guide should direct a common proven productive programming pattern.

All in all, what I try to say is that I would not disregard any feature request, because if someone asked for it, it is because it expresses something that at some point they would have liked to have. Features that are decided are not worth to be put on the language should at least be allowed to become experimental (they hardly harm someone over there). Which features are chosen to be developed should be based on the developer’s preference. Usually the most requested features are the most likely to be developed first, that is a natural gradient of where humans put their interest.

I understand that this might not be highly requested, however if I get interested, I could even personally attempt to implement this, I would just like to hear opinions about the possibilities and the best semantic approach.

munificent commented 4 years ago

Complete member access disallowance might not be the best way to deal with it, this could be dealt with on a higher abstraction layer than the language itself.

That has real practical negative consequences. Let's say you maintain a package foo, that has a class Foo:

class Foo {
  // Stuff...
}

A user of your package has a class Bar:

class Bar {
  int _privateField;
}

Everything is working fine. Later, you decide to make a change to Foo that is purely an implementation detail and should affect no users:

class Foo {
  String _privateField;
  // Other stuff...
}

This works fine today. Maintainers of packages can add and remove private fields confident that they will not harm their users. If privacy is no part of the language, then from the language's perspective Bar._privateField would now be an erroneous override of Foo._privateField and the code would have a compile time error. This is why, for example, Python name mangles members that start with a double underscore.

The confusion can easily be avoided by making clear private access is an “advanced” or “dev” feature which should be avoided unless it is really understood what is being done and making it clumsy to access it (for example requiring special keyword or enabling a special mode in the pubspec).

Library authors can accomplish this now using naming and multiple libraries. It's fairly easy to separate out advanced members just by doing good API design.

I believe there should always be as much freedom as possible, because this allows exploration and creativity to express.

You are only thinking about the freedom of the consumer of a library. But what of the freedom of the maintainer of the library? Privacy gives maintainers freedom because it gives them more safe space where they can change their library while being confident that they do not break users. That freedom in turn benefits consumers because it makes it easier and faster for maintainers to evolve their libraries and provide new functionality to users.

All in all, what I try to say is that I would not disregard any feature request, because if someone asked for it, it is because it expresses something that at some point they would have liked to have.

I didn't close the original issue by discarding it out of hand. Dart is almost ten years old. We have had this discussion around privacy — including exactly the arguments of how it interacts with personal freedom — many many times over the years. The point that we've arrived at is based on careful deliberation. All encapsulation features in a language make trade-offs between what the consumer and maintainer of a library are allowed to do. There is no silver bullet that gives total freedom to both sides.

The point we've chosen closely follows most other object-oriented languages and seems to strike a decent balance.

Note that Dart, unlike Java, C++, and C# gives consumers an important affordance. Packages are distributed as source. If a consumer wants to do something unsupported, they can always copy the library and edit the source themselves. Given that, I see little upside to taking flexibility away from package maintainers. It is already hard enough to change a class in Dart without potentially breaking users as it is.

icatalud commented 4 years ago

This works fine today. Maintainers of packages can add and remove private fields confident that they will not harm their users. If privacy is no part of the language, then from the language's perspective Bar._privateField would now be an erroneous override of Foo._privateField and the code would have a compile time error. This is why, for example, Python name mangles members that start with a double underscore.

In both proposed solutions it is considered that private fields can be redefined, it has to be like that to make it backwards compatible.

“private NameOfSuperClass _nameOfPrivateVariable”

The "NameOfSuperClass" must refer to the class where the private member is defined. It's not an elegant solution, but is the workaround I thought of to make it compatible with the current implementation.

But what of the freedom of the maintainer of the library? Privacy gives maintainers freedom because it gives them more safe space where they can change their library while being confident that they do not break users.

What I disagree about this argument is that the maintainer shouldn't make himself responsibly of what users do when it is clearly directed (by analyzer warnings, access barriers, etc) that private members access should be avoided as maintainers do not make responsible for any changes to them.

As an example of what I try to say imagine this statement: “High buildings are dangerous because people can fall or jump off from them.” The previous statement is undoubtedly true, it provides a good reason to prevent us from constructing high buildings. However that reason is far from being a good justification. To decide whether or not high buildings should be allowed, all the reasons should be taken into account, leveraged and attempt to find solutions to all the negative reasons. If that exercise is done in the high buildings statement, we could get to a conclusion like: “High building can be dangerous, but it could be allowed to build them if the constructions are regulated to enforce security mechanisms that would completely dissipate falling risks and make it hard for people to jump off from them.”

In the case of private members you are right that it constraints library maintainers, but that is as long as it is nowhere advised and no barriers are put to prevent people from using private members. However at the moment you put clear barriers, like all the ones I listed, you pass the responsibility to the user. We do build high building, but we protect people from jumping off them by putting numerous barriers. Now if someone decides to break the windows and jump anyways, that is out of your hands (there is people that crosses from building to building walking on a rope and I'm glad that it is possible to do that).

I didn't close the original issue by discarding it out of hand. Dart is almost ten years old. We have had this discussion around privacy — including exactly the arguments of how it interacts with person freedom — many many times over the years. The point that we've arrived at is based on careful deliberation. All encapsulation features in a language make trade-offs between what the consumer and maintainer of a library are allowed to do. There is no silver bullet that gives total freedom to both sides.

Then this reasons should be documented somewhere for future reference and further debate (in case someone has new ideas). The first time I posted I didn't find anything about this deliberation and they gave a short unsatisfying answer. Later the debate prolonged and it became more interesting, but that is after the thread got closed. I understand that you have already discussed this for years, but you have to consider that new developers have no idea about this.

lrhn commented 4 years ago

If experience has taught us (and every other language platform developer) anything, it is that users do not care why their code breaks, not even if it is "their own fault" for using unsupported or undocumented APIs or using reflection to access private members (look, e.g., towards Java for examples of both), they just don't want their code to stop working. And if a feature is there, no matter how much its use is discouraged, it will be used when it's the only (or just easiest) way to solve a problem, no matter how short-term a solution it is intended to be,. Breaking code which uses the feature, no matter for what reason, will make people unhappy.

If I believed for one second that users of the feature would silently accept that their code stops working at some random point in the future, I'd be more inclined to be positive about it, but no actual experience supports that. No matter how sadly.

And when they don't, a package writer will prefer to prevent access rather than spend time dealing with complaints about breaking changes to private implementation, because that is simply better use of their time.

Because of this, the incentives for this feature are such that it would pit library writers and users against each other in a tug-of-war for control.

As a library maintainer, I'd quickly start asking for a way to define really private members, where I can disable foreign access completely. Then I'd start using those for everything. Or I'd start renaming my private variables each week, to discourage anybody depending on them. Not out of spite, but to avoid anybody actually depending on my private names in their production code. Because the moment that happens, changing those names is a breaking change with serious repercussions.

If I change my class internals completely, so that there is no way to access the data that was previously accessed from a private name, then someone has production code which is broken and which can't easily be fixed. They will ask me to undo, or at least delay, the change. No amount of "but it's private" and "you weren't supposed to" arguments will change that.

You can access private members today, using dart:mirrors. The only issue with that is that mirrors are not available everywhere. You can take a copy of the libraries and make the members public. If you are locking yourself to a specific version anyway, then that's definitely doable. There are options which do not enshrine the ability to break abstractions into the language itself.

I might be more critical than most package writers because I'm also writing the platform libraries. Platform libraries are not versioned the same way as packages. One day, your SDK, or your customer's SDK, has been updated and you want to keep running your existing code. Any breaking change introduced into in the platform libraries will break someone somewhere. That, at least, suggests that if we allow private member access override, it should not apply to platform library members.

I do understand the ideal behind allowing anyone to do anything as long as they are considered solely responsible for their own problems from doing something discouraged. I'd love to live in that world, but experience suggests that we do not. That approach does just not scale to a large ecosystem of people who need to keep their code running in production.

(I'm also not into having a "dev" mode with more power, because eventually someone will release a program that needs to run in "dev" mode, and ask users to enable the "dev" mode for their production code. If a feature exists, it will be used in production.)

lrhn commented 4 years ago

Now, if we were going to provide some access to private members, there are many levels of access that we can provide with some amount of access control on the package developer side, from little power to little control.

If we were going for allowing multiple libraries to share private names, then I'd probably prefer a "friend library" feature where one library explicitly opts in to having the same private names as another library. That avoids the issues of writing private _SuperClassName _privateName when those names may match privates from different libraries. It would still mean that libraries would have to export their private members, something they do not currently do. The private members are simply not in the export scope,. This is still a very dangerous and intrusive operation, but one which we can detect statically by only looking at import statements, and we can prevent such code from being published. It only allows you to be friend with one library, not that that can't be worked around in practice.

rrousselGit commented 4 years ago

I definitely agree with the previous answer.

Alternatively, there are things like show/hide directives. We're free to make everything public, and use these directives to expose only what's desired.

There's also the @visibleForTesting decorator, which is used on public members to flag them as "private" and trigger a warning if used outside of the package.

You're also free to name use something other than _ for your private variables, such as $. This would achieve strictly the same thing as what you asked for, just using a different name convention.

rrousselGit commented 4 years ago

There's also a good example of a problem caused by "public but shouldn't be used" recently:

Electron https://www.google.com/amp/s/www.imore.com/apple-rejecting-electron-apps-mac-app-store-because-private-api-calls%3famp

I personally don't like the idea, because it decreases the stability of the ecosystem.

People will use it. If they did not, this change would be useless. But if people use this feature, then it makes package versioning hell. What would normally be a +0.0.1 increment may break some package. This will make things incredibly difficult on the long run.

icatalud commented 4 years ago

@rrousselGit wrote:

But if people use this feature, then it makes package versioning hell. What would normally be a +0.0.1 increment may break some package.

Genenerally what I'm trying to ask for is not free access to private members, but constrained access and I'm trying to figure out through debate with the Dart team under which constraints would it be “safe” to grant private access. It is possible to make full private access available always and still be safe if the right "locks" are put when publishing packages into pub.dev. For example packages that access private members could be:

Personally I prefer any of the previous solutions over "strongly" obfuscating private member access in the language itself. I like a programming language to be a sandbox where I can play with anything within the possibilities of imagination, however I also think constraints should be enforced when publishing tools that others will use. It's like some people like to be naked when they are at their home, but they don't go naked into the street.

It is my belief that the best place to put the constraints of a programming language is at the publishing layer, but not at the core of the language.

@lrn wrote:

You can access private members today, using dart:mirrors. The only issue with that is that mirrors are not available everywhere. You can take a copy of the libraries and make the members public. If you are locking yourself to a specific version anyway, then that's definitely doable. There are options which do not enshrine the ability to break abstractions into the language itself.

That is my complaint, that I have to manipulate the library, download it, do stuff that distracts me from what I wanted to do (intuitively it should be direct to do). I have to think where to download the repository, how to do it, etc, which nowadays I do without complaining so much because I got used to it, but when I started coding in Dart I was within this happy pubspec domain which would automate all the work for me and I didn't have to think about any of that, allowing me to concentrate fully on my task. When you are flowing with an idea, you want everything to happen as easily and intuitively as possible, because distractions take you out of the "spot" and uninspire you. That's one of the purposes of making everything simply and intuitive, to allow people to "flow", spend less time on tedious tasks. So whenever something can be done in a series of seemingly simple steps that could be automated, those simple steps should not be disregarded, as they mean that someone will have to read them, understand them and perform them (which takes considerable more time the first time).

I'd probably prefer a "friend library" feature where one library explicitly opts in to having the same private names as another library.

I like the friend class approach much more than my original proposal, I guess the same friend word can apply to packages (“friend package”). What would happen if you want to access private members with the same name in the super class and in the super super class (far fetched case)?

That, at least, suggests that if we allow private member access override, it should not apply to platform library members.

I would hope for having complete freedom, but having constraint at the publishing layer.

I do understand the ideal behind allowing anyone to do anything as long as they are considered solely responsible for their own problems from doing something discouraged. I'd love to live in that world, but experience suggests that we do not. That approach does just not scale to a large ecosystem of people who need to keep their code running in production.

I cannot agree with the “large ecosystem of people” statement when some of the most used languages in the world like Python or JS allow you to do anything you can imagine, they are completely open. How safe it is and how many problems this produces? Probably plenty of them, but somehow people are managing through it. This does not mean that they are doing it the right way, I like that Dart actively takes care about this and provides this consistent package dependency system.

Key point

(I'm also not into having a "dev" mode with more power, because eventually someone will release a program that needs to run in "dev" mode, and ask users to enable the "dev" mode for their production code. If a feature exists, it will be used in production.)

That comment is for me the key to our disagreement from a philosophical standpoint: It is my belief that we should live in a world that is protective under the existence of rules and guidelines that are clear and well established, but everyone is responsible for their own actions. This is for me the essence of freedom. If you stay within the rules and guides you will be guaranteed safety-consistency, however you are free to go outside and explore. Smoking or drinking alcohol is probably bad for your health, doing base jumping or any kind of extreme sport is dangerous (you are risking your life) but I don’t have the right to command what you should do or not. It is certainly the duty of the society to be informative about risks, protect people from doing “bad things” by putting barriers that discourage or prevent legal access to negative things, however what the society should never do is to force anyone to do something. When people make themselves responsible for others actions, what ends up happening is that you end up having a dictatorship, because the only way one someone can feel safe about what others do, is by directing all their actions. Companies often produce this controlling behavior when they blame responsibility on a hierarchical basis instead of personal basis. The controlling behavior is inevitable because anyone would be afraid if they are responsible for something someone else is doing (a free person could do anything). Responsibility should be enforced by fizcaling the accomplishment of rules and guidelines and not by constraining freedom. Rules do constrain freedom, but in an illusory way, the possibility to act crossing the rules is still there (the feeling of freedom remains).

My interpretation of those kind of statements is that behind them the limit from protective was crossed to controlling, which is that the Dart team is making themselves responsible for how people use the language, even though people are freely choosing to cross the rules and guidelines. The government cannot prevent people from parking on ilegal spots, and inevitable you will see people complaining, insulting the policeman, etc when their car was damaged when it was being moved, however what the government replies to the person that complains is very simple: “Didn’t you read the big sign that says no parking? The rules are clear, it’s properly advised everywhere that you should not have parked there, you knew that you were doing something illegal and choose freely and consciously to do it anyways, therefore you are responsible for breaking the law and your complaint is invalid”.

When people say “but they will do it anyways”, I will probably almost always agree, however I find it paternalistic and arrogant that someone else decides to choose for them in advance based on that statement. I like to have my freedom and to decide for myself what is good or bad for me and it would make me furious if others chose to decide for me based on their interpretation of what is good or bad, because truthfully if they were really convinced about the right thing, then they would have been able to convince me and not hide options from my reach (a direct attack to freedom).

Going to the numbers, yes, in some cases what has been claimed to be negative consequences will happen, however I wouldn’t expect more than 5% of the people using unsafe dev mode libraries. When their production systems break and they go complain to the Dart team, then just like the government you can tell them: “Didn’t you read what dev mode is about? Knowingly and willing you chose to break the guidelines and enable dev mode following the 10 steps that are placed on the language page full of warnings, therefore your complaint is invalid.” And if for any reason that 5% increased and became 50% percent, then that would be giving very relevant information about something that is probably not being done well from the Dart side. One of the beautiful things about freedom is that it exposes hidden-secret information through behavioral patterns, which is probably the most useful information one might ever have, because our imagination and observations are always limited.

lrhn commented 4 years ago

(I'm also not into having a "dev" mode with more power, because eventually someone will release a program that needs to run in "dev" mode, and ask users to enable the "dev" mode for their production code. If a feature exists, it will be used in production.)

That comment is for me the key to our disagreement from a philosophical standpoint: It is my belief that we should live in a world that is protective under the existence of rules and guidelines that are clear and well established, but everyone is responsible for their own actions.

It's not just the control. Another goal of the Dart design process is to create as coherent an ecosystem as possible. With modes, you risk splitting the ecosystem into code running in different modes, each incompatible with each other. One of the original design goals of Dart was precisely "no modes!", and we are still sticking to that. There is one Dart language. The only mode we have is "assertions-enabled" mode.

Every language feature comes with a cost, not just a benefit. Sometimes it's just implementation cost or opportunity cost, but in other cases it's a cost borne by the community.

One of the things that we have users actively asking for, is a way to lock down a class and prevent, e.g., subclassing, because they know, from bitter experience, that if you allow subclassing, then you can no longer change the API without breaking other people. You will see Java style guides saying "make everything final unless it needs to be non-final", and for the same reason: It prevents other people from making dependencies on your code that locks you into choices that you might have wanted to change. ("Subclassing: Design for it or prohibit it"). This is a fact of software engineering at large, and we are trying to design the language for that. You disagree with that choice, and for reasons I'd love to be true, but it's not a choice we have made out of spite or need to control others. It's something we believe is the better choice, based on costly experience by both ourselves and our users.

As language designers, we could allow every suggested feature, and let it be up to the users to use features responsibly. That would be reneging on our responsibility as designers. It would put the burden of complexity and choice onto our users instead of making it ourselves. I see one of our responsibilities to be to reduce complexity for the end user, by making informed choices up-front on what to allow and what not to allow, and to make the mental model needed for using and understanding the language as simple as possible (but no simpler), so that users will easily do what is best for them (falling into the pit of success). No, we're not there yet :smile:. That's what I'd consider good design, not just "design".

munificent commented 4 years ago

The "NameOfSuperClass" must refer to the class where the private member is defined. It's not an elegant solution, but is the workaround I thought of to make it compatible with the current implementation.

This still breaks encapsulation. Consider:

some_package.dart

class Foo {
  var _field;
}

class Bar extends Foo {}

The maintainer of some_package should be able to freely move _field from Foo to Bar without breaking users of either class. (For things like mixins, refactoring like this is fairly common.) From outside of the library, no user should know or care which class the field is declared in.

A syntax that refers to a private field specifically by way of the superclass where it happens to be defined makes the entire inheritance chain more brittle.

Smoking or drinking alcohol is probably bad for your health, doing base jumping or any kind of extreme sport is dangerous (you are risking your life) but I don’t have the right to command what you should do or not.

Smoking is a good analogy. Most of the code in your program is not code you wrote or maintain. It's packages that you transitively depend on. And when the authors of those packages use brittle features that break your program, you are affected even though you didn't knowingly choose to take those risks. You inhaled their second-hand smoke. And indeed, because of those effects, smoking is outlawed in many places where members of the public congregate.

In your own code, you are free to make all of your members public and then you get the exact behavior that you want. Dart already supports writing code in the style you want. What you are asking for is to get the behavior you want from other peoples' code without their consent. You want to smoke in someone else's house.

I don't see how it's helpful for the language to encourage that. Especially given that, as I noted, you can trivially make a copy of "their house" for your own use and change it however you like.

It is my belief that the best place to put the constraints of a programming language is at the publishing layer, but not at the core of the language.

That prevents compilers from using these constraints for optimization purposes. For example, today even a simple modular Dart compiler can determine that a private method is not called and eliminate it from the compiled output.

Static types are themselves a "constraint" on what programs are allowed to do. Dart's initial static type system was outside of the language's core semantics exactly like you propose. What we found was that it made it much harder to build efficient compilers and users were constantly frustrated when programs erroneously violated those constraints.

Going to the numbers, yes, in some cases what has been claimed to be negative consequences will happen, however I wouldn’t expect more than 5% of the people using unsafe dev mode libraries.

Given the large size of the average transitive dependency graph in Dart applications, if 5% of packages use this feature, then almost all Dart programs will transitively contain code that use this feature.

icatalud commented 4 years ago

@lrn wrote:

With modes, you risk splitting the ecosystem into code running in different modes, each incompatible with each other.

Advanced mode is a way of putting it, but it doesn’t really have to be a mode, it should be purely a static analysis thing. By default the static analysis can be constrained as it is now, a friendly consistent sandbox, but by adding some options to the analysis_options yaml file for example it could be possible to enable advanced (or “dev”) static mode, which would allow playing more with the language. Only experienced users would get there, because by default this is disabled.

You disagree with that choice, and for reasons I'd love to be true, but it's not a choice we have made out of spite or need to control others.

There is definitely an intention to prevent (control) a bad usage. Think of it like this, if the option currently existed, would you consciously disable the option to someone if you had to tell them person-to-person that you will not allow them to use such feature because they will misuse it? I’m going to guess that your answer would be no (correct me if I’m wrong). And if so, what this means is that there is an attempt to disallow people (someone) to use a versatile tool on the basis of assuming that they will make bad use of it, more particularly, that they will disregard all the rules and guidelines using this versatile tool in production mode and the Dart team is making himself responsible for the consumers choices. The thing is that you are not telling this to them directly, because an easier approach was taken which is choosing to constraint the language in all possible situations where something “bad” could happen, so the bad feeling of having to tell someone “you will not have this available because you will misuse it” never got the chance to present. I claim that this is what is behind, either that or being afraid of telling someone the right thing when they break the rules of the tools they are using and then they go complain to the manufacturer, which “refactoring” a quote from my previous post would be:

“Didn’t you read what dev mode is about? Knowingly and willing you chose to break the guidelines and enable dev mode following the 10 steps that are placed on the language page full of warnings, therefore you are fully responsible and there is only you to blame for your actions.

If right now you had a switch in your hand where you could decide providing or not providing dev mode and you had all of Dart users looking at you, would you confidently “turn off” (dev mode disallowance)? And what would you tell them? “This is for your own good”? All the reasons given are valid, but in the end that is what is what is being said, because all the protections could be set in the form of rules, guidelines and barriers, but not banning options. I think the Dart team should gather in a meeting and make that exercise, one by one they are asked this question directly and they have to justify it to the person they have in front (assume the decision would be affecting the person in front), because then you can get a visceral feeling of what is the true cost of a decision like this.

Every language feature comes with a cost, not just a benefit. Sometimes it's just implementation cost or opportunity cost, but in other cases it's a cost borne by the community.

Consistency is something that should be seeked in math, science and computers, it helps humans progress and live happier. Attempting to make humans behave consistently though, will lead either to exodus or a revolution, because humans are not meant to be consistent, humans are meant to understand and follow rules (but also sometimes break them). Rules will always be broken and the more freedom for humans to act there is, the more inconsistency there will be. It can be assumed that more freedom will lead to more unexpected-inconsistent behavior (a cost), but that unexpected-inconsistent behavior is where exploration and creativity happen.

Rules and guides do not constraint the limits of what is possible, they direct the road to take, where the mass will go, where the average behavior will be.

A programming language is a tool, a very sophisticated tool, but it is still just a tool, “bad usage” will not physically harm or disrespect anyone, it’s a harmless tool. There should be given as much freedom as possible to perform harmless actions with harmless tools, but directed under rules and guides (such that in the majority of cases they will be correctly used, but never in all of them). I agree that if you give more freedom, like allowing a “dev” mode will cause more problems that not having it, because the more degrees of freedom that are available the more problems that start showing up, but that is not only because of freedom itself, but also because when there are more features available there are also more users attracted and playing around (more things going on), complaining or requesting that a certain “dev mode” option is enabled also in standard Dart. But all of that is a good thing, because it means people are exploring and it would actually help Dart determine what is the optimal set of features which should be available, because perhaps the choices made were not the best. It’s impossible to predict all possible use cases, or perfectly determining the best set of features that should be available, and that’s why giving degrees of freedom is so useful. The other day I read issue #251 where people required the name of the methods as String in a way I would have never imagined (I still do not completely understand it) and it had a lot of popularity, which is a very graphic example of what happens in freedom. Tools are modeled assuming a lot of things and taking into account certain experiences, however leaving the door of exploration open allows finding the optimal set of features according to real life usage, being quite a common event that practice is does not behave very closely to theory. Imagine Dart got it wrong, in reality the majority of consumers didn’t want to have a super consistent language, that they preferred that packages were more free to manipulate at the expense of dependencies breaking sometimes or that library maintainers preferred to have more users rather than more freedom to manipulate their libraries (if they had to choose one or the other). All of this is impossible to know beforehand, the “what if” is always there and choices need to be made, but giving room to freedom gives the chance to some of the “what ifs” to be explored by a small subset of people.

based on costly experience by both ourselves and our users.

That is a very subjective statement, how costly and what happened more precisely (could you elaborate)? Would that costly experience still happen if the right barriers where put, like the ones that have been discussed here? If you elaborate the bad experience I’m sure it is possible to learn from it and to think on the right barriers to put. My intuition tells me that completely banning freedom from the developer is not the only choice (it’s probably the easiest though).

I see one of our responsibilities to be to reduce complexity for the end user, by making informed choices up-front on what to allow and what not to allow.

I read that as an elegant way of avoiding saying directly the reasons behind the decisions and that’s what bothers me. There is no right or wrong way to be, controlling is not something wrong, some people do prefer to be more controlled and have options removed because it is less complex (just like you stated it). But my complaint is that things should be said by their true nature (what they are) and not by elegantly trying to make them look as something different, because the intention of that is to also get support for an idea from the group that would generally not agree with that. And I believe “freedom” is a popular attractive word nowadays and saying “I believe Dart should control the users from making bad use of the language by consciously preventing the existence of tools that if misused could lead to production software errors” does not sound popular, because many software developers would like to be trusted as intelligent responsible people that are able to take their own choices within the broadest set of options that are displayed and organized in a way that is clear to understand.

What would you prefer if were at home?

If you were coding in Dart at your home for your personal project or for fun, would you prefer to have freedom of choice available to do anything that comes into your mind or would you prefer that the features that can be considered useless for production did not exist?

Not everything happens in production and not all Dart code are libraries where other projects will depend on. I think we can agree that the majority of Dart projects are private personal projects, for them privacy access or full language freedom is completely reasonable and viable. Big projects could definitely use a more constrained language standard and it is good that that option exists, but it is the project owners who should decide best what is more suitable for their project.

munificent commented 4 years ago

@icatalud, that is 1,439 words in a single comment, and I see you've written several other comments here and elsewhere in short fashion. Please be sensitive to the fact that it takes real time and effort for others to read and process your words. Time we spend doing that is time we can't spend fixing bugs, adding features, or doing other work. Now multiply that by the large number of people who follow the Dart issue tracker.

Some time on your end spent editing your words to something more concise, and choosing which points you consider most important while omitting others can materially make the project more efficient. Thanks.

icatalud commented 4 years ago

@munificent I would normally agree with comment when it's related to technical things, however this is a very complex subject that I think relates to something deep in the culture which is what I'm trying to expose. It's really not possible to say this in a few points.

It's complicated, because on one side this doesn't seem like the right place to have this discussion, but on the hand what I'm saying is the root of the feature I'm requesting and generally it is denied to be positive.

Is there another channel where some would like to participate where we could have this discussion on a longer term (assuming some are interested)?

How can this issue get to a conclusion? Under what terms could privacy access be allowed in your opinion? (but thinking rationally about the points and options given where there is no risk to break dependencies in public packages)

munificent commented 4 years ago

Is there another channel where some would like to participate where we could have this discussion on a longer term (assuming some are interested)?

This is probably the best venue, for better or worse. (We have a mailing list too, but it doesn't seem to get much use.)

How can this issue get to a conclusion?

I think from the language team's perspective we have already concluded. Like I said before we have had this exact discussion many times with many people. I don't believe you have raised points yet that weren't already raised and considered.

I respect that this is something you feel strongly about, but it doesn't seem to be a high priority for the Dart ecosystem as a whole. In the absence of that, it doesn't make sense for it to be a priority to the language team.

icatalud commented 4 years ago

@munificent wrote:

The maintainer of some_package should be able to freely move _field from Foo to Bar without breaking users of either class.

A lot of pre-assumptions mentioned during the posts are disregarded in that comment. Numerous times I mentioned assume all possibilities, even the most restrictive ones like enforcing a fixed library version. If a fix library version is enforced there will never be any kind of conflict when the owner changes the library.

I don't see how it's helpful for the language to encourage that. Especially given that, as I noted, you can trivially make a copy of "their house" for your own use and change it however you like.

It's not so trivial for those who has never done it and it's inconvenient, specially when the library is updated. Even if the requirement was a fixed library version, the library version can be manually be updated and in many cases there will be no conflict when doing that. From the perspective of automation to facilitate the developers work, that is very convenient.

That prevents compilers from using these constraints for optimization purposes. For example, today even a simple modular Dart compiler can determine that a private method is not called and eliminate it from the compiled output.

I'm sorry, but I do not completely follow that point. If could elaborate more please. I though what you are saying about eliminating private method was what you call "tree-shaking"? Wouldn't the same be done with public or private methods? I'm not sure what that is, but I just assumed it was the compiler analyzing the code in some kind of graph (classes and methods would be the nodes) and it would eliminate nodes (classes and methods) that cannot be reached from the roots (like the garbage collector). That's the only way it would make sense to me to eliminate members and I do not see how privacy could make a difference.

Given the large size of the average transitive dependency graph in Dart applications, if 5% of packages use this feature, then almost all Dart programs will transitively contain code that use this feature.

A completely arbitrary statement that has no practical value. The 5% is my invention, and even then assuming probabilistically that the packages dependencies between each other has that consequence is an oversimplification that is most likely wrong. Deep dependencies come usually from the "mainstream" packages that will most likely not be using features like private member usage (because it's undesired to use and it also subtracts library score from pub.dev). I would expect private members to be used mainly in small experimental libraries, because if the library ever started becoming successful, the owners would probably make the extra effort to make it as "clean" as possible.

I think from the language team's perspective we have already concluded.

And what is the conclusion being more explicit? So far I have not seen a single argument that is conclusive about banning access to private members under all circumstances. All the possible problems can be managed by including restrictions. Eradicating any chance of granting private member access can only be justified on irrationality.

I respect that this is something you feel strongly about, but it doesn't seem to be a high priority for the Dart ecosystem as a whole. In the absence of that, it doesn't make sense for it to be a priority to the language team.

I wasn't asking to make this a priority of the Dart team, especially if it is an idea that is generally rejected from inside. I was trying to create debate and bring under rational terms this subject to a point where it could be reasonable to implement such a feature without jeopardizing current stability and package dependency consistency.

I wanted this feature request to remain open with a conclusion on the best terms on which this feature could be implemented such that someone that became interested (either me or some external developer) could attempt to do it. And then perhaps it could be added as an experimental feature. Would that be possible?

leafpetersen commented 4 years ago

@icatalud I appreciate that this is a topic you feel strongly about, and I appreciate that you've put a good deal of care into articulating your position. I'm sorry you don't find the responses you've received persuasive. I think though that this topic has exhausted its usefulness, at least in this forum. As the co-lead of the team, along with @lrhn who has already spoken above, I can say very definitively that this proposal is not going to be taken up by the team under any foreseeable circumstances. Even if I found your argument convincing (I don't) this issue is nowhere close to the top of the list of our priorities.

Feel free to continue this discussion on one of the relevant discussion lists if desired, but I'd ask that for the purposes of this issue tracker, we consider this discussion closed. Thanks!

icatalud commented 4 years ago

Skip until summary section below for shorter content.

@leafpetersen wrote:

I appreciate that this is a topic you feel strongly about, and I appreciate that you've put a good deal of care into articulating your position.

The reason why I keep articulating is because I'm not getting understood at the points I'm making, so I try to express them in a different way, expecting the ideas to get correctly understood. I can read in the responses that "Not a good idea" is getting replied but then I gave arguments that keep getting disregarded. For example in my previous post I said:

I wasn't asking to make this a priority of the Dart team, especially if it is an idea that is generally rejected from inside. I was trying to create debate and bring under rational terms this subject to a point where it could be reasonable to implement such a feature without jeopardizing current stability and package dependency consistency.

But then you reply:

I can say very definitively that this proposal is not going to be taken up by the team under any foreseeable circumstances.

Which is disregarding what I said, you are giving a justification to close the issue based on a request that has never been done.

I'm sorry you don't find the responses you've received persuasive.

It’s not that they not are persuasive, they are persuasive only on the basis of disregarding the conditions I gave. In other words, by fully considering my points the responses given are no longer reasonable. We shouldn’t feel sorry about not finding persuasive a bad justification, if anything that’s something to be glad about.

As the co-lead of the team, along with @lrhn who has already spoken above,

I do not see how mentioning that aspect gives more validity to any of your statements. You or any other member of the team have opinions that are equally valid (I expect so at least) and should close feature requests based on only one reason: logical validity of an idea. It is my understanding that this forum is supposed to keep track of the Dart feature requests, ideas that are considered or dismissed if they are not clearly explained, there is a contradiction or they are out of context (the scope escapes the language itself), but not if someone just doesn't like it on a subjective basis. If that is not the case then there should be a document where all ideas are organized by subject where anyone that wished a new feature could refer to.

I can say very definitively that this proposal is not going to be taken up by the team under any foreseeable circumstances.

That statement is dismissing what I said that it’s not about the Dart team taking the issue, it is about leaving the idea open for future reference or in case an external developer wished to implement it voluntarily.

Even if I found your argument convincing (I don't)

The fact that you find an argument convincing or not is rather irrelevant. Convincing is a subjective term that depends mostly on personal preferences (it’s biased). What you should consider is if the argument is logically valid or not. If an idea is logically valid and someone finds it useful, then that is a reason strong enough to put it in a bag of ideas and then some of the people wishing to develop new features might find it, like it (based on their biased personal preference) and maybe implement it. I don’t find any argument about bungee jumping convincing, but I cannot argue that arguments in favor of doing it are valid.

Summary

Summarizing the contents of the whole discussion, two reasons have mainly been given to reject this issue:

  1. Dart team is generally not in favor of it and have currently other priorities
  2. Private member access would break the consistency of the package system

Number 1 is subjective and it’s not a reason to close the issue because external developers could implement it or it could be considered to be implemented in the future if it was a common request. Consumers preferences are generally more important that Dart team members personal preferences (the team produces the product for consumers) and leaving the issue open gives the opportunity to perceive those preferences. Number 2 has been logically proven to be an invalid reason when the right barriers are enforced to enable private member access.

Given that there are no reasonable justifications to close this issue, closing it can only be based on a whim, something that is certainly a disrespectful action towards the people that would like this feature and towards me, for making a considerable effort to express my point of view as eloquently and clearly as I could. I would like to kindly request that this issue remains open, because closing it is certainly a moral mistake. I can agree that there is no reason to keep discussing, because both points of view are quite clear, although it would be good to define the preferred syntax (the friend approach seemed the best) and conditions in case anyone ever wished to implement this feature.

leafpetersen commented 4 years ago

It is my understanding that this forum is supposed to keep track of the Dart feature requests

That is not correct. From our Readme:

This repository is a place for the Dart language team to work on language changes and features, and to solicit and accept feedback and requests.

This is the repository in which we do our work. We are very open to receiving and considering feature requests, but this repository is not intended to be an open ended list of all features that anyone has ever requested.

Given that there are no reasonable justifications to close this issue, closing it can only be based on a whim, something that is certainly a disrespectful action towards the people that would like this feature and towards me

There is nothing whimsical about closing this, and there is no disrespect intended. It is simply not in the charter of this issue tracker to keep issues open indefinitely that we have no intention of acting on.

This thread is now closed.