Open StefanKarpinski opened 3 years ago
I like all of that.
Duplicate of https://github.com/JuliaLang/julia/issues/8000
I don't think using Module
should be replaced by something unintuitive like import Module...
, as it was probably the most popular variant pre-1.6.
I feel that way julia would change to more python-like style when you shouldn't from module import *
unless you are not writing quick scripts for yourself; so it would be less convenient.
When someone wants to extend an explicitly imported generic function, they have to fully qualify it.
Did you mean "an implicitly imported generic function"?
I think it's all good, except I would drop import Foo...
since we don't need two ways to do the same thing. I assume the goal was to make something close to as easy as using Foo
for interactive use... what if using Foo
was still allowed, but only in interactive use cases. In a file, only import
variants work, but leave using
for the interactive case. Just like scope depends on interactive vs file.
How is the current "using
multiple modules in a single line" variation intended to be written in this scheme?
E.g., would using X, Y
become import X, Y: ...
?
I apologize in advance for bikeshedding, but maybe it would be more ergonomic to base the new import system on the defaults adopted by using
:
using Foo.Bar, Foo.Bar2 as Baz, Foo.Bar3 as _
The above creates soft bindings to the exported symbols within Bar
, Bar2
and Bar3
, and also makes Bar
and Baz
available.
using Foo.Bar: baz, qux
Creates hard (extendable) bindings to Bar.baz
and Bar.qux
, but does not import any other bindings implicitly. Bar
is visible as well, and can be optionally renamed with as
or discarded with _
.
using Foo.Bar: baz, qux, ...
Same as before, but also creates soft bindings to everything else in Bar
, inspired by varargs syntax.
Edit: importing just the module name without any contained bindings could be done with
using Foo.Bar: _
~Edit 2: AFAICT, this version of the proposal would only add new functionality to using
, without changing any existing semantics (unlike line 2 in Stefan's table). If true, this may allow us to start cleaning things up without waiting for 2.0. (import
would continue its redundant existence until 2.0 is released). But it would take someone more familiar with Julia's module system to tell whether the above proposal is truly non-breaking.~
I feel that way julia would change to more python-like style when you shouldn't
from module import *
unless you are not writing quick scripts for yourself; so it would be less convenient.
I hate from Module import blah
with all my guts and will die on that hill. One of the major benefits of being able to do all imports with a single keyword is that a block imports can look more consistent and is easier to read. If some imports look like import Foo
and then interspersed with that are these hideous from Bar import baz
statements, then not only is that benefit lost, but I might as well just stick needles in my eyes and then I don't have to worry about this problem anymore.
I think @s-valent's point is not that we should adopt that syntax, but rather a comment that this proposal would put us down the same road as Python, which discourages from module import *
, when using
seems to be the most common variant (and is mandated by two popular style guides).
I absolutely did not suggest to add from module import *
, the point of my message was that using Module
is a really nice way to do the same thing as from module import *
in Python.
Removing using
in favor of import Module...
would change it to something less elegant, and switch focus to mainly use import Module
over import Module...
(as it would feel the preferable way to go + it's shorter)
Looking at the table, the new form doesn't appear simpler or clearer to me. I'd argue it makes the syntax more complex. Some of the changes can also be done without removing using
.
Additionally, and as mentioned above, how would the current using A, B, C
work? import A, B, C: ...
looks like you are doing only import A, B; using C
, while import A..., B..., C...
is lengthy.
I would vote to keep the using
keyword.
The proposed hard vs soft binding changes would get a 👍 from me.
It sounds like with the proposal there's no way to have the semantics of using Foo: bar
where bar
is explicitly imported but needs to be fully qualified to be extended. I'm a fan of always fully qualifying when extending, and IMO it's nice to be able to enforce that (although it could also be caught by linting).
Here are the things one may want to do when importing a package, along with how to do them currently:
import Foo
using Foo
import Foo: bar, baz
using Foo: bar, baz
The main motivations for my proposed changes are:
import
and using
;import
instead of having to change back and forth between import
and using
because of the aforementioned subtle and confusing semantic difference.A non-objective is to make it possible to import things from multiple packages in a single statement. I'm fine with that when there's an obvious and convenient syntax for it, but it's just not one of my motivations.
One of the annoyances of the current import
is that import Foo: bar
does not make Foo
available, even though you often want to refer to it. As a result, if you want Foo
to be available, you either need to write import Foo
and import Foo: bar
or write import Foo: Foo, bar
. Both are pretty annoying. The motivation for the current design was that it gives you the simplest most atomic import behavior: it makes a single binding in a package available without doing anything else. If it did more, as I'm proposing, then you need a way to opt out of that behavior. Now, however, we have a way to opt out of that behavior by writing import Foo as _: bar
. So at this point I think it would make sense to make Foo
available whether you write import Foo
or import Foo: bar
.
The explicitly named import versus implicit import rule addresses the subtle distinction between import
and using
and seems like a clearer rule to me in general. I would also be ok with requiring qualification to extend imported functions, but there's definitely cases where that's annoying. It does also seem like something that a linter would be better for.
The reason to use import Foo: ...
as a syntax for importing "the rest" i.e. the exported names in Foo
is so that you can do all imports from a package on a single line. Otherwise, if you want to do some combination of 2 and 3 or 4 in the list off use cases, you are forced to do using Foo
and import Foo: bar, baz
which is annoying and ugly. With the ...
syntax, you can instead just write import Foo: bar, baz, ...
which is much nicer.
It would also be viable to keep using Foo
as a syntax for import Foo: ...
. That would still allow all imports to be written with import
which is actually the main thing I care about. I would probably keep using that in the REPL since it's convenient, but use import
only in packages, which helps organize and clarify the imports.
just FYI, a keyword from
might be necessary for #4600 to distinguish the semantic between files and module identifiers.
I'll repeat what I said before: over my dead body are we introducing from Foo import bar
syntax.
I'll repeat what I said before: over my dead body are we introducing from Foo import bar.
Well not exactly from Foo import bar
, that proposal's from
can only take strings, so it is distinguishable with module identifiers, but could also be import "path/to/file.jl": bar, baz
if you don't feel ":
less readable, but both can potentially affect this issue. (since #4600 needs an explicit semantic to distinguish file path)
For what it's worth, I'm pretty far from sold on the file import discussed in #4600, but I think that's fairly orthogonal to this issue. If we have import "path/to/file.jl": bar, baz
then I think it should load that file in an anonymous module and import the names from it, which doesn't really interact with these changes in any way that I can see.
I'll repeat what I said before: over my dead body are we introducing from Foo import bar.
Maybe wer're just using it wrong :smile:. I could see from
being useful for specifying an optional source file, to replace the include("foo.jl"); using .Foo
pattern:
import Foo from "foo.jl" as F: bar, baz, ...
OK. Sorry for the noise.
You don't need the from
in that proposal though. You could write import "foo.jl" as Foo: bar, baz, ...
. But that's also more relevant to #4600 than it is to this issue.
Actually, I was thinking of the case where "foo.jl" contains an actual, declared module Foo. If import "foo.jl"
imports the file as an anonymous module and brings its identifiers into the current scope then the result would be (I think) that Foo would be brought into scope, without any names exported by it. import "foo.jl": bar, baz
would simply fail in that case, because "foo.jl" does not contain such names at the top level. It would also not work for importing something from a file that contains multiple modules. But maybe I'm thinking too far ahead.
We should really have this discussion on #4600, not here.
using
is nice in that people would wonder what's got introduced to the namespace (then they find out about export
), import Foo...
or import Foo:...
has the from Foo import *
vibe... which is usually a bad practice.
This probably isn't a popular opinion, but I'd be all for just removing the using Foo
/ import Foo...
syntax altogether. (Or perhaps mandate that it work in the REPL only.) Which would accomplish the stated goals of this proposal.
The equivalent Python syntax from Foo import *
brings nothing but heartache, the number of junior dev projects using it makes me want to scream, and never ever never using imports with this behaviour is the hill that I'm quite ready to die on.
using
is nice in that people would wonder what's got introduced to the namespace (then they find out aboutexport
),import Foo...
orimport Foo:...
has thefrom Foo import *
vibe... which is usually a bad practice.
I don't really get this. If it's "usually a bad practice" how does the syntax vibe being similar to from Foo import *
cause an issue? If anything having to write import Foo...
or import Foo: ...
will make people think twice about doing this since it makes it explicit that you are importing an ill defined set of names.
This probably isn't a popular opinion, but I'd be all for just removing the using Foo / import Foo... syntax altogether.
This isn't going to happen since it's way too annoying to have to explicitly import all names for interactive use. There's a case to be made that it shouldn't be used in other situations, but it's really convenient there too and because of the way the mechanism is designed, it doesn't actually cause any problems unless two different packages export the same name with a different meaning, in which case you get a clear and specific error message. Perhaps things are worse in Python, but this isn't that big a deal in Julia. My objection to using
isn't the "danger" but that I don't like having to use inconsistent keywords for my imports and I don't like having to have multiple import lines for the same package.
how does the syntax ... cause an issue
I meant that people would think import Foo: ...
is equivalent to from Foo import *
(i.e. import everything into namespace); where using Foo
is different enough that people may stop a second and wonder what is actually happening. The main difference is ...
looks like an "everything" qualifier at a glance.
But it's a 2nd order effect, you can always explain on-site (when giving a talk maybe).
I agree that the ...
syntax could be misleading, and clunky when wanting to use multiple packages at once. I think one thing that wasn't really motivated is why only one keyword is desired. The various propositions above can be done while keeping using
.
If the main reason is "having two keywords can be confusing", then, to me it doesn't seem like it's worth it to cause such a break. I haven't seen anyone complain about this issue.
Mixing different keywords in the import section is visually messy, making it much harder to tell where the block of keywords is. Let's consider the imports in this file: https://github.com/JuliaLang/Pkg.jl/blob/master/src/Artifacts.jl, which I picked pretty much at random. Currently they look like this:
import Base: get, SHA1
using Base.BinaryPlatforms
using Artifacts
import Artifacts: artifact_names, ARTIFACTS_DIR_OVERRIDE, ARTIFACT_OVERRIDES, artifact_paths,
artifacts_dirs, pack_platform!, unpack_platform, load_artifacts_toml,
query_override, with_artifacts_directory, load_overrides
import ..set_readonly
import ..GitTools
import ..TOML
using ..MiniProgressBars
using ..PlatformEngines
import ..pkg_server, ..can_fancyprint, ..DEFAULT_IO, ..printpkgstyle
import ..Types: write_env_usage, parse_toml
using SHA
That's frankly, pretty messy looking. How would I prefer for this to look? Something like this:
import ..:
can_fancyprint, DEFAULT_IO, GitTools, MiniProgressBars..., pkg_server,
PlatformEngines..., printpkgstyle, set_readonly, TOML
import ..Types: parse_toml, write_env_usage
import Artifacts:
artifact_names, ARTIFACT_OVERRIDES, artifact_paths, ARTIFACTS_DIR_OVERRIDE,
artifacts_dirs, load_artifacts_toml, load_overrides, pack_platform!,
query_override, unpack_platform, with_artifacts_directory
import Base: get, BinaryPlatforms..., SHA1
import SHA: ...
This uses the import
keyword uniformly, putting all imports from each module on a single line. Once they are in this uniform syntax, you can sort the lines and give some order to the imports. If they're mixed up between import and using, then that doesn't work and it looks messy no matter how you organize it. That's not just an aesthetic problem: it makes the code harder to read. The second version is much easier to visually parse and understand what's going on.
As a more fair comparison, this is how I would write the imports with using:
using Artifacts, Base.BinaryPlatforms, SHA
using ..MiniProgressBars, ..PlatformEngines
import ..set_readonly, ..GitTools, ..TOML, ..pkg_server, ..can_fancyprint, ..DEFAULT_IO, ..printpkgstyle
import Base: get, SHA1
import Artifacts: artifact_names, ARTIFACTS_DIR_OVERRIDE, ARTIFACT_OVERRIDES, artifact_paths,
artifacts_dirs, pack_platform!, unpack_platform, load_artifacts_toml,
query_override, with_artifacts_directory, load_overrides
import ..Types: write_env_usage, parse_toml
I don't feel like there is much of a readability difference.
There is a small difference with the import-only version above in that it's missing the equivalent of using Artifacts
. Maybe you'd need to add ...
at the end of the import list.
So to be clear, @jebej, you are arguing for the status quo?
So to be clear, @jebej, you are arguing for the status quo?
Regarding the using
keyword, yes.
I understand the desire to only have only one keyword, but I feel that advantages are not worth the breakage and the loss of the concise using A, B, C
syntax.
I'm fine with leaving the using A
syntax as long as I can use import
for everything. Having both import A: b
and using A: b
with very slightly different meanings seems like a mess though.
This probably isn't a popular opinion, but I'd be all for just removing the
using Foo
/import Foo...
syntax altogether. (Or perhaps mandate that it work in the REPL only.) Which would accomplish the stated goals of this proposal.The equivalent Python syntax
from Foo import *
brings nothing but heartache, the number of junior dev projects using it makes me want to scream, and never ever never using imports with this behaviour is the hill that I'm quite ready to die on.
Just for posterity, I feel I should mention explicitly that using Foo
is not the same as from Foo import *
, because using Foo
only gets you the names that were explicitly exported from Foo
and are hence part of it's public interface.
One thing I'd really like is a way to using
a function but exclude a name that I know will cause a collision. E.g. something like
using Foo except bar
I know this is semi-orthogonal, but I think it'd be important to think about how a unified syntax like this might be generalized further. Would the above (currently non-syntax) become something like
import Foo: ... except bar
? I guess that's not as bad as it seemed in my head, but it does feel a little less clear than the using
version.
Sorry if this is noise. But from the perspective of someone which is not a developer of "core" packages, the distinction of using
and import
is somewhat reassuring. I do not import
something very often, because it is not that common to me to have to extend functions from other packages. When I need to, the fact that I need to import
the function with a clearly different syntax makes the code clearer. I usually do that just before the new method definition.
I understand that for packages that are extending many functions of other packages having one syntax is nice, and I do not see any reason to not allow all that flexibility to import
. But I think that an important (and growing) part of the Julia users will only require using
packages most of the time. import is, from my point of view, a more flexible and advanced way to interact with a package codebase, and I just feel that some confusion could be avoided for newbies if it was documented like that, even if if import
turns out to have overlapping behaviours with using
.
(as a side note: why not import Foo: *
? I know splatting has its place in Julia, but the asterisk is universal...)
@lmiq because using
is whitelist-based (only import what is export
ed), import Foo: *
suggests other wise.
@Moelf but is import Foo...
any clearer? I do not see why. IMHO, it is simply not clear about what it does (one would have to refer to the docs, and learn that things that are not exported have to be always explicitly imported anyway).
import Foo: *
This suggests importing the *
operator from Foo
.
A way of evaluating how confusing or clarifying this change might be for users, is to compare the current table in the summary of module usage in the docs with how it would look after the change.
If I got the intention right, the proposed change would mean: (a) dropping the second line (using MyModule: x, p
); (b) adding MyModule
in the column "What is brought into scope" everywhere; (c) changing the first cell (using MyModule
) by import MyModule...
.
From my point of view, (a) does indeed simplify things; (b) makes it more consistent - import MyModule: x
is more than import MyModule
, not just different; (c) fits nicely with the whole scheme if it does not remain in the first row, but becomes the last one. But definitely I would keep using
as an alias for convenience.
I don't see the point of import Foo: ... except bar
. The only reason to do that is that some other package, say Baz
, exports bar
as well and we want to import Baz.bar
instead of Foo.bar
. In that case, however, it would be much clearer and more foolproof to write import Baz: bar
to make sure that we get that one.
This would fix https://github.com/JuliaLang/julia/issues/39235
In fact, this is #39235.
ERROR: StackOverflowError:
Stacktrace:
[1] goto(::#39235) (repeats 79984 times)
@ Main ./REPL[5]:1
Of course, that is why it would fix it. Oh, and additionally, it would fix https://github.com/JuliaLang/julia/issues/29275.
FYI, as someone who has high myopia, using
is much easier to read than ...
or *
or any other punctuation simply because words are a bit larger and has a quite unique visual pattern comparing to punctuations. (same reason why I prefer from
over :
in #4600 ) but I guess not a problem for most people 🤷♂️
make all explicit bindings hard and all implicit bindings soft
Would we lose the ability of making explicit soft bindings, as in
using Foo: some_function
? Personally, in packages I always prefer to have an explicit list of symbols I am using from other packages and not rely on export lists, so using Foo
is something I try to avoid in package source code.
My personal favorite would be to make the import statement fully generic and have the export statement support the same syntax to allow for different ways of exporting.
Okay, so where are we?
import list, elements, to, be, imported
#well yes, syntactically kinda looks like
return multi, argument, response
in the latter case it's syntactical sugar for a tuple. So what happens if we'd interpret the import statement as using a tuple?
We could ask, what about named tuples? This immediately leads to an intuitive understanding of named imports.
import OtherModule: foo=bar, foo2=bar2
Optionally we can think whether we'd prefer/allow writing bar as foo
over foo=bar
, kinda like syntactical sugar. (I would prefer as
).
I remember a certain rule about individual labels after a semicolon in named tuples, i.e., (;label)
expanding to (label=label)
. For the context here, what would binding Foo.bar
to bar
mean? Exactly, that would be a hard binding!
#old:
using Foo: soft
import Foo: hard
#new (long):
import Foo: soft, hard as hard
#new (short by named tuple mechanics):
import Foo: soft; hard
So including the splat operator this could lead to:
#old
using Foo
using Foo: soft, binding
import Foo: hard, bind
#new
import Foo: ..., soft, binding; hard, bind
#maybe also allow
import Foo
#as shorthand for
import Foo: ...
#which previously was
using Foo
On the way to even more generalization, I now can ask, what happens if I splat behind the semicolon (=the hard binding area)? We'll get back to that later.
Now, that we have a generic import, let's make export generic as well.
export Bar: soft, binding, ... #would export soft and binding from Bar and also all exports from Bar -> reexport
export bind # exports local "bind" as usual
I could see some uses for that reexporting
Given that we have a hard binding for import, want it fully generic and still need a good use for the splat in the hard import, what could we do with the semicolon syntax for export?
we could make them named exports!
- well, yes, but those wouldn't be any better than just exporting a renamed variable. So that won't provide any advantages...
Thinking a bit more
We could make this "hard export" kinda like talking to the hard import splat!
I.e., a soft import splat will just ignore the hard export and vice versa. The "hard import"-splat will import (and bind) everything that's in "hard export" and the "soft import"-splat will import (softly) everything that's in the "soft export". That way a developer would be able to have 2 different export sets: one for ordinary soft binding, used for ordinary calling usage. And one which will go into the splat that does a hard bind. Which makes it perfect for marking functions whose purpose is to be extended. Hard exports and soft exports don't have to be disjunctive, but they can. That also makes a lot of sense by itself, given the purpose of the hard bindings. From the documentation level this is basically hinting which functions are meant to be used and which are meant to be extended (kinda like User vs Extension exports), while forcing neither to be imported. The remaining fields which aren't part of either list, could be assumed to be internal in their nature.
module Exporter
export soft, soft2, soft3; hard, hard2
end
module ImportSoft
import Exporter: ...
#loads soft, soft2 and soft3 without doing a hard bind
end
module ImportHard
import Exporter: soft; soft2, ...
# will soft bind "soft" and hard bind "soft2", "hard" and "hard2" but not load soft3
end
well, in order to be able to have both export sets disjunctive, we cannot make either be included in the other set. More precisely, automatically exporting soft-export bindings as hard exports doesn't make sense because those usually aren't meant to be extended, and the other way around doesn't make sense either in many cases. For those cases where I want all soft exports to be exported as hard exports as well, we could make the "splat" in the hard export refer to all soft exports and vice versa. The case that both exports are splatted would just make both sets the same. (=union of all explicit labels)
Introduce an exclamation mark for exclusion if needed. This would need special casing the operator itself but since functions cannot start with an exclamation mark, and it usually referring to negation I find the choice quite intuitive. (aside of !!
which would mean to exclude the operator)
module Bar
import Foo: ..., !excluded
#imports everything exported by Foo EXCEPT "excluded"
end
import Module as Alias: <soft binding labels>; <hard binding labels>
export <goes into soft binding splat>; <goes into hard binding splat>
export OtherModule: <into soft binding splat>; <into hard binding splat> #-> reexport
I hope all these ideas spark a fruitful discussion whether we need a keyword instead of the semicolon for better readability or whether this is fine just as proposed. Given that we already have trained eyes for spotting the semicolon thanks to keyword arguments, I find it a valid option. Especially since you only need the semicolon part if you're a library developer or intend to extend a library (and don't want to use full qualification). An ordinary user who is just chaining method calls can stick to not using semicolon at all.
Coming from Common Lisp where I have acquired a very opinionated idea of readable code, I am not very keen on using ...
or import X: ...
. In my own Julia code, I prefer to almost always use import Foo as F
, as it makes code very clear at call sites which module a symbol comes from, rather than polluting a module with external symbols. Perhaps I am biased for thinking this makes for clearer code, but that's what I think. I would like to see more of an emphasis on style guides (and 2.0 proper) in favor of this idea. That way code reads very clearly without the verbosity of long chains of dotted module qualifiers, or the worse case of not knowing where a symbol even comes from when first learning a piece of code. Just my 2 cents. Take it or leave it :)
Regardless of the eventual syntax for using
or import
, I wish they also could be called like functions, e.g., using(:MyRepo)
. My use case is in make.jl
files for Documenter where the repo name appears in many places. I end up using eval(:(using $repo))
which is tolerable but a bit ugly...
It probably won't happen but I had to mention it.
I would very much like to have just one import keyword in Julia 2.0. I don't much care if it's
import
orusing
but since we talk about it as "importing" things, that may be more natural. With https://github.com/JuliaLang/julia/issues/39187 and some requirement for explicitly requesting extension of external generic functions, this would be possible.import Foo
import Foo
Foo
import Foo: Foo, bar
import Foo: bar
import Foo: bar
importsFoo
alsoimport Foo: bar
import Foo as _: bar
Foo as _
to discard that nameusing Foo
import Foo...
using Foo
import Foo: ...
using Foo; using Foo: bar
import Foo: bar, ...
using Foo as _
import Foo as _: ...
Foo as _
to discard that nameIn 2.0 I think we should eliminate the distinction between the "soft binding" that
using
creates and the hard binding thatimport
creates and just make all explicit bindings hard and all implicit bindings soft: if you asked for it by name, it's a hard binding, if you didn't, it's a soft binding. When someone wants to extend an implicitly imported generic function, they have to fully qualify it.