Add method stub for update! or add Base.Defs

[tbreloff]

There are many community packages which define methods like:

update!(obj::MyType, args...) = # update obj

Some packages export it, some don't. It causes name clashes or API annoyances when you can't import or export the method.

Can we add a stub to Base? It doesn't need to be exported... just some way for us to all use the same method.

function update! end

And on a more general note... can we possibly add a submodule to Base which allows this sort of community sharing by allowing a package to add a brand new method to Base? I'm envisioning:

module A
    # no definition of update! yet
    import Base.Defs: update!

    export update!

    type AT end
    update!(obj::AT) = obj
end

module B
    # there's now a Base.Defs.update!... add to it
    import Base.Defs: update!

    export update!

    type BT end
    update!(obj::BT) = obj
end

julia> using A, B

julia> update!
update! (generic function with 2 methods)

This would solve so many problems in package dependencies that exist only because we can't do this.

[JeffBezanson]

Related: #2327

[tbreloff]

Yes thanks for the link. The feeling seems to be "that would be nice, but the current situation is fine". If it's a technical hurdle to overcome, I can understand the hesitancy, but I don't think that's the case. We let people add to existing Base methods, with a rough social contract for proper use... this is even less likely to cause issues, but would make life in package-world so much easier.

[tkelman]

Could easily be done in a package as EmptyMethods.jl without needing to wait for any base changes, right? And if base decides to export something in the future then it would only need to be fixed in once place, EmptyMethods could change to import the base version if isdefined there.

[tbreloff]

@tkelman Isn't the problem that you can't add a new method to an external module? If you can do this now then I agree it can go in a package, but I'm not sure how to do it.

[tkelman]

Such a package would just be a list of names. If anyone thinks multiple packages should be extending the same generic function instead of defining their own that would conflict, you would add it to the list. It wouldn't export anything, just be a single home namespace for common method tables to be based from.

[tbreloff]

So, lets say I create a package named MethodStubs.jl and added a bunch of stubs like:

function update! end
# etc

Then:

1) Would the core devs accept this into JuliaLang and promote this as a solution? 2) Would package developers be happy to add this dependency?

[johnmyleswhite]

I think I'm a far outlier (and I've radically changed my position over the last few years), but I'd personally like to see Julia stop encouraging the use of global function names for functions that don't have a single unambiguous SLA that's being reimplemented for new types. I'm cool with a totally generic sort that's available in the global namespace, but even gradient has turned out to be a problem that annoys me pretty regularly now since I want it to mean something different than it means in Base.

At a certain level of abstraction, update! strikes me as generic. But I think that level of abstraction isn't high enough for me to use update! in a generic way that I believe should work across all packages safely. I can't just write code like the following:

function foo(x, y)
  for i in 1:n
    update!(x, y)
  end
end

Unlike something like sort!, I really have no idea what that code would go because the meaning of update! isn't invariant across all types.

I think update! is a special case where multiple dispatch and namespaces are in direct conflict: to use update! without module-level namespace qualification, we really need it to be a method whose namespace is defined by its first argument, so it's written as x.update!(y).

[tkelman]

The downside of multimethods (or maybe the way we do them? do CLOS or Dylan do any differently here?) is that method tables are shared global mutable state.

[lobingera]

I somehow understand both Tom's and John's position. And all this might lead to a turning point in julia development.

Anything you read about Julia very early states: Multiple Dispatch. As far as i understand it, means: function name PLUS types of the argument(s) define, which code is executed. Following this (and i might have wrong concept of multiple dispatch) logic says: Julia should support anything that adds a new definition of name plus arguments regardless how name is defined. Locally, part of a module, exported.

[tbreloff]

I'd personally like to see Julia stop encouraging the use of global function names for functions that don't have a single unambiguous SLA that's being reimplemented for new types.

While I totally see the motivation for this, and the elegance of it:

• It's not practical -- there are too many potential meanings of a method
• It's not fair -- who gets to decide what the "one true meaning" is?
• The implications are verbose and ugly

One point that I think is very important... context matters. Just as with spoken language, we don't need to explicitly state everything, we infer meaning from the entirety of a sentence. This allows us to have many definitions of the same word. Consider:

BirdLifeCycles.hatch(egg)
EvilVillains.hatch(plan)

Do we really want/need to specify the module??

One of the great things about Julia is that it (usually) allows us to write code as we speak... using context in the form of multiple dispatch. Allowing many simultaneous definitions of a method, without the need to explicitly attach module ownership, is (IMO) akin to allowing us to code the way we think and speak.

tl;dr I want to code with context... it's natural and efficient. To do this I need to attach many different meanings to a method.

[tkelman]

Do we really want/need to specify the module??

I would, for the sake of code clarity. Context that you understand won't be obvious to all readers. Julia allows really short code that's optimized for author keystrokes, but I don't think that's always the style we should encourage for common library code. Not suggesting we go full Java here, but there's a balance.

[Sacha0]

It's not practical -- there are too many potential meanings of a method It's not fair -- who gets to decide what the "one true meaning" is? The implications are [...] ugly

These points could stand identically in an argument against namespace pollution :).

[johnmyleswhite]

Anything you read about Julia very early states: Multiple Dispatch. As far as i understand it, means: function name PLUS types of the argument(s) define, which code is executed. Following this (and i might have wrong concept of multiple dispatch) logic says: Julia should support anything that adds a new definition of name plus arguments regardless how name is defined. Locally, part of a module, exported.

I think if you follow this idea far enough, you conclude that all function names should live in the global namespace and be distinguished solely by the types of their arguments.

In fact, the initial public release of Julia behaved almost exactly in this way. And it was not pleasant to work with, as you essentially had to recreate the concept of namespaces using types-as-names: instead of having Base.parse and JSON.parse, you needed to have parse(s::String) and parse(::JSON, s::String). Dealing with this was possible, but it wasn't very enjoyable for me or the others who rejoiced when the module system was added.

[lobingera]

@johnmyleswhite I agree with you that it's certainly not a good idea to run a language without any namespace/modularisation/encapsulation features. Still, there are situations in which you want to combine methods into the same 'context' (as Tom call's it above). Like extending the API of a module. Or this "import X from Y as Z" situations.

[bramtayl]

I agree with @johnmyleswhite. I think that the idea can be distilled down to a general convention:

If the name of your module is a type or file format, do not define general functions with a matching desired return type/file format as an argument. Instead, rely on the module annotation for dispatch. This could do away with functions like, for example, write_csv. This also suggests the general principle of organizing modules around a return type.

[lobingera]

@bramtayl

If the name of your module is a type or file format, do not define general functions

else?

[bramtayl]

Else, if return type/file format dispatch is desired and it cannot be inferred from input argument types, do define general functions with a return type/file format as an argument. I suppose this would also require a type system for file formats (which would also help on dispatch for functions like read). And it would require stubs. Although I think much of this could be avoided with a more strict organization of modules around return type. And it might be worth figuring out how to make a return type being dispatched on available for optimization.

[tbreloff]

There are times when a method is completely ambiguous (parse(s) --> Expr vs parse(JSON, s) --> JSON vs parse(Int, s) --> Int) and there absolutely needs to be a specification of some sort of the module or type that is appropriate. In this specific case, I think parse is inappropriate altogether and these should be treated as constructors:

Expr(s)
JSON(s)
Int(s)

as opposed to the current behavior:

julia> parse("5")
5

julia> typeof(ans)
Int64

julia> parse("sin(5)")
:(sin(5))

julia> typeof(ans)
Expr

But there are many other cases which are not ambiguous, and it's those non-ambiguous cases that I'm focused on.

To be clear, I'm not trying to do away with namespaces or explicit coding for "library code". I'm trying to make it easier for packages to collaborate naturally, without making a massive web of unnecessary package dependencies, by adding "new definitions of a verb in distinct and non-ambiguous contexts".

[tkelman]

given #2327 was closed, I think this should be as well