JuliaLang / julia

The Julia Programming Language
https://julialang.org/
MIT License
45.14k stars 5.43k forks source link

Unclear closure behavior in asynchronous and parallel tasks #23130

Open appascoe opened 6 years ago

appascoe commented 6 years ago

Discourse thread for reference: https://discourse.julialang.org/t/inconsistent-results-based-on-number-of-processes/5191

At the very least, there's a documentation issue. The example of a network server that can service multiple connections has a critical flaw because, we believe, the changes in closures from 0.4 to 0.5:

server = listen(2000)
while true
    sock = accept(server)
    @async begin
        x = readline(sock)
        write(sock, x)
    end
end
In [1]: import socket

In [2]: s = socket.socket()

In [3]: t = socket.socket()

In [4]: s.connect(('', 2000))

In [5]: t.connect(('', 2000))

In [6]: s.send('check 1\n')
Out[6]: 8

In [7]: t.recv(1024)
Out[7]: 'check 1'

Because a second connection was opened before there was a write to the first connection, sock got overwritten, causing the write on the first connection to be echoed on the second. This makes asynchronous and parallel tasks more difficult to reason about, and it's not obvious that this is expected/desired behavior.

vtjnash commented 6 years ago

Yes, that example would be better written as

server = listen(2000)
while true
    let sock = accept(server)
        @async begin
            x = readline(sock)
            write(sock, x)
        end
    end
end
appascoe commented 6 years ago

The practice of enforcing closures should probably also be a section in the Parallel Computing chapter as well.