invenia / Memento.jl

A flexible logging library for Julia
https://invenia.github.io/Memento.jl/latest
Other
87 stars 14 forks source link

Add helpful warning on logger serialization failure #114

Closed omus closed 5 years ago

omus commented 5 years ago

I've encountered an issue with the following MWE:

using Distributed, Memento
const LOGGER = getlogger()

addprocs(1)

@everywhere begin
    using Memento

    struct AsyncHandler <: Handler{Union{}, IOBuffer}
        buf::IOBuffer
        channel::Channel{Record}

        function AsyncHandler(buf::IOBuffer)
            channel = Channel{Record}(Inf)
            handler = new(buf, channel)

            task = @async process_record!(handler)
            bind(channel, task)

            return handler
        end
    end

    function process_record!(handler::AsyncHandler)
        while isopen(handler.channel)
            record = take!(handler.channel)
            println(handler.buf, record[:msg])
        end
    end

    function Memento.emit(handler::AsyncHandler, record::Record)
        put!(handler.channel, record)
    end

    push!(getlogger(), AsyncHandler(IOBuffer()))
end

pmap(1:3) do i
    info(LOGGER, i)  # ERROR: cannot serialize a running Task
end

Originally I encountered this issue with using the handler in CloudWatchLogs.jl which also uses an @async task for handling logs.

The obvious answer is to not use a shared logger across multiple processes but in practise most packages as it'll be slower than using the logger available on each worker. Unfortunately the pattern of the referencing a global logger is pretty common so I expect that this will not be the last time it will be encountered.

This PR adds in a warning when serialization of a Logger fails.

codecov[bot] commented 5 years ago

Codecov Report

Merging #114 into master will increase coverage by 1.06%. The diff coverage is 71.42%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #114      +/-   ##
==========================================
+ Coverage    96.5%   97.56%   +1.06%     
==========================================
  Files          11       12       +1     
  Lines         286      288       +2     
==========================================
+ Hits          276      281       +5     
+ Misses         10        7       -3
Impacted Files Coverage Δ
src/Memento.jl 100% <ø> (ø) :arrow_up:
src/exceptions.jl 50% <50%> (ø)
src/loggers.jl 98.95% <80%> (-1.05%) :arrow_down:
src/handlers.jl 97.5% <0%> (+2.37%) :arrow_up:
src/records.jl 100% <0%> (+3.33%) :arrow_up:
src/formatters.jl 100% <0%> (+6%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update dccd27d...ed97c9b. Read the comment docs.

omus commented 5 years ago

Feel free to squash