invenia / Memento.jl

A flexible logging library for Julia
https://invenia.github.io/Memento.jl/latest
Other
88 stars 14 forks source link

`@test_log` should support partial matching of `msg` #57

Closed spurll closed 6 years ago

spurll commented 6 years ago

The @test_warn and @test_error macros in Base.Test some functionality that would be very helpful for Memento to support. From the documentation:

Test whether evaluating expr results in STDERR output that contains the msg string or matches the msg regular expression. If msg is a boolean function, tests whether msg(output) returns true. If msg is a tuple or array, checks that the error output contains/matches each item in msg. Returns the result of evaluating expr. [Emphasis added.]

I don't know that supporting msg::Function or msg::Array is necessary, but having checking based on a partial match of the message ("output contains the msg string or matches the msg regular expression") would be helpful because we often won't know exactly what message we're going to get. (The can contain some sort of UUID or something that's difficult/impossible to predict.)

Matching the functionality of the Base.Test macros would be great, but if @test_log could support testing whether the handler's message contains msg (rather than equals msg) at a minimum that would be really helpful. (It would eliminate a few dozen lines of code from several tests.)

Examples

Here are some trivial examples of what Base does that I'd like to see work in Memento as well:

julia> using Base.Test, Memento

julia> logger = getlogger()
Logger(root)

julia> @test_warn "Oh no!" warn("Oh no!")       # Full match

julia> @test_warn "Oh" warn("Oh no!")           # Partial match ("Oh no!" contains "Oh")

julia> @test_warn r"Oh\s.+!" warn("Oh no!")     # Regex match

julia> @test_log logger "warn" "Oh no!" warn(logger, "Oh no!")
[warn | root]: Oh no!

julia> @test_log logger "warn" "Oh" warn(logger, "Oh no!")
[warn | root]: Oh no!
Test Failed
  Expression: #336#handler.found == ((Memento.Test.String)("warn"), (Memento.Test.String)("Oh"))
ERROR: There was an error during testing

julia> @test_log logger "warn" r"Oh\s.+!" warn(logger, "Oh no!")
ERROR: MethodError: Cannot `convert` an object of type Regex to an object of type String
This may have arisen from a call to the constructor String(...),
since type constructors fall back to convert methods.
Stacktrace:
 [1] Type at /Users/gem/.julia/v0.6/Memento/src/test.jl:64 [inlined]
 [2] macro expansion at /Users/gem/.julia/v0.6/Memento/src/test.jl:18 [inlined]
 [3] anonymous at ./<missing>:?