search

JuliaLang / TOML.jl

A fast TOML parser for TOML 1.0 written in Julia
MIT License
34 stars 15 forks source link

Debugging the lcov.info file #26

Closed DilumAluthge closed 3 years ago

codecov-commenter commented 3 years ago

Codecov Report

Merging #26 (65e954f) into master (ff2f1bb) will increase coverage by 28.51%. The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           master      #26       +/-   ##
===========================================
+ Coverage   66.66%   95.17%   +28.51%     
===========================================
  Files           3        3               
  Lines         681      477      -204     
===========================================
  Hits          454      454               
+ Misses        227       23      -204
Impacted Files Coverage Δ
src/print.jl 93.75% <0.00%> (+24.94%) :arrow_up:
src/parser.jl 95.32% <0.00%> (+29.78%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update ff2f1bb...65e954f. Read the comment docs.

DilumAluthge commented 3 years ago

It seems like some of the jobs are just not generating the .jl.XXX.cov files. That seems like it might be a bug in the julia-runtest action.

KristofferC commented 3 years ago

Even so, all jobs should execute almost the same code so it should still show things as covered?

DilumAluthge commented 3 years ago

Even so, all jobs should execute almost the same code so it should still show things as covered?

Correct.

There's one (or more) suspicious things going on here.

DilumAluthge commented 3 years ago

The lcov.info files that are being generated on CI just seem.... incorrect.

DilumAluthge commented 3 years ago

Okay, here is a breakthrough:

It's not just the lcov.info file that is incorrect. The .jl.XXX.cov files are also incorrect, at least on Julia 1.0.

Look at one of the Julia 1.0 jobs that does have .jl.XXX.cov files. Look at the parsers.jl.XXX.cov, and specifically at the same place we've been looking, i.e. the reinit! function on line 121. That coverage information is incorrect:

From https://github.com/JuliaLang/TOML.jl/pull/26/checks?check_run_id=2747983477

        - function reinit!(p::Parser, str::String; filepath::Union{Nothing, String}=nothing)
       32     p.str = str
       20     p.current_char = EOF_CHAR
       20     p.pos = firstindex(str)
       20     p.prevpos = 0
       20     p.column = 0
       20     p.line = 1
       20     p.marker = 0
       20     p.root = TOMLDict()
        -     p.active_table = p.root
        -     empty!(p.dotted_keys)
        -     empty!(p.chunks)
        -     empty!(p.inline_tables)
        -     empty!(p.static_arrays)
        -     empty!(p.defined_tables)
        -     p.filepath = filepath
       20     startup(p)
       20     return p
        - end
DilumAluthge commented 3 years ago

But how would coverage generation be broken on Julia 1.0? We haven't even made a new release of Julia 1.0.x recently.