fixing issues - Githubissues

TheCedarPrince commented 10 months ago

@Farreeda also, I re-enabled the GitHub actions to run the test suite and as you can see, tests are failing. I checked out your branch onto my computer and ran the test suite. Here was the test log I got:

     Testing Running tests...
[ Info: Eunomia data source is downloaded!
[ Info: Global database dialect set to: sqlite
[ Info: Global schema set to: main
[ Info: cdm_source table generated internally
[ Info: location table generated internally
[ Info: care_site table generated internally
[ Info: concept_ancestor table generated internally
[ Info: concept table generated internally
[ Info: concept_relationship table generated internally
[ Info: condition_era table generated internally
[ Info: cost table generated internally
[ Info: dose_era table generated internally
[ Info: drug_strength table generated internally
[ Info: note table generated internally
[ Info: cohort_attribute table generated internally
[ Info: person table generated internally
[ Info: provider table generated internally
[ Info: source_to_concept_map table generated internally
[ Info: cohort table generated internally
[ Info: concept_synonym table generated internally
[ Info: visit_occurrence table generated internally
[ Info: vocabulary table generated internally
[ Info: fact_relationship table generated internally
[ Info: domain table generated internally
[ Info: drug_era table generated internally
[ Info: procedure_occurrence table generated internally
[ Info: death table generated internally
[ Info: note_nlp table generated internally
[ Info: concept_class table generated internally
[ Info: device_exposure table generated internally
[ Info: payer_plan_period table generated internally
[ Info: drug_exposure table generated internally
[ Info: observation table generated internally
[ Info: relationship table generated internally
[ Info: specimen table generated internally
[ Info: observation_period table generated internally
[ Info: visit_detail table generated internally
[ Info: condition_occurrence table generated internally
[ Info: measurement table generated internally
[ Info: metadata table generated internally
GetDrugExposures Tests: Test Failed at /home/thecedarprince/FOSS/OMOPCDMCohortCreator.jl/test/sqlite/getters.jl:193
  Expression: Drug_exposure == GetDrugExposures(test_ids, sqlite_conn)
   Evaluated: 20×2 DataFrame
 Row │ person_id  drug_exposure_id
     │ Float64    Float64
─────┼─────────────────────────────
   1 │     573.0           26318.0
   2 │    1332.0           60926.0
   3 │     576.0           26418.0
   4 │    4550.0           54785.0
   5 │    3895.0           47027.0
   6 │    3199.0           38712.0
   7 │     476.0           21897.0
   8 │    1154.0           52764.0
  ⋮  │     ⋮             ⋮
  14 │    1892.0           86272.0
  15 │     817.0            9745.0
  16 │     820.0            9791.0
  17 │    1746.0           21217.0
  18 │    2230.0           27036.0
  19 │    1911.0           87122.0
  20 │    2636.0           31813.0
                     5 rows omitted == 505×2 DataFrame
 Row │ person_id  drug_exposure_id
     │ Float64    Float64
─────┼─────────────────────────────
   1 │      40.0            1807.0
   2 │      40.0            1809.0
   3 │      40.0            1812.0
   4 │      40.0            1820.0
   5 │      74.0            3404.0
   6 │      74.0            3406.0
   7 │      74.0            3408.0
   8 │      74.0            3410.0
  ⋮  │     ⋮             ⋮
 499 │      53.0             679.0
 500 │      53.0             680.0
 501 │      53.0             681.0
 502 │      53.0             682.0
 503 │      53.0             683.0
 504 │      53.0             684.0
 505 │      53.0             685.0
                   490 rows omitted

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.9.2+0.x64.linux.gnu/share/julia/stdlib/v1.9/Test/src/Test.jl:478 [inlined]
 [2] macro expansion
   @ ~/FOSS/OMOPCDMCohortCreator.jl/test/sqlite/getters.jl:193 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.9.2+0.x64.linux.gnu/share/julia/stdlib/v1.9/Test/src/Test.jl:1498 [inlined]
 [4] top-level scope
   @ ~/FOSS/OMOPCDMCohortCreator.jl/test/sqlite/getters.jl:189
GetDrugConceptIDs Tests: Test Failed at /home/thecedarprince/FOSS/OMOPCDMCohortCreator.jl/test/sqlite/getters.jl:201
  Expression: drug_concept_ids == GetDrugExposures(drug_exposure_ids, sqlite_conn)
   Evaluated: 20×2 DataFrame
 Row │ drug_exposure_id  drug_concept_id
     │ Float64           Float64
─────┼───────────────────────────────────
   1 │          26318.0        4.02132e7
   2 │          60926.0        4.02132e7
   3 │          26418.0        4.02133e7
   4 │          54785.0        1.11808e6
   5 │          47027.0        4.01625e7
   6 │          38712.0        1.90591e7
   7 │          21897.0        4.02133e7
   8 │          52764.0        4.02132e7
  ⋮  │        ⋮                 ⋮
  14 │          86272.0        4.02133e7
  15 │           9745.0        1.71367e6
  16 │           9791.0        1.91339e7
  17 │          21217.0        4.02368e7
  18 │          27036.0        4.0165e7
  19 │          87122.0        4.02132e7
  20 │          31813.0        1.12743e6
                           5 rows omitted == 0×2 DataFrame
 Row │ person_id  drug_exposure_id
     │ Float64?   Float64?
─────┴─────────────────────────────

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.9.2+0.x64.linux.gnu/share/julia/stdlib/v1.9/Test/src/Test.jl:478 [inlined]
 [2] macro expansion
   @ ~/FOSS/OMOPCDMCohortCreator.jl/test/sqlite/getters.jl:201 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.9.2+0.x64.linux.gnu/share/julia/stdlib/v1.9/Test/src/Test.jl:1498 [inlined]
 [4] top-level scope
   @ ~/FOSS/OMOPCDMCohortCreator.jl/test/sqlite/getters.jl:197
GetDrugConceptIDs Tests: Error During Test at /home/thecedarprince/FOSS/OMOPCDMCohortCreator.jl/test/sqlite/getters.jl:204
  Got exception outside of a @test
  FunSQL.ReferenceError: cannot find amount_value in:
  let DRUG_EXPOSURE = SQLTable(:DRUG_EXPOSURE, …),
      q1 = From(DRUG_EXPOSURE),
      q2 = q1 |> Select(Get.drug_concept_id, Get.amount_value),
      q3 = q2 |> Limit(20)
      q3
  end
  Stacktrace:
    [1] validate(t::FunSQL.RowType, ref::FunSQL.SQLNode, ctx::FunSQL.AnnotateContext)
      @ FunSQL ~/.julia/packages/FunSQL/Ufc3L/src/annotate.jl:868
    [2] validate(t::FunSQL.BoxType, ref::FunSQL.SQLNode, ctx::FunSQL.AnnotateContext)
      @ FunSQL ~/.julia/packages/FunSQL/Ufc3L/src/annotate.jl:844
    [3] gather_and_validate!(refs::Vector{FunSQL.SQLNode}, n::Vector{FunSQL.SQLNode}, t::FunSQL.BoxType, ctx::FunSQL.AnnotateContext)
      @ FunSQL ~/.julia/packages/FunSQL/Ufc3L/src/annotate.jl:886
    [4] link!(n::FunSQL.SelectNode, refs::Vector{FunSQL.SQLNode}, ctx::FunSQL.AnnotateContext)
      @ FunSQL ~/.julia/packages/FunSQL/Ufc3L/src/annotate.jl:1133
    [5] link!(boxes::Vector{FunSQL.BoxNode}, ctx::FunSQL.AnnotateContext)
      @ FunSQL ~/.julia/packages/FunSQL/Ufc3L/src/annotate.jl:957
    [6] link!(ctx::FunSQL.AnnotateContext)
      @ FunSQL ~/.julia/packages/FunSQL/Ufc3L/src/annotate.jl:943
    [7] render(catalog::FunSQL.SQLCatalog, n::FunSQL.SQLNode)
      @ FunSQL ~/.julia/packages/FunSQL/Ufc3L/src/render.jl:64
    [8] #render#311
      @ ~/.julia/packages/FunSQL/Ufc3L/src/render.jl:10 [inlined]
    [9] (::var"#89#93")(q::FunSQL.SQLNode)
      @ Main ~/FOSS/OMOPCDMCohortCreator.jl/test/sqlite/getters.jl:205
   [10] |>(x::FunSQL.SQLNode, f::var"#89#93")
      @ Base ./operators.jl:907
   [11] macro expansion
      @ ~/FOSS/OMOPCDMCohortCreator.jl/test/sqlite/getters.jl:205 [inlined]
   [12] macro expansion
      @ ~/.julia/juliaup/julia-1.9.2+0.x64.linux.gnu/share/julia/stdlib/v1.9/Test/src/Test.jl:1498 [inlined]
   [13] top-level scope
      @ ~/FOSS/OMOPCDMCohortCreator.jl/test/sqlite/getters.jl:205
   [14] include(fname::String)
      @ Base.MainInclude ./client.jl:478
   [15] macro expansion
      @ ~/FOSS/OMOPCDMCohortCreator.jl/test/runtests.jl:35 [inlined]
   [16] macro expansion
      @ ~/.julia/juliaup/julia-1.9.2+0.x64.linux.gnu/share/julia/stdlib/v1.9/Test/src/Test.jl:1498 [inlined]
   [17] macro expansion
      @ ~/FOSS/OMOPCDMCohortCreator.jl/test/runtests.jl:35 [inlined]
   [18] macro expansion
      @ ~/.julia/juliaup/julia-1.9.2+0.x64.linux.gnu/share/julia/stdlib/v1.9/Test/src/Test.jl:1498 [inlined]
   [19] top-level scope
      @ ~/FOSS/OMOPCDMCohortCreator.jl/test/runtests.jl:31
   [20] include(fname::String)
      @ Base.MainInclude ./client.jl:478
   [21] top-level scope
      @ none:6
   [22] eval
      @ ./boot.jl:370 [inlined]
   [23] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:280
   [24] _start()
      @ Base ./client.jl:522
Test Summary:                                       | Pass  Fail  Error  Total   Time
OMOPCDMCohortCreator                                |   43     2      1     46  21.5s
  SQLite Helper Functions                           |    2                   2   0.8s
  SQLite Getter Functions                           |   33     2      1     36  19.6s
    GetDatabasePersonIDs Tests                      |    1                   1   4.2s
    GetDatabaseYearRange Tests                      |    1                   1   0.6s
    GetPatientGender Tests                          |    1                   1   1.3s
    GetPatientRace Tests                            |    1                   1   0.2s
    GetPatientAgeGroup Tests                        |    2                   2   1.9s
    GetPatientVisits Tests                          |    2                   2   0.4s
    GetMostRecentConditions Tests                   |    2                   2   1.2s
    GetMostRecentVisit Tests                        |    2                   2   0.9s
    GetVisitConcept Tests                           |    1                   1   0.2s
    GetVisitCondition Tests                         |    2                   2   0.2s
    GetPatientEthnicity Tests                       |    1                   1   0.2s
    GetVisitDate Tests                              |    2                   2   0.3s
    GetDrugExposures Tests                          |          1             1   0.8s
    GetDrugConceptIDs Tests                         |          1             1   0.4s
    GetDrugConceptIDs Tests                         |                 1      1   2.1s
    GetPatientGender multiple dispatch Tests        |    1                   1   1.3s
    GetPatientRace multiple dispatch Tests          |    1                   1   0.1s
    GetPatientAgeGroup multiple dispatch Tests      |    1                   1   0.6s
    GetPatientVisits multiple dispatch Tests        |    1                   1   0.7s
    GetMostRecentConditions multiple dispatch Tests |    1                   1   0.4s
    GetMostRecentVisit multiple dispatch Tests      |    2                   2   0.1s
    GetVisitConcept multiple dispatch Tests         |    1                   1   0.0s
    GetVisitCondition multiple dispatch Tests       |    2                   2   0.0s
    GetPatientEthnicity multiple dispatch Tests     |    1                   1   0.2s
    GetVisitDate multiple dispatch Tests            |    2                   2   0.0s
    GetDrugExposures multiple dispatch Tests        |    1                   1   0.4s
    GetDrugConceptIDs multiple dispatch Tests       |    1                   1   0.8s
  SQLite Filter Functions                           |    7                   7   0.8s
  SQLite Executors Functions                        |    1                   1   0.3s
ERROR: LoadError: Some tests did not pass: 43 passed, 2 failed, 1 errored, 0 broken.
in expression starting at /home/thecedarprince/FOSS/OMOPCDMCohortCreator.jl/test/runtests.jl:30
ERROR: Package OMOPCDMCohortCreator errored during testing

Thankfully, there are only a few errors! Once you finish up with the small comments I made and fixing the test suite here, we are set to merge!

codecov[bot] commented 10 months ago

Codecov Report

Merging #54 (10d67d8) into main (77126d0) will increase coverage by 0.59%. The diff coverage is 76.11%.

@@            Coverage Diff             @@
##             main      #54      +/-   ##
==========================================
+ Coverage   73.36%   73.96%   +0.59%     
==========================================
  Files           6        6              
  Lines         199      265      +66     
==========================================
+ Hits          146      196      +50     
- Misses         53       69      +16

Files Changed	Coverage Δ
src/getters.jl	`82.38% <76.11%> (-3.98%)`	:arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

TheCedarPrince commented 10 months ago

Hey @Farreeda , I investigated further about the reason why the CodeCov bot is marking this as unsuccessful. It's because it detected a few dispatches and functions that were not part of the test suite (or so it thinks). Just to be safe, could you look through this log here: https://app.codecov.io/gh/JuliaHealth/OMOPCDMCohortCreator.jl/pull/54/blob/src/getters.jl and where it is marked with red, could you see if we are missing a test for that dispatch? For example, this segment:

Seems to show there isn't a test that is covering the DataFrame-based dispatch for the GetVisitDate function. For the functions that have this issue, could you check to see if there are tests missing and if there are, could you add them in? Right now, we'll still have to skip the GetPatientState and the GetDrugAmounts as Eunomia really doesn't have that information. Then I think the bot should be mostly satisfied. Or at least we should have a Code Coverage of higher than 69.43% reported:

JuliaHealth / OMOPCDMCohortCreator.jl

fixing issues #54

Codecov Report