search

datactive / bigbang

Scientific analysis of collaborative communities
http://datactive.github.io/bigbang/
MIT License
149 stars 52 forks source link

refactoring load_data and open_list_archives o archive.py #500

Closed sbenthall closed 2 years ago

sbenthall commented 2 years ago

Addresses #435

I haven't worked through the notebooks yet to see if they are effected by this change.

I couldn't remember if the new Codecov runs the notebooks or not...

sbenthall commented 2 years ago

I'm getting lots of errors in the notebooks.

I don't think we've added automated testing of the notebooks into our group workflow yet. But one way to do this is:

pip install nbval
pytest --nbval-lax examples/

I'm getting tons of errors and need to figure out why.

sbenthall commented 2 years ago

Ok, the notebooks were not passing locally for many unrelated reasons.

Automated tests are now passing here. I request a review.

sbenthall commented 2 years ago

Great feedback, @npdoty . Thank you. I will try to address as many of these points as I can in the next round of edits.

codecov-commenter commented 2 years ago

Codecov Report

Merging #500 (fb72448) into main (0b4794a) will increase coverage by 0.26%. The diff coverage is 75.16%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #500      +/-   ##
==========================================
+ Coverage   74.82%   75.09%   +0.26%     
==========================================
  Files          21       22       +1     
  Lines        3019     3047      +28     
==========================================
+ Hits         2259     2288      +29     
+ Misses        760      759       -1
Flag Coverage Δ
unittests 75.09% <75.16%> (+0.26%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
bigbang/mailman.py 61.42% <20.00%> (-1.22%) :arrow_down:
bigbang/parse.py 48.24% <28.12%> (-7.86%) :arrow_down:
bigbang/archive.py 71.01% <91.07%> (+7.14%) :arrow_up:
tests/unit/test_archive.py 95.74% <95.74%> (ø)
bigbang/w3crawl.py 91.20% <100.00%> (ø)
tests/unit/test_mailman.py 97.70% <100.00%> (+0.53%) :arrow_up:
bigbang/listserv.py 79.60% <0.00%> (ø)
bigbang/analysis/listserv.py 64.35% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 0b4794a...fb72448. Read the comment docs.

sbenthall commented 2 years ago

@npdoty I've address those issues which I could easily cover in the scope of the original #435.

It seems like this is bringing up many issues of broader scope concerning BigBang's architecture. I'd recommend not letting these broader issues block this PR, which is at least an incremental improvement.