search

HEPCloud / decisionengine

HEPCloud Decision Engine framework
Apache License 2.0
6 stars 25 forks source link

Print number of pickled bytes of source-produced data #629

Closed knoepfel closed 2 years ago

knoepfel commented 2 years ago

This is to aid debugging.

codecov[bot] commented 2 years ago

Codecov Report

Merging #629 (62ad8f8) into master (97aed84) will decrease coverage by 0.10%. The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #629      +/-   ##
==========================================
- Coverage   97.35%   97.25%   -0.11%     
==========================================
  Files          47       47              
  Lines        2838     2839       +1     
  Branches      449      449              
==========================================
- Hits         2763     2761       -2     
- Misses         50       52       +2     
- Partials       25       26       +1
Flag Coverage Δ
python-3.10 96.97% <100.00%> (+<0.01%) :arrow_up:
python-3.6 96.81% <100.00%> (-0.11%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...c/decisionengine/framework/engine/SourceWorkers.py 100.00% <100.00%> (ø)
.../decisionengine/framework/engine/ChannelWorkers.py 88.15% <0.00%> (-2.64%) :arrow_down:
.../decisionengine/framework/engine/DecisionEngine.py 91.82% <0.00%> (-0.20%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 97aed84...62ad8f8. Read the comment docs.

jcpunk commented 2 years ago

I'm wondering if we should look at passing the data as json rather than a pickle if this is "too large" (pandas has a to_json/from_json if I recall)