lemon24 / reader

A Python feed reader library.
https://reader.readthedocs.io
BSD 3-Clause "New" or "Revised" License
455 stars 38 forks source link

Fix #281 Single entry without id and link fails entire feed #282

Closed mirekdlugosz closed 2 years ago

mirekdlugosz commented 2 years ago

This fixes #281.

ParseError in _feedparser_entry is caught and re-emitted as warning. This way we salvage as much as we can from feeds that contain valid and invalid entries.

_process_feedparser_dict re-raises ParseError with slightly different message if all entries are invalid. This is to maintain interface compatibility with JSON Feed parser.

Reproducer:

Download planetpython-buggy.xml.txt and run:

$ python -m reader --db db.sqlite --feed-root /tmp/ add --update 'file:planetpython-buggy.xml.txt' -vvvv

On master, this will raise exception; with this patch, it will emit warning UserWarning: file:planetpython-buggy.xml.txt: entry with no id or link fallback, but otherwise will work.

$ ./run.sh coverage-all
<snip>
TOTAL                                              10326    423   2963     90    95%

======================================== 1703 passed, 3 skipped, 3 xfailed, 18 warnings, 70 subtests passed in 30.91s =========================================
Wrote HTML report to htmlcov/index.html
Name    Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------
---------------------------------------------------
TOTAL    3396      0   1219      0   100%

22 files skipped due to complete coverage.
$ ./run.sh typing
Success: no issues found in 74 source files
codecov[bot] commented 2 years ago

Codecov Report

Merging #282 (f9a3f94) into master (1d60303) will increase coverage by 0.04%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #282      +/-   ##
==========================================
+ Coverage   95.17%   95.22%   +0.04%     
==========================================
  Files          75       75              
  Lines       10311    10394      +83     
  Branches     1475     1486      +11     
==========================================
+ Hits         9814     9898      +84     
+ Misses        410      409       -1     
  Partials       87       87              
Impacted Files Coverage Δ
src/reader/_parser.py 100.00% <100.00%> (ø)
tests/test_parser.py 99.43% <100.00%> (+<0.01%) :arrow_up:
tests/test_plugins_twitter.py 98.76% <0.00%> (-0.68%) :arrow_down:
src/reader/_plugins/twitter.py 85.55% <0.00%> (+1.22%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 1d60303...f9a3f94. Read the comment docs.

lemon24 commented 2 years ago

Thank you for opening the issue and submitting a fix, appreciate it!

lemon24 commented 2 years ago

@mirekdlugosz: reader 2.13 is now available on PyPI, including this PR.

mirekdlugosz commented 2 years ago

Thank you!