Skallwar / suckit

Suck the InTernet
Apache License 2.0
748 stars 39 forks source link

Include fragments on html files #207

Closed AOx0 closed 1 year ago

AOx0 commented 1 year ago

My use case requires suckit to maintain fragments within paths, for example, href="#sub-title" becomes href="index.html#sub-title". I re-enabled the fragment part only for the downloaded Html files. The visited hash map only records the Url's path without storing the fragment.

One downside of the PR implementation is the introduction of an additional argument.

pub fn to_path(url: &Url, with_fragment: bool) -> String { ... }

I'm making this PR in case anybody needs the feature. From my point of view, it is essential.

Have a nice day. This is related to #99 and #100

codecov[bot] commented 1 year ago

Codecov Report

Merging #207 (c92f96c) into master (0c8164c) will increase coverage by 0.29%. The diff coverage is 66.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #207      +/-   ##
==========================================
+ Coverage   30.81%   31.11%   +0.29%     
==========================================
  Files          18       18              
  Lines        3274     3278       +4     
==========================================
+ Hits         1009     1020      +11     
+ Misses       2265     2258       -7     
Impacted Files Coverage Δ
src/scraper.rs 24.48% <0.00%> (-0.26%) :arrow_down:
src/url_helper.rs 100.00% <100.00%> (ø)
src/lib.rs 20.84% <0.00%> (+0.28%) :arrow_up: