Bugs fixes, adjustments, and cleaning.

[bunsenmurder]

Description

As the title implies, this pull request encompasses a few bug fixes, a few adjustments and some code cleaning. Major changes I've made will be described here; any minor details or specifics can be found within the commit messages.

Fixes

• For all providers except Glassdoor Dynamic, I changed how keywords and cities were encoded into the queries being sent to the job providers. This fixes issue #80.
• Previously, futures within the delay_threader function were being deleted whether a blurb was parsed or not.

Adjustments

• Previously, we would assign headers to each get/post request. Now the headers are set as the default headers for each providers session object (reference). If different headers are needed for a specific request, they can be overridden by setting the headers when making that request.
• For Glassdoor_static, we only need to make a POST request initially when querying the Glassdoor location api and making an initial search. I've replaced any unnecessary POST requests with GET requests.
• Any errors in the delay_threader are captured by the logger.

Context of change

Please add options that are relevant and mark any boxes that apply.

• [ ] Software (software that runs on the PC)
• [ ] Library (library that runs on the PC)
• [ ] Tool (tool that assists coding development)
• [x] Other

Type of change

Please mark any boxes that apply.

• [x] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [ ] This change requires a documentation update

How Has This Been Tested?

• Ran the funnel using quoted searches and comparing to actual results on each providers website.

Checklist:

Please mark any boxes that have been completed.

• [x] I have performed a self-review of my own code.
• [x] I have commented my code, particularly in hard-to-understand areas.
• [ ] I have made corresponding changes to the documentation.
• [x] My changes generate no new warnings.
• [ ] I have added tests that prove my fix is effective or that my feature works.
• [ ] New and existing unit tests pass locally with my changes.
• [ ] Any dependent changes have been merged and published in downstream modules.

[codecov-commenter]

Codecov Report

Merging #84 into master will decrease coverage by 0.02%. The diff coverage is 60.00%.

@@            Coverage Diff             @@
##           master      #84      +/-   ##
==========================================
- Coverage   58.37%   58.34%   -0.03%     
==========================================
  Files          13       13              
  Lines        1146     1150       +4     
==========================================
+ Hits          669      671       +2     
- Misses        477      479       +2     

Impacted Files	Coverage Δ
jobfunnel/glassdoor_dynamic.py	16.37% <0.00%> (-0.15%)	:arrow_down:
jobfunnel/monster.py	28.14% <33.33%> (+0.53%)	:arrow_up:
jobfunnel/jobfunnel.py	47.70% <50.00%> (+0.02%)	:arrow_up:
jobfunnel/glassdoor_static.py	28.57% <60.00%> (+0.68%)	:arrow_up:
jobfunnel/glassdoor_base.py	57.89% <66.66%> (-0.73%)	:arrow_down:
jobfunnel/indeed.py	87.32% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 0a246cb...7f27900. Read the comment docs.

[bunsenmurder]

Thanks @PaulMcInnis ! Btw, the codecov bot doesn't seem to like some my changes. Is there anything specific I need to add when making pull requests?

[PaulMcInnis]

Its ok, its just unhappy about reduction in coverage - should be possible to widen the threshold so it doesn't fail for so little a reduction.

On Sat, Jul 11, 2020 at 3:16 PM Erich M notifications@github.com wrote:

Thanks @PaulMcInnis https://github.com/PaulMcInnis ! Btw, the codecov bot doesn't seem to like some my changes. Is there anything specific I need to add when making pull requests?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/PaulMcInnis/JobFunnel/pull/84#issuecomment-657114743, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAYKY2LFNYOLLN42VZJZXRDR3C3A3ANCNFSM4OWZRRPQ .

--

• Paul McInnis