Review matching and de-duplication logic

[mejackreed]

From @peetucket on February 16, 2016 22:20

There are a number of places where we don't seem to de-duping correctly. Related to #21

• [ ] Re-map out what the matching logic is
• [ ] Fix issues with logic
• [ ] Ensure dedupe logic ignores final "." in title (in looking at PubMed and WoS records that should have been deduped, I am noticing that the PubMed title has a period and the WoS record doesn't. should become unnecessary if #83 is done first, see https://jirasul.stanford.edu/jira/browse/CAP-68 for other info)
• [ ] see https://jirasul.stanford.edu/jira/browse/CAP-61 for additional info, and some related issues than may be corrected in a separate branch

Copied from original issue: sul-dlss/sul-pub#85

[mejackreed]

From @darrenleeweber on May 11, 2016 1:23

I've found a spec that will hit some de-dup code, it's in

• https://github.com/sul-dlss/sul-pub/blob/master/spec/lib/science_wire_harvester_spec.rb#L326-L335

It can exercise some of the code at

• https://github.com/sul-dlss/sul-pub/blob/master/lib/science_wire_harvester.rb#L259-L280