Down sample perf - Githubissues

anhaidgroup / py_entitymatching

BSD 3-Clause "New" or "Revised" License

185 stars 48 forks source link

Down sample perf #16

Closed Yashg19 closed 8 years ago

Yashg19 commented 8 years ago

Created a separate program for down_sampling
Updated init.py to add down sampling function
Optimized down sampling code by changing _inv_index and _probe_index code
Updated down sampling code to reflect Magellan manual with respect to parameters
Added unit test cases for down sampling
Added benchmark test cases for down sampling

@sanjibkd, @paulgc - please do my code review

sanjibkd commented 8 years ago

@Yashg19 You might want to add try-except block in your benchmark test cases for the example data sets because none of them will be available with the package by default

sanjibkd commented 8 years ago

@Yashg19 Also try to pylint your code and try to achieve a score of at least 8

Yashg19 commented 8 years ago

Added comments and changes as recommended by Pradap.

Yashg19 commented 8 years ago

@sanjibkd @paulgc @kvpradap - please review if I have implemented all review comments given by you all :)

Yashg19 commented 8 years ago

@paulgc @sanjibkd - all review comments are incorporated. I have tested the code but wasn't able to run nose test again to re-verify the changes. I have tested the code separately with different input parameters. Thank you so much for your valuable comments.

While running nose test or asv run - I was getting the error:

import magellan as mg --> from magellan.blocker.attr_equiv_blocker import AttrEquivalenceBlocker -->-->from joblib import Parallel, delayed

ImportError: No module named joblib

@kvpradap - copying you as the changes are ready to pull if there are no more review comments.