dedupeio / dedupe-examples

:id: Examples for using the dedupe library
MIT License
406 stars 214 forks source link

run csv_example.py failure #115

Closed listen0428 closed 2 years ago

listen0428 commented 4 years ago

Hi all,

when I run csv_example.py, there is an error, what should I do? 32/10 positive, 5/10 negative Do these records refer to the same thing? (y)es / (n)o / (u)nsure / (f)inished / (p)revious f Finished labeling clustering... Traceback (most recent call last):n File "csv_example.py", line 149, in clustered_dupes = deduper.partition(data_d, 0.5) File "/proj/xsjhdstaff2/shengl/usr/local/anaconda3/lib/python3.7/site-packages/dedupe/api.py", line 170, in partition pair_scores = self.score(pairs) File "/proj/xsjhdstaff2/shengl/usr/local/anaconda3/lib/python3.7/site-packages/dedupe/api.py", line 106, in score self.num_cores) File "/proj/xsjhdstaff2/shengl/usr/local/anaconda3/lib/python3.7/site-packages/dedupe/core.py", line 252, in scoreDuplicates fillQueue(record_pairs_queue, record_pairs, n_map_processes) File "/proj/xsjhdstaff2/shengl/usr/local/anaconda3/lib/python3.7/site-packages/dedupe/core.py", line 283, in fillQueue chunk = tuple(itertools.islice(iterable, chunk_size)) File "/proj/xsjhdstaff2/shengl/usr/local/anaconda3/lib/python3.7/site-packages/dedupe/api.py", line 254, in pairs con.close() File "/proj/xsjhdstaff2/shengl/usr/local/anaconda3/lib/python3.7/tempfile.py", line 805, in exit self.cleanup() File "/proj/xsjhdstaff2/shengl/usr/local/anaconda3/lib/python3.7/tempfile.py", line 809, in cleanup _shutil.rmtree(self.name) File "/proj/xsjhdstaff2/shengl/usr/local/anaconda3/lib/python3.7/shutil.py", line 494, in rmtree _rmtree_safe_fd(fd, path, onerror) File "/proj/xsjhdstaff2/shengl/usr/local/anaconda3/lib/python3.7/shutil.py", line 452, in _rmtree_safe_fd onerror(os.unlink, fullname, sys.exc_info()) File "/proj/xsjhdstaff2/shengl/usr/local/anaconda3/lib/python3.7/shutil.py", line 450, in _rmtree_safe_fd os.unlink(entry.name, dir_fd=topfd) OSError: [Errno 16] Device or resource busy: '.nfs0000000002d81be2000026cb'

fgregg commented 2 years ago

i don't konw. looks like it's having trouble cleaning up the a tempfile.