search

DeNederlandscheBank / name_matching

Other
128 stars 43 forks source link

Performance Improvement #11

Open versatile712 opened 1 year ago

versatile712 commented 1 year ago

Hi, glad to see a wonderful package for matching firms! I am now matching 400w firms to a list of 3w firms, but the speed is slow since it uses only one cpu core on my server. Is there any convenient workaround to run it parallelly? :)

mnijhuis-dnb commented 1 year ago

You can run it parallelly, I added a notebook with an example of how you can do it using the python implementation of ray. In a later version I will see how the package can support multiprocessing out of the box