search

segment-any-text / wtpsplit

Toolkit to segment text into sentences or other semantic units in a robust, efficient and adaptable way.
MIT License
624 stars 36 forks source link

Recursion in init #96

Closed jonvaughan closed 7 months ago

jonvaughan commented 1 year ago

Hi - hopefully this is just something I am constructing incorrectly but I am getting recursion in init which results in an error with wtpsplit==1.2.0.

My code is running inside joblib but is just doing:

self.sentence_splitter = WtP("wtp-canine-s-12l") for sentence in self.sentence_splitter.split(text, lang_code=self.language): yield sentence

And I get:

process_worker call_item = call_queue.get(block=True, timeout=timeout) File "/usr/lib/python3.8/multiprocessing/queues.py", line 116, in get return _ForkingPickler.loads(res) File "/usr/local/lib/python3.8/dist-packages/wtpsplit/init.py", line 115, in getattr return getattr(self.model, name) File "/usr/local/lib/python3.8/dist-packages/wtpsplit/init.py", line 115, in getattr return getattr(self.model, name) File "/usr/local/lib/python3.8/dist-packages/wtpsplit/init.py", line 115, in getattr return getattr(self.model, name) [Previous line repeated 988 more times] RecursionError: maximum recursion depth exceeded

Let me know if any thoughts

Thanks

Jon

bminixhofer commented 1 year ago

Can you try this again with 1.2.1? It might be related to #95. If it still does not work, please give me a self-contained code snippet so I can try reproducing it.

bminixhofer commented 7 months ago

Closing this as stale. Please reopen if this pops up again.