Cannot pickle generator object

[riwish]

Summary of bug

The explain_prediction method behaves differently for tokens with a length greater than size 8.

The error message is: TypeError: cannot pickle 'generator' object

The error is found in: text_eplainability > generation > return_types.py at lline 195 self._neighborhood_instances = copy.deepcopy(self._provider.get_children(self._original_instance))

Environment information

• Platform: OS independent
• Python version: 3.9
• explabox version: 0.9b4

Reproducing the bug

Steps to reproduce the behavior: This bug happens to appear for multiple cases. The easiest way to reproduce is to follow the demo-drug dataset steps and use a sentence with 8 or more words in the explain section.

Solutions Attempted

The error seems to be gone by removing the copy: self._neighborhood_instances = self._provider.get_children(self._original_instance)

Expected behavior

Output widget

[MarcelRobeer]

My suggestion would be to prevent direct usage of copy.deepcopy(...) throughout the explabox, but rather make a new feature enhancement in genbase that allows for a safe_deepcopy(...) function that checks if objects can be pickled, and if not transforms them into a format that allows pickling.

[MarcelRobeer]

Solution presented above was not needed, because it was not the overall sequence that was a generator but one subpart thereof. This subpart (the new untokenized string) should have been a sequence anyway, so we fixed that in text_explainability==0.6.4.