When used in a dataset, the document gets serialized to json like structure which does not know tuples, so they get converted to lists. This is a workaround to automatically convert the tokens field back to tuples when the document is created from a dataset.
When used in a dataset, the document gets serialized to json like structure which does not know tuples, so they get converted to lists. This is a workaround to automatically convert the
tokensfield back to tuples when the document is created from a dataset.