search

nicolay-r / AREkit

Document level Attitude and Relation Extraction toolkit (AREkit) for sampling and processing large text collections with ML and for ML
https://nicolay-r.github.io/arekit-page/
MIT License
58 stars 3 forks source link

SQlite rows reader generalization -- use `**kwargs` #553

Open nicolay-r opened 3 months ago

nicolay-r commented 3 months ago

550 related minor feature

It is possible to simplify SQLiteReader implementation as follows:

class CustomSQliteReader(BaseReader):

    def __init__(self, storage_type, **storage_kwargs):
        self._storage_kwargs = storage_kwargs
        self._storage_type = storage_type

    def extension(self):
        return ".sqlite"

    def read(self, target):
        return self._storage_type(path=target, **self._storage_kwargs)

Which allow us to provide any customized storage. For example, in the case of ARElight we may support iteration over joined tables. See https://github.com/nicolay-r/ARElight/issues/144 as the related request for this feature.