telekom / nlu-bridge

MIT License
4 stars 5 forks source link

Improve interface for loading data from vendor formats #16

Closed kengelbrecht closed 2 years ago

kengelbrecht commented 2 years ago

Currently, we include a function load_data() in a vendor module that allows loading data from the vendor format as NLUdataset.

Motivation for this design:

Problems:

Possible solutions:

Sugested solution:

nlubridge
    datasets.py
    vendors
        rasa.py
        watson.py
        ...
    dataloaders
        rasa.py
        huggingface.py
        ...

Import data loaders directly from nlubridge. Function names are of the form from_rasa() and os on.

Loading data is then equivalent to how it is done in Pandas. Dataset formats that do not correspond to an intent recognition vendor (like huggingface) then don't cause confusion within the vendors subpackage.

kengelbrecht commented 2 years ago

Implemented now in branch https://github.com/telekom/nlu-bridge/tree/feature/fix_lazy_imports