Currently, we include a function load_data() in a vendor module that allows loading data from the vendor format as NLUdataset.
Motivation for this design:
keep this code with the vendor implementation because it is often closely related
unify access by always using the same name for the function instead of function names like load_from_rasa
Problems:
Having the functionality of the function defined by its module path is awkward when using it in scripts
Still requires to know the module names which are "hidden" for vendor classes as vendors are imported directly from nlubridge.vendors
Ensuring loading the function does not fail because of missing dependencies for the vendor which are not required for load_data() comes with tradeoffs
Loading from other formats like huggingface does not have a proper place in the project
Possible solutions:
keep the load_data() functions with vendor-specific names like load_rasa_data in separate modules which require only the required dependencies. functions could be loaded from package level like e.g. from nlubridge.loaders import load_rasa_data.
Same as above but access data loaders as NLUdataset methods (NLUdataset.from_rasa())
Same as above but access dialogs from nlubridge directly.
Keep load_data in the vendor module but rewrite so that missing dependencies are ignored when importing load_data(). This has the drawback that the vendor can be imported but on creation will fail due to missing dependencies, which I think can be a bit confusing.
Same as above but enable access of loader in the form of from nlubridge.vendors import load_rasa_data.
Import data loaders directly from nlubridge. Function names are of the form from_rasa() and os on.
Loading data is then equivalent to how it is done in Pandas. Dataset formats that do not correspond to an intent recognition vendor (like huggingface) then don't cause confusion within the vendors subpackage.
Currently, we include a function load_data() in a vendor module that allows loading data from the vendor format as NLUdataset.
Motivation for this design:
load_from_rasa
Problems:
Possible solutions:
load_rasa_data
in separate modules which require only the required dependencies. functions could be loaded from package level like e.g.from nlubridge.loaders import load_rasa_data
.NLUdataset.from_rasa()
)from nlubridge.vendors import load_rasa_data
.Sugested solution:
Import data loaders directly from nlubridge. Function names are of the form
from_rasa()
and os on.Loading data is then equivalent to how it is done in Pandas. Dataset formats that do not correspond to an intent recognition vendor (like huggingface) then don't cause confusion within the vendors subpackage.