Pale-Blue-Dot-97 / Minerva

Minerva project includes the minerva package that aids in the fitting and testing of neural network models. Includes pre and post-processing of land cover data. Designed for use with torchgeo datasets.
MIT License
20 stars 1 forks source link

Make some large third-party libraries optional #161

Closed Pale-Blue-Dot-97 closed 1 year ago

Pale-Blue-Dot-97 commented 1 year ago

Issue

The size of the stack required to meet minerva's requirements are now so large that it is causing some issues. Some of the dependencies are barely used but included to add compatibility mainly for 3rd-party ML frameworks such as onnx and wandb.

Solution

Use importlib to try and import optional dependencies at runtime for specific and appropriate features, such as this example:

def _optional_import(module: str, name: str = None, package: str = None):
    import importlib
    try:
        module = importlib.import_module(module)
        return module if name is None else getattr(module, name)
    except ImportError as e:
        if package is None:
            package = module
        msg = f"install the '{package}' package to make use of this feature"
        raise ValueError(msg) from e

https://stackoverflow.com/questions/563022/whats-python-good-practice-for-importing-and-offering-optional-features

Pale-Blue-Dot-97 commented 1 year ago

Added this optional_import in b32dd44e91ef2154bee6922d9b1523634294e91f

Pale-Blue-Dot-97 commented 1 year ago

Removed onnx2torch (and thereby onnx) dependency in 67ecd460f9857ff3174d2ae6a25b0aaac8218aa7

Pale-Blue-Dot-97 commented 1 year ago

Removed the only explicit tensorflow usage in making confusion matrices in 44e921bdd8b46a836dd930b1d2b8027ccde854ab

Pale-Blue-Dot-97 commented 1 year ago

tensorflow is now only required for tensorboard usage via torch. Made it optional and working by 9464a885eacac51d5b02afa85adc5b52fb2d276e