Only `data_loader.data_loaders` is plural

[itsnamgyu]

I had a really minor question about the plural naming of the data_loader/data_loaders.py module.

Is there a specific reason that only this module is plural?

Thanks

[deeperlearner]

I think the proper way to utilize this template is putting many files in folder like the following structure.

├── configs/
│   ├── config1.json
│   └── config2.json
│
├── data_loaders/
│   ├── data_loader1.py
│   └── data_loader2.py
│
├── models/
│   ├── model1.py
│   ├── model2.py
│   ├── metric.py
│   └── loss.py
│
├── trainers/
│   ├── trainer1.py
│   └── trainer2.py
│  
└── utils/ - small utility functions
    ├── util.py
    └── *.py

And use importlib to handle module import. You can check my Pytorch-Template to see more detail.

[itsnamgyu]

Thanks for the introduction to the importlib library. However, based on the way that the config system selects modules based on string configs (e.g., along the lines of getattr(models.model, config.model_name))**, I'm not sure if the example given aligns with how the template works.

** Simplification of the internal logic in parse_config.py

[deeperlearner]

part of example config:

    "models": {
        "Net_G": {
            "module": ".model1",
            "type": "Generator"
        },
        "Net_D": {
            "module": ".model2",
            "type": "Discriminator"
        },
        "Net_C": {
            "module": ".model3",
            "type": "Classifier"
        }
    },

revise init_obj in parse_config.py:

    def init_obj(self, keys, module, *args, **kwargs):
        """
        Returns an object or a function, which is specified in config[keys[0]]...[keys[-1]].
        In config[keys[0]]...[keys[-1]],
            'is_ftn': If True, return a function. If False, return an object.
            'module': The module of each instance.
            'type': Class name.
            'kwargs': Keyword arguments for the class initialization.
        keys is the list of config entries.
        module is the package module.
        Additional *args and **kwargs would be forwarded to obj()
        Usage: `objects = config.init_obj(['A', 'B', 'C'], module, a, b=1)`
        """
        obj_config = get_by_path(self, keys)
        try:
            module_name = obj_config['module']
            module_obj = importlib.import_module(module_name, package=module)
        except KeyError:  # In case no 'module' is specified
            module_obj = module
        class_name = obj_config['type']
        obj = getattr(module_obj, class_name)
        kwargs_obj = self._update_kwargs(obj_config, kwargs)

        if obj_config.get('is_ftn', False):
            return partial(obj, *args, **kwargs_obj)
        return obj(*args, **kwargs_obj)

more detail in my repo Pytorch-Template

[deeperlearner]

If you don't use importlib, you may need to import many modules manually. I found it's very annoying so I revise it to let importlib handle them.

[itsnamgyu]

Oh I see! Thanks for the tip :)