Update feature extractor

[ganow]

We have implemented the followings:

• type annotations and docstrings for dl/torch/torch.py
• test code for FeatureExtractor()
• Stop using eval() function in FeatureExtractor()

Note:

• We have changed layers argument from an optional keyword argument to a positional argument. The reason for the interface change is based on this.
• We have not implemented the test code for ImageDataset dataset yet.

This PR will have the breaking change in bdpy's API. FeatureExtractor() now only accepts the layer name which has the following formats:

• encoder.<layer> can be directly accessible by getattr (e.g., conv1): layer_object = getattr(encoder, layer)
• layer is the combination of the instance variable name and index (e.g., features[3]): layer_object = getattr(encoder, layer_name)[index]
• layer is hierarchically composed of multiple objects (e.g., resblock.conv1): layer_object = getattr(getattr(encoder, parent_name), child_name)
• any combinations of the above situations (e.g., transformer.resblocks[3].attn)

[ganow]

@ShuntaroAoki Question: Why FeatureExtractor() accepts layer=None?

https://github.com/KamitaniLab/bdpy/blob/bdc866716aab675b311846528a4a275bb075d0fe/bdpy/dl/torch/torch.py#L11-L14

If we set it as self.__layer = None, the following code will raise an error.

https://github.com/KamitaniLab/bdpy/blob/bdc866716aab675b311846528a4a275bb075d0fe/bdpy/dl/torch/torch.py#L23-L26

[ganow]

I found the useful method nn.Module.get_submodule(target: str) -> nn.Module. We can use it like:

class MockModule(nn.Module):
    def __init__(self):
        super(MockModule, self).__init__()
        self.layer1 = nn.Linear(10, 10)
        self.layers = nn.Sequential(
            nn.Conv2d(1, 1, 3),
            nn.Conv2d(1, 1, 3),
            nn.Module(),
            nn.Sequential(
                nn.Conv2d(1, 1, 4),
                nn.Conv2d(1, 1, 8),
            )
        )
        inner_network = self.layers[-2]
        inner_network.features = nn.Sequential(
            nn.Conv2d(1, 1, 5),
            nn.Conv2d(1, 1, 5)
        )

mock = MockModule()
mock.get_submodule('layer1')  # -> Linear(in_features=10, out_features=10, bias=True)
mock.get_submodule('layers.0')  # -> Conv2d(1, 1, kernel_size=(3, 3), stride=(1, 1))
mock.get_submodule('layers.2.features.0')  # -> Conv2d(1, 1, kernel_size=(5, 5), stride=(1, 1))
mock.get_submodule('layers.3.0')  # -> Conv2d(1, 1, kernel_size=(4, 4), stride=(1, 1))

Perhaps we should implement _parse_layer_name() by using it.

Pros of using get_submodule():

• While the current implementation only accepts pre-assumed finite formats, get_submodule() can handle arbitrary syntax (at least ideally)
• Type safe
• No need to take care of functions that perform parsing on their own.

Cons:

• Since get_submodule() only works for nn.Module, it will not accept access to instance variables for other general Python objects. (We originally did not assume at the type level that anything other than nn.Module would be input though.)

Reference: https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.get_submodule