Problem using NCO

[AlexKnowsIt]

I am trying to alter the BachelierNet and change its allocation-layer to the NCO-layer. I read on issue #110, that you can use the NCOalgorithm inside the Resample-layer. It seems to me, that its also possible to run it without the resample, as there is a cvxpy-layer for the allocation already implemented. However, if I try to run my custom network

class NCO_Netz(torch.nn.Module, Benchmark):
    """Combination of recurrent neural networks and NCO.
    """

    def __init__(self, n_input_channels, n_assets, n_cluster, hidden_size=32, max_weight=1, shrinkage_strategy='diagonal', p=0.3):
        self._hparams = locals().copy()
        super().__init__()
        self.norm_layer = torch.nn.InstanceNorm2d(n_input_channels, affine=True)
        self.transform_layer = RNN(n_input_channels, hidden_size=hidden_size)
        self.dropout_layer = torch.nn.Dropout(p=p)
        self.time_collapse_layer = AttentionCollapse(n_channels=hidden_size)
        self.covariance_layer = CovarianceMatrix(sqrt=True, shrinkage_strategy=shrinkage_strategy)
        self.channel_collapse_layer = AverageCollapse(collapse_dim=1)
        self.portfolio_opt_layer = NCO(n_clusters=n_cluster, n_init=10, init='random', random_state=None)
        self.gamma_sqrt = torch.nn.Parameter(torch.ones(1), requires_grad=True)
        self.alpha = torch.nn.Parameter(torch.ones(1), requires_grad=True)

    def forward(self, x):
        """Perform forward pass.
        Parameters
        ----------
        x : torch.Tensor
            Of shape (n_samples, n_channels, lookback, n_assets).
        Returns
        -------
        weights : torch.Torch
            Tensor of shape (n_samples, n_assets).
        """
        # Normalize
        x = self.norm_layer(x)

        # Covmat
        rets = x[:, 0, :, :]
        covmat = self.covariance_layer(rets)

        # expected returns
        x = self.transform_layer(x)
        x = self.dropout_layer(x)
        x = self.time_collapse_layer(x)
        exp_rets = self.channel_collapse_layer(x)

        # gamma
        gamma_sqrt_all = torch.ones(len(x)).to(device=x.device, dtype=x.dtype) * self.gamma_sqrt
        alpha_all = torch.ones(len(x)).to(device=x.device, dtype=x.dtype) * self.alpha

        # weights
        weights = self.portfolio_opt_layer(covmat, exp_rets)

        return weights

While trying to train I receive the error message

Epoch 1:   0%|          | 0/354 [04:02<?, ?it/s]

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-37-ab2bfba227c9> in <module>()
----> 1 history = run.launch(epochs)

4 frames

/usr/local/lib/python3.7/dist-packages/deepdow/experiments.py in launch(self, n_epochs)
    264 
    265                     # Forward & Backward
--> 266                     weights = self.network(X_batch)
    267                     loss_per_sample = self.loss(weights, y_batch)
    268                     loss = loss_per_sample.mean()

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051             return forward_call(*input, **kwargs)
   1052         # Do not call functions when jit is used
   1053         full_backward_hooks, non_full_backward_hooks = [], []

<ipython-input-26-17d07f2a5b84> in forward(self, x)
     45 
     46         # weights
---> 47         weights = self.portfolio_opt_layer(covmat, exp_rets)
     48 
     49         return weights

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051             return forward_call(*input, **kwargs)
   1052         # Do not call functions when jit is used
   1053         full_backward_hooks, non_full_backward_hooks = [], []

/usr/local/lib/python3.7/dist-packages/deepdow/layers/allocate.py in forward(self, covmat, rets)
    146 
    147             for c in range(self.n_clusters):
--> 148                 in_cluster = torch.where(cluster_ixs == c)[0]  # indices from the same cluster
    149                 intra_covmat = covmat[[i]].index_select(1, in_cluster).index_select(2, in_cluster)  # (1, ?, ?)
    150                 intra_rets = None if rets is None else rets[[i]].index_select(1, in_cluster)  # (1, ?)

TypeError: where(): argument 'condition' (position 1) must be Tensor, not bool

Probably I was just missusing a parameter but I can't figure out which one. What is the torch.where() expecting? The only thing that I changed from the BachelierNet is the allocation-layer and the squaring of the covmat.

[jankrepl]

Thank you for the question:))

Could you please share a reproducible code example that gives you this error?

[AlexKnowsIt]

Yes of course :)

https://gist.github.com/AlexKnowsIt/47fb35f8d9ad10ccf957bd583ce846c3

[jankrepl]

Perfect! Thank you for sharing.

It seems like some of your assets have constant returns. More specifically, if you put a breakpoint inside of the forward of your NCO_Netz and inspect rets[0, :, 0] you will get

tensor([-0.1323, -0.1323, -0.1323, -0.1323, -0.1323, -0.1323, -0.1323, -0.1323,
        -0.1323, -0.1323, -0.1323, -0.1323, -0.1323, -0.1323, -0.1323, -0.1323,
        -0.1323, -0.1323, -0.1323, -0.1323], grad_fn=<SelectBackward>)

The covmat[0, 0] will then look like this

tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0.], grad_fn=<SelectBackward>)

Similarly, The covmat[0, 28] will then look like this

tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0.], grad_fn=<SelectBackward>)

This becomes a problem here https://github.com/jankrepl/deepdow/blob/eb6c85845c45f89e0743b8e8c29ddb69cb78da4f/deepdow/layers/allocate.py#L138 since there will be a bunch of 0/0 and the corrmat will contain nans.

Anyway, I only checked it really briefly, so I hope that it is really the root cause. However, I might be mistaken. Anyway, the error message is not really helpful in this case I have to admit:)

Hope that helps!

[jankrepl]

Closing due to inactivity