Auto adjust number of layers for GWNET

[SonghuaHu-UMD]

When testing GWNET, one issue was found when I increased the length of input windows.

Default settings are applicable only when the input_window is shorter than 12. People should adjust the number of blocks, number of layers, or kernel size, in order to ensure the final receptive field >= input_window. If not, the model performance would be low.

For example, when my input window is 24, the loss of the model under the default setting (blue curve) is much higher than the model with increased number of layers (orange curve). Since the default setting generates a receptive field = 13, which is greater than 12 but smaller than 24.

I added some codes to automatically adjust the number of layers based on the length of input windows. The idea is from equation 11 in the authors' other paper "Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks":

receptive_field = (1 + (kernel_size - 1) blocks (2 ** layers - 1)) >= input_window

Some testing code:

import numpy as np

blocks = 4
kernel_size = 2
layers = 3
receptive_field = 1
input_window = 12
for b in range(blocks):
    additional_scope = kernel_size - 1
    new_dilation = 1
    for i in range(layers):
        # dilated convolutions
        new_dilation *= 2
        receptive_field += additional_scope
        additional_scope *= 2
print(receptive_field == (1 + (kernel_size - 1) * blocks * (2 ** layers - 1)))

blocks = 4
for input_window in [6, 12, 24, 48]:
    n_layers = np.int(np.round(np.log((((input_window - 1) / (blocks * (kernel_size - 1))) + 1)) / np.log(2)))
    print('input_window: %s, blocks: %s, kernel_size: %s, --> n_layers: %s' % (
        input_window, blocks, kernel_size, n_layers))

Which outputs:

input_window: 6, blocks: 4, kernel_size: 2, --> n_layers: 1
input_window: 12, blocks: 4, kernel_size: 2, --> n_layers: 2
input_window: 24, blocks: 4, kernel_size: 2, --> n_layers: 3
input_window: 48, blocks: 4, kernel_size: 2, --> n_layers: 4

As shown, the number of layers can be auto-adjusted based on the length of input windows.