Temporal regularization experiment with solar energy data

[deploy-soon]

Some experiment results which contain validation loss among several loss functions. Solar energy data is from Multivariate Time series Data sets.

Solar Energy The raw data is in http://www.nrel.gov/grid/solar-power-data.html : It contains the solar power production records in the year of 2006, which is sampled every 10 minutes from 137 PV plants in Alabama State.

shape: (52560, 137)

[deploy-soon]

Base Matrix Factorization

• code: 10d386
• loss function
• Adam optimizer(learning rate=0.0005)
• train ratio: 0.8
• metric:
• result

Factors	20	30	40	50
NRMSE	0.3628	0.3660	0.3709	0.3722

[deploy-soon]

L2 Reg Matrix Factorization

• code: 10d386
• loss function
• Adam optimizer(learning rate=0.005)
• train ratio: 0.8
• metric:
• result - validation loss

Factors	20	30	40	50
lambda=5.0	0.9730	0.9754	0.9762	0.9810
lambda=0.5	0.4304	0.4310	0.4295	0.4291
lambda=0.05	0.3570	0.3504	0.3498	0.3496
lambda=0.005	0.3588	0.3631	0.3663	0.3634

[deploy-soon]

Temporal L2 Reg Matrix Factorization

• code: 10d386
• loss function
• Adam optimizer(learning rate=0.0005)
• train ratio: 0.8
• metric: result
• temporal lambda = lambda / 10.0

Factors	20	30	40	50
lambda=5.0	0.7732	0.7709	0.7741	0.7751
lambda=0.5	0.3979	0.3970	0.3976	0.3975
lambda=0.05	0.3564	0.3488	0.3432	0.3421
lambda=0.005	0.3589	0.3668	0.3672	0.3629

• temporal lambda = lambda / 1.0

Factors	20	30	40	50
lambda=5.0	0.7663	0.7676	0.7674	0.7635
lambda=0.5	0.3910	0.3901	0.391	0.3926
lambda=0.05	0.3336	0.3096	0.2971	0.2943
lambda=0.005	0.356	0.3626	0.3401	0.3338

• temporal lambda = lambda / 0.1

Factors	20	30	40	50
lambda=5.0	0.7470	0.7434	0.7417	0.7432
lambda=0.5	0.3787	0.3806	0.3787	0.3795
lambda=0.05	0.2979	0.2781	0.2693	0.2644
lambda=0.005	0.3269	0.2946	0.2776	0.2707

• temporal lambda = lambda / 0.01

Factors	20	30	40	50
lambda=5.0	0.7403	0.7376	0.7391	0.7393
lambda=0.5	0.3764	0.3773	0.3752	0.3736
lambda=0.05	0.2892	0.277	0.2708	0.2679
lambda=0.005	0.2920	0.2693	0.2607	0.2573

[deploy-soon]

• code: 0ea4c5
• train: 0.7, validate: 0.15, test: 0.15

NRMSE	lambda=0.1 factors=30	lambda=0.1 factors=40	lambda=1.0 factors=30	lambda=1.0 factors=40
Vector Embedding	0.2988	0.2881	0.2908	0.2858
Matrix Embedding	0.2965	0.2865	0.2873	0.2840
Tensor Embedding	0.3140	0.3135	0.2926	0.2950
RNN	0.3367	0.3457	0.3283	0.3219
LSTM	0.3362	0.3278	0.3082	0.3026
GRU	0.3307	0.3185	0.3044	0.2994

[deploy-soon]

table	factors=20 lags=5 lambda_x=0.5	factors=20 lags=50 lambda_x=0.5	factors=20 lags=100 lambda_x=0.5	factors=40 lags=5 lambda_x=0.5	factors=40 lags=50 lambda_x=0.5	factors=40 lags=100 lambda_x=0.5
MatrixMF	0.5084	0.4871	0.4885	0.4685	0.4544	0.4529
LSTMMF	0.4818	0.487	0.4862	0.4484	0.4622	0.455
GRUMF	0.4816	0.4869	0.4874	0.4485	0.4564	0.4574

[deploy-soon]

experiments for RNN cell output

• factors: 60
• lambda_x: 5.0
• GRUMF

table	n_layers=1	n_layers=2	n_layers=3
lags=100 hidden_dim=64	0.273	0.3405	0.9529
lags=100 hidden_dim=128	0.3405	0.343	0.3429
lags=100 hidden_dim=256	0.3229	0.2783	0.3297
lags=200 hidden_dim=64	0.2873	0.2849	0.341
lags=200 hidden_dim=128	0.2707	0.2881	0.3401
lags=200 hidden_dim=256	0.3363	0.2734	0.0

[deploy-soon]

experiments between embeddings and MLP models -lags =100

test loss and number of parameters	table	factors=20	factors=40	factors=80
name=VectorMF	0.3006 1054040	0.3164 2107980	0.3266 4215860
name=MatrixMF	0.298 1055940	0.2849 2111880	0.2865 4223760
name=TensorMF	0.3299 1093940	0.3587 2267880	0.3715 4855760
name=MLPVectorMF	0.2714 1080052	0.28 2133992	0.2829 4241872
name=MLPMatrixMF	0.2939 1576180	0.2802 3152360	0.27 6304720
name=MLPTensorMF	0.271 1571316	0.2561 3142376	0.2518 6284496

[deploy-soon]

• lags = 100
• hidden_dim = 256

table	name=AttnMF	name=AttnLSTMMF
factors=20 kernels=64	0.2485 1064140	0.2549 1175284
factors=20 kernels=128	0.2482 1073100	0.2537 1198068
factors=20 kernels=256	0.2463 1091020	0.2553 1243636
factors=40 kernels=64	0.2487 2124280	0.2499 2244584
factors=40 kernels=128	0.2431 2135800	0.2486 2267368
factors=40 kernels=256	0.2437 2158840	0.0 0
factors=80 kernels=64	0.2454 4251760	0.2498 4383184
factors=80 kernels=128	0.2452 4268400	0.2502 4405968
factors=80 kernels=256	0.2452 4301680	0.258 4451536

[deploy-soon]

Test loss for several recurrent layers and dimensions just LSTM.

• dropout rate: 0.3
• lambda_x = 5.0
• lags: 100

LSTM	factors=20	factors=40	factors=80
n_layers=1 hidden_dim=64	0.2499 1077256	0.246 2137616	0.2397 4258336
n_layers=1 hidden_dim=128	0.2481 1133320	0.2451 2200080	0.2371 4333600
n_layers=1 hidden_dim=256	0.2354 1343752	0.243 2423312	0.236 4582432
n_layers=2 hidden_dim=64	0.257 1110536	0.2542 2170896	0.2525 4291616
n_layers=2 hidden_dim=128	0.2481 1265416	0.2539 2332176	0.2912 4465696
n_layers=2 hidden_dim=256	0.253 1870088	0.2526 2949648	0.2503 5108768