search

djanloo / cmepda

An LSTM + Encoder network for UHECR AirShowers
1 stars 2 forks source link

Subnet tests #15

Open djanloo opened 2 years ago

djanloo commented 2 years ago

Since the LstmEncoder is a compound net, we should test the resolution of each subnet to ensure a "dividi et impera" workflow.

We should not take for granted that optimizing a part of the net should lead to the whole net improving, but we'll never know if we don't try.

djanloo commented 2 years ago

Found a major issue in training the encoder subnet since apparently keras can train a network with 4 output values with one target value. Adding a final 1-dense layer would fix the problem.

luciapapalini commented 2 years ago

Screenshot 2022-07-13 at 22 02 54 Does this need a comment? ... (LSTM)

luciapapalini commented 2 years ago

I'm starting to think that what you've said in the first comment is correct. Maybe optimising the modules separately is not improving the performance of the whole network.

djanloo commented 2 years ago

@luciapapalini LSTM with the last 1-Dense is a Bugatti screenlstm

djanloo commented 2 years ago

Encoder is awful again

encoder_shit_again

djanloo commented 2 years ago

Now with 512 -> 256 -> ... -> 4 seems to learn a kind of trend. sub_net_train