LSTM for time series analysis

[rmattson1008]

Strongly believe that LSTM/GRU is a good architecture for our problem because they lend themselves well to short-term dependencies in a sequence. It seems like the development of fission or fusion would be pretty state dependent - i.e. fission occurring would depend on how well the cell is battling a pathogen right before a fission event, not on whatever interaction the cell may have had with the pathogen hours before.

Big design choice: what representations to use for each time point, sampled from a frame? Though this is a big question we know that we have tools capable of learning the spatial features (i.e. ResNet).

• We can start from scratch and get some autoencoders rolling (good for anomaly detection but might be a challenge for me to train right)
• We can nudge the initial frames to emphasize "strands" of protein vs gaps and use TearingNet (read one paper in class and got excited)
• we could go with the cnn style embeddings trained on classification (I hate this I don't like relying on control/llo/mdivi when we know they share key behaviors),
• we could do some cnn embedding with triplet loss (this seemed to work well for The Bird Lady at scipy).
• we can set up a "quick" and "user-friendly" way to label frames and beg Amr to give us more specific labels to train on.
• we can forget embedding the images and use time series data we already have like EiGenValUeS and inTeMediAte rePresEnTations or graphs.

General questions: How much do we care about "beginnings" and "endings"? I'm pretty sure that in an LSTM if you learn a pattern that exists at the end of a sequence you won't necessarily recognize that pattern if it shows up at the beginning of another sequence. Is this a feature, like do we want a model to expect a certain storyline in-line with the biological story? Or is it a bug, because our data is manufactured and we don't want to learn the story of how massive amounts of a toxin progress in a cell. If it is the latter we could maybe sample from different portions of the 24 hour videos to soften our expectations of how a sequence progresses.

[rmattson1008]

This is wrong - same lstm module is applied to every time step