An idea

[gottacatchenall]

Hi Tim,

Seeing this gave me an idea for a post-PhD project, which could be a possible sequel or companion book on the web and arxiv on DNNs in particular, i.e. "Deep Learning for Biodiversity Scientists" or similar

I think (and I wonder if you agree) that the primary obstacle to adoption of deep learning is many ecologists interface with models through single functions provided by packages that take structured data as an input, do everything for you, and return a God object with all results. It tends to be hard for deep learning models to fit into that framework, because each question typically requires some tinkering with the model structure and internals that is generally easier to work with Flux/Torch/Keras directly. Even though these are relatively high-level compared to the days of everything at the TF level, I don't think there are many (any?) resources on using these libraries specifically in the context of ecology.

I have a (very loose) structure in mind, something like

Preliminaries

• Review of topics in the other book
  • Classification v Regression
  • Loss functions
  • Stochastic Gradient Descent
• The perceptron as multivariate LM
  • Nodes and weights
• Regression with nonlinear basis functions
  • Adaptable nonlinear bases
• Activation functions
• Meta-topics that are kind of preliminaries
  • Batching
  • Hyperparameter and model tuning
  • CUDA and a bunch of inconvenient truths about GPUs
  • Universal approximation theorems

Supervised

• Simple MLP for interaction classification
  • Two layers, PCA cooccurrence features, just like Strydom & Catchen (et al 2021)
• Spicy MLP for SDM w/ observation model (e.g. like Joseph 2020 but smaller dataset)
• CNNs
  • Small camera trap datasets for both species identification and sequential images identifying movement
• Simple RNN forecasting Examples
  • Occupancy, LPI, community time-series?
  • Soundscape classification
• GNNs
  • Idk of a great ecological dataset for this where it isn't overkill

Unsupervised and semisupervised

• Autoencoders
  • Introduction to feature learning, learn a scalar representation of penguins that are used with an NBC
• Generative Adversarial Networks
  • Not sure what a good ecological example is
• Deep models for reinforcement learning
  • Simple example of RL for sampling design
• The Transformer and Self-Attention
  • Maybe more time-series examples? Idk

[tpoisot]

That makes a lot of sense -- I was originally thinking about this for my sabbatical (but then ended up doing other things). I'm not going to think about it more until I'm done with this class, but we can certaintly revisit this in Jan./Feb.