Neurons, perceptrons and synapses. Concepts from biology continue to commence their second innings, this time in machine learning. My question deals with another such term- 'head'.
On Pg 6, under the section for 'Related Methods to Alleviate Catastrophic Forgetting', you refer to 'Multihead Output Layers'. Here, you mention that this study does not go into much detail for their applications and it chooses to focus on situations 'in which the same output units must be shared between tasks'.
On Pg 5, you refer to them again when you say that 'multiheaded network architecture, while potentially effective, can be impractical for real-world implementations.'
Given these 2 properties of such networks, what role do you see them playing in future research that attempts to link methods from neuroscience to machine learning?
Thank you for your sharing your fascinating findings. I continue to be amazed at the parallels between advances in deep learning and human intelligence. I must confess- I am a tad wary about the two converging (if not one overtaking the other). Nonetheless, research like yours certainly holds up a mirror to what we take for granted, and allows us to fully understand the whirring cogs running inside our skulls even as we post and reply to these comments.
Neurons, perceptrons and synapses. Concepts from biology continue to commence their second innings, this time in machine learning. My question deals with another such term- 'head'.
On Pg 6, under the section for 'Related Methods to Alleviate Catastrophic Forgetting', you refer to 'Multihead Output Layers'. Here, you mention that this study does not go into much detail for their applications and it chooses to focus on situations 'in which the same output units must be shared between tasks'.
On Pg 5, you refer to them again when you say that 'multiheaded network architecture, while potentially effective, can be impractical for real-world implementations.'
Given these 2 properties of such networks, what role do you see them playing in future research that attempts to link methods from neuroscience to machine learning?
Thank you for your sharing your fascinating findings. I continue to be amazed at the parallels between advances in deep learning and human intelligence. I must confess- I am a tad wary about the two converging (if not one overtaking the other). Nonetheless, research like yours certainly holds up a mirror to what we take for granted, and allows us to fully understand the whirring cogs running inside our skulls even as we post and reply to these comments.