search

uchicago-computation-workshop / nicolas_masse

Repository for Nicolas Masse's presentation at the CSS Workshop (1/13/2019)
0 stars 0 forks source link

Losing One's Head- Why More May not be Merrier #10

Open policyglot opened 5 years ago

policyglot commented 5 years ago

Neurons, perceptrons and synapses. Concepts from biology continue to commence their second innings, this time in machine learning. My question deals with another such term- 'head'.

On Pg 6, under the section for 'Related Methods to Alleviate Catastrophic Forgetting', you refer to 'Multihead Output Layers'. Here, you mention that this study does not go into much detail for their applications and it chooses to focus on situations 'in which the same output units must be shared between tasks'.

On Pg 5, you refer to them again when you say that 'multiheaded network architecture, while potentially effective, can be impractical for real-world implementations.'

Given these 2 properties of such networks, what role do you see them playing in future research that attempts to link methods from neuroscience to machine learning?

Thank you for your sharing your fascinating findings. I continue to be amazed at the parallels between advances in deep learning and human intelligence. I must confess- I am a tad wary about the two converging (if not one overtaking the other). Nonetheless, research like yours certainly holds up a mirror to what we take for granted, and allows us to fully understand the whirring cogs running inside our skulls even as we post and reply to these comments.

w4rner commented 5 years ago

How long til AI renders human polyglots impotent?

w4rner commented 5 years ago

@policyglot can you explain title pls?