11/5: Alison Gopnik

[ehuppert]

Comment below with questions or thoughts about the reading for this week's workshop.

Please make your comments by Wednesday 11:59 PM, and upvote at least five of your peers' comments on Thursday prior to the workshop. You need to use 'thumbs-up' for your reactions to count towards 'top comments,' but you can use other emojis on top of the thumbs up.

[nwrim]

Thanks for the innovative paper! I vaguely remember reading a brief summary of this account in another book somewhere (probably from one of your co-authors) and agreeing to this, and I agree on this account even more after reading the paper.

I have two questions:

1) I am interested in intuitive physics a lot, and when I think of it with comparison to intuitive psychology (theory of mind, etc), I sometimes think that we should learn intuitive physics much faster than intuitive psychology. This is because I think intuitive physics has relatively more fixed rules (gravity, collision, etc) than complex rules of social interaction. Do you think the shift from exploration to exploitation occurs faster in some domain/category, or do you think it is a general principle that is shared will all domain/category?

2) The exploration-exploitation trade-off is ideally addressed if we spend roughly the first 1/3 of our resource on exploration, and focus on exploitation in the later stage. Seeing humans going through an exploration stage as a child, this used to be a computationally efficient solution when the average human life expectancy was much lower than now. However, with expected human life expectancy reaching over 70 nowadays, this might not be the ideal setup, and perhaps we need a longer "exploratory" stage. I think there are some cultural shifts that kind of tries to expand the exploration stage - we get educated more, and parents are expected to take care of their child longer (I guess this could also account for the fact that modern society is much more complex than the societies before). Do you think the extension of the exploratory stage is possible in a developmental/cognitive sense? If so, do you think our culture is moving toward the extension?

[rkcatipon]

Dr. Gopnik, thank you for sharing your work! As a layperson in psychology, this was a fascinating read. I am curious about the concept and utility of play as children age into adulthood and what factors may derisk "particular local optima". For example, you mention that adults do better in lab tasks but children have broader hypothesis. It seems that a wider realm of possibility may be the space for innovation in work but it is less rewarded in adults. Rather we are taught to be outcomes-driven at a young age, which is sometimes at odd with play. I was wondering if there were any ways to test the effects of social-group settings on the ability to play. Let's say you were stranded on an island with a volleyball like Tom Hank's character in Castaway. Does solitude encourage exploration, i.e. naming a volleyball Wilson and keeping an imginary friend?

[JadeBenson]

Thank you for sharing your interesting paper! How does schooling affect children's development shift between explore-exploit? You mention how young children (under 5) vs 5-7 year old's experience this shift. It seems that traditional schooling encourages students to exploit rather than explore, and if we wanted to elongate this explore period how might we adapt our pedagogy?

[afchao]

I'm curious what an alternative to your argument might consist of and, in general, about the nature of evolutionary claims. I am not convinced it is possible to demonstrate that the relatively exploratory period of childhood enabled by our characteristically long development is part of the reason we develop into such cognitively complex beings. Couldn't it just as easily be the case that our exploratory behavior is a consequence, rather than a cause, of our extensive development and cognitive complexity? Is it even possible to disentangle these two explanations for the underlying observation that childhood and exploration precede adulthood and exploitation? More broadly, how can we test theories about the evolutionary predicates of modern phenomena?

Thank you for sharing your paper and presentation with our group!

[wanxii]

Although it's not covered in the paper, it's mentioned in the workshop email that "We are ... exploring these learning methods through experiments putting children and AIs in the same virtual environments." Since you've now set up a linkage between reinforcement learning algorithms and childhood cognition in the paper, I'm really interested in learning more about possible follow-up studies in your plans involving the combination of algorithms and human subjects i.e. whether you would design lab experiments to compare childern's cognitive process with algorithms and how or whether you would utlize human subjects' data to calibrate computer programs for future cognitive studies.

Thanks and look forward to the presentation!

[wanitchayap]

Dear Prof.Gopnik, thank you so much for this comprehensive, amazing paper. I am extremely excited to hear you talk about and discuss your idea this Thursday! Life history, learning, adaptation, and risk-taking have been concepts that I am intrigued for a while, so please pardon me if I ask too many questions.

First of all, I agree with both of @nwrim 's points. Especially the second point, where we indeed see the push in the field to consider emerging-adult as another formal life stages (see Arnette). My further question would be where do you think this extension roots in different levels of analysis? Do you think that this is just at a social-level phenomenon, or in fact, it is deeper to the level of psychological or neural one? As for the first point, I think this ties really well to your point of how the qualitative shifts of pre-school to school-age and of school-age to adolescents are different. Do you think the differences can be attributed to the different complexities of the learned concepts like Nak Won hypothesizes? Is it also possible that there are certain cognitive (and maybe social) advantages to explore concepts in different domains at different points in time or at different rates? Do you think there is also a significant trade-off/tension of exploring among different domains even at the very explorative stage of the learners?

Also, I would like to bring individual differences, environment, and life history strategies into the picture. I think the Adaptive Calibration Model of stress responsivity is a great framework to think about.

Similarly to what you mention about the relationship between different learning types and different development strategies in animals, to what extent does this relationship apply to humans learning trajectory?

Considering individual differences from external factors, I think favoring an extended explorative period might not be the optimal strategy in a certain extreme environment, for example, when parental cares and resources are extremely scarce. In that case, it might be better for an individual to cut the explorative period short and favor the exploitative strategy simply because the explorative period cannot be afforded. That is, it might be better for an individual to choose the fast LH strategy instead of the slow one in the fast-slow continuum. However, some research also suggests that individuals on the slow-end of the fast-slow continuum tend to be more risky and impulsive even at their full maturity. From my understanding, exploration also entails risk-taking and impulsivity to some extent. Then, it would seem that the trade-off between explore and exploit and its general temporal trajectory cannot be applied to such cases? Such individuals wouldn't be able to afford a long explorative period in early life, but at the same time, wouldn't be able to employ the exploitative strategy at their maturity too...? How could we explain such trajectories theoretically as well as empirically?

In addition to individual differences from external factors, there are also some internal individual differences in learning. According to the ACM model, when external factors are the same (on the abundant end), learning and LH trajectories would depend on how responsive an individual is toward the environment (sensitive vs buffered individuals in the ACM's typology, and Dandelion vs Orchid children in developmental psychology's). How much do you think the trade-off between exploration and exploitation differs between the two types of learners? Have you seen any empirical study addressing this?

More general questions: as @wanxii mentioned, I just want to ask you your idea of how we should apply AIs and reinforcement learning with the concept of learning, life history, and maybe evolutionary behavioral science in general. How do you see the field moving toward? What should be the general goal or approach the researchers should take?

FinalIy, I also totally agree with @afchao . This is probably the biggest challenge faced by the field. Do you think that by incorporating computational methods appropriately and correctly, this problem can be solved (or at least ameliorate)? Or do you think it is unlikely, and it is just the nature of the field and how we approach and formulate theory/hypothesis in the field?

I am so sorry about this long (and hopefully not silly) questions/ideas. Again, I really am looking forward to your talk 😄

[yierrr]

Thanks for this interesting paper! I really liked how the paper discusses situations in other species as well—now I think of it, my cats did explore a lot more than they do now. This paper actually reminds me of another paper about mere reaction effects, which states that simply reacting to someone's action would actually induce he/she to repeat the action. Hence, I am thinking whether the fact that older people exploit more might be due to the long-term reaction-induced repetition of behaviors. If so, since mere reaction effect can be seen as a bias, do you think it would be an evidence suggesting that biases are by nurture? Thank you so much!

[hhx2207061197]

Thanks for sharing. I'm also curious about how school education affects children's developmental transition between exploration and exploitation? It seems that traditional school education encourages students to develop rather than explore. If we want to extend this exploration period, how will we adapt to our teaching method? Thank you very much!

[Anqi-Zhou]

Thanks for the sharing! So, my question is what kind of role the schooling is in this explore-exploit shift for children? Does less schooling help to extend the explore period? What's the potential mechanism behind the shift?

[ChivLiu]

Thank you for this wonderful presentation! An interesting fact is that many kids nowadays could have gained access to electronic devices easily. When they use apps such as Twitter and Tik Tok, they could learn about the adults' world early and begin to think of themselves as more psychologically mature. It is very similar to the Over-imitation you talked about in the paper. My question is that would those behaviors help kids to learn things faster and have more determined and strong minds when they face difficuties? Or would those only create some fixed thinking that they are mature but that thinking gets crushed when they get problems which are very difficult to handle with?

[linghui-wu]

This idea of comparing the learning pattern of Children and AI is amazing. Though the paper does not mention it, I do hope you can introduce to us your methodology on how to link children’s cognitions to reinforcement learning algorithms. Looking forward to your presentation!

[mintaow]

Hi! I am particularly interested in section 5, where the paper talks about multiple evidence for an explore–exploit the developmental shift. What intrigues me is that it seems some of the evidence indicates there is not always a linear shift among different aging phases. For instance, "adolescents were actually more likely to infer the unlikely social hypothesis than either school-aged children or adults and were as cognitively flexible as the preschoolers."

So I am wondering if it is possible to quantify the explore-exploit development shift to summarize an explicit pattern? (For example, an "age - causal learning ability“ line graph) Thanks!

[Lynx-jr]

Thanks for this illuminating paper! I appreciate the content of explore-exploit and like the black blocks for cost experiment a lot. My questions are fairly simple: how will children’s cognitions patterns change current reinforcement learning algorithms? Do we expect that in the coming years there will be "children version" reinforcement learning and "adult version reinforcement learning applied to different scenarios? Thanks!

[TwoCentimetre]

I wonder if this finding can be used to develop artificial intelligence or other machine learning project, and how it can be transplanted from mere psychological research to a broader area. I am curious how this finding can make a difference in the real world rather than just a theory in the paper. Thank you.

[xzmerry]

Thanks for sharing! The explore-exploit trade-offs and its relationship to childhood recognition and computer sciences algorithms are really illuminating. I have some questions related:

1. You mentioned two types of search (high-temperature and low-temperature search) in the article, and they could bring differences in the learning process between human beings and animals, as well as to give implications for computer sciences algorithms. Can the converse be true (computer sciences help us to learn human recognition process)? Such as simulating the observed patterns based on algorithms in the same logic, and comparing it with the real-world experiments or observations?

2. it is mentioned in the workshop introduction that you conducted experiments but I don't see that in the given paper. I wonder if you could elaborate more on that.

[wu-yt]

Thank you so much for sharing the interesting research! I think comparing AI to children is a good approach, but there are still a lot of unknowns regarding to children’s learning process, would it influence the research?

[Raychanan]

Hi Dr. Gopnik, A comment above mentions the impact that schooling may have on the developmental shift of children between explore-exploit. I'm suddenly curious: since preschool education may be substantially different in nature from schooling, is there anything special about the impact of preschool education on children (explore-exploit)?

[bakerwho]

Thanks for sharing your fascinating work, Dr. Gopnik!

I'm curious about the emergence of 'taste' or 'preference' (or politically speaking, 'ideology') from the early, relatively open phases of explore-exploit . For example, many young children may not fully understand the religion they are being raised in - but are often able to identify their own tastes in food while respecting dietary restrictions (e.g. vegetarianism or avoiding beef or pork) that their religious upbringing may have 'guided' or 'shaped'. As the explore-exploit paradigm delineates a schema for various aspects of life - what is your understanding of the flexibility (or lack thereof) of shifting these schema?

[mikepackard415]

Thanks for sharing! Given that this workshop is mostly populated with 20-somethings who are in a master's program, I'm curious how we should think about high school, college, and graduate school in terms of exploration-exploitation. If I had to guess I'd say the consensus would be that we do more exploration now than we did in our late teen and college years, at least academically speaking. Is academia an exception to what we might come to think of as a general rule?

[MengChenC]

Thank you for your innovative viewpoints regarding the exploration-exploit tradeoff in the paper. I think this explanation offers a new approach when constructing computational model, or more practically, learning new things in an effective and efficient way. Therefore, I am wondering if it is still possible for adults to adopt the measure of learning as young children, and how can we mimic the way of exploration as similar as possible? Thank you.

[bowen-w-zheng]

Thanks for sharing this work. Really interesting! Some studies showed a probability matching phenomenon in human adults. Participants are presented with two choices (A & B), each with a fixed probability of getting rewards (say 70% for A and 30% for B). After some repetitions, participants were found to converge to a strategy that chooses A and B 70% and 30% of times respectively, a clear deviation from the optimal strategy - always choosing A. I think this is an example where humans adults fail to exploit even with repeated learning. Would you expect children to perform worse in this task? Is there any other example where exploration persist even after repeated exposure to rewards or punishment?

[Dxu1]

Thank you for sharing! The explore-exploit trade off is very interesting! A quick question on exploration and early education. Would you expect to see differences in exploration among kids in different education system, where some may have more fixed restrictions (lots of homework and less time for "exploration" (literally)) and some may more flexibility in the curriculum?

[a-bosko]

Thank you for the wonderful paper! It was very interesting to learn about the explore-exploit dilemma. It is really important to understand how in our own research, we might lean towards a "good enough" solution, rather than taking a risk to learn more about the unknown.

It is also really interesting to think about play; as kids, we are often encouraged to explore new environments and take in new information. One question that I have is, how does the "play" behavior exhibit itself in adults? Is it possible that those who are more exploratory and are more open to new experiences are more likely to succeed? Also, can personality play a factor in explore vs. exploit behavior?

Thank you very much!

[william-wei-zhu]

Thank you for the paper. This paper mainly focuses on explore-exploit trade-off from a cognitive development stand point for early childhood. I wonder if similar theory also applies from a morality development stand point as well?

[k-partha]

I remember reading about a study where children who chose "one cookie now" vs "two later" were shown to have lower average SATs in high school. Do you think this finding can be framed in the context of the role of intelligence in the exploration-exploitation tradeoff?

Additionally, do you think that the psychometric trait openness is an ideal candidate for capturing predominantly "explore" preferences?

[hesongrun]

Thanks a lot for the presentation! The idea of drawing connection between children and AI is quite fascinating. However, I think that machine learning may be a completely different regime from human learning. To make an analogy, planes do not fly like birds. They do not flap their wings. The building block of artificial neural network is not from the biological neural network. There is also no clear evidence that human brain performs backward propagation as NN does. Therefore, I am wondering if it is valid to draw inference by comparing machine learning with human learning?

[timqzhang]

Thank you for your presentation, and it is indeed an interesting paper connecting AI and children learning. Actually I think both AI and process of children learning are currently "blackbox" that are not clearly tackled, so it is a good time to have an approach to relate them? I mean, we may get some biases if we don't know the true picture (though it is how estimation works).

[MkramerPsych]

Thank you for sharing your research with us!

I was curious if you know anyone or plan to conduct analyses comparing children to adults using representational similarity analysis as a framework? RSA allows for a unit-agnostic method of comparing across subjects in neuroscience that would allow for the comparison of a child and adult brain in terms of the representational dissimilarity of information in specific brain regions. I'm specifically curious if the argument you make in terms of neuroscience supporting an explore-exploit shift could be reflected in representational dissimilarity between child and adult brains.

[jsoll1]

This explore-exploit dilemma seems super neat! How do you link the results of the 'blinker' block experiment to the explore/exploit dynamic? It doesn't seem to entirely follow that the results imply that adults are worse at exploration that children (though I totally believe it!).

[SoyBison]

Thanks for coming to our workshop. One of the interesting things you bring up is how computational models still pale in comparison to even children. I wonder how much this can be put up to a sort of "life intuition" like common sense which a child has that a computer system does not. A computer system is of course only equipped to solve the particular problem, but a child is a more general intelligence, even if they are not as equipped as adults, or even juveniles.

[chiayunc]

Thank you for sharing your wonderful work. My question is regarding the comparison between AI and human child learning. I think that although they appear to have similar output or even comparable trajectories, do you think they are fundamentally different?

[bjcliang-uchi]

Thank you for this presentation. I have to say this topic is beyond my knowledge, but I am wondering, from an economic perspective: how do the concepts of risk and uncertainty matter to the explore-exploit dilemma? It seems to me that to explore or to exploit is more like a calculation of expected payoff rather than a matter of taste.

[Yilun0221]

Thank you for the presentation! I love this topic which is kind of related to brain science. My concern is similar what Songrun @hesongrun has mentioned. Every child is born very differently with others in terms of their talents, personalities and resources they are exposed to, which will certainly affect their learning process. I wonder how you deal with these settings for machine?

[chrismaurice0]

Hello Dr. Gopnik! Thank you for sharing your paper with us. Quite a few of my peers have been asking about the effects of schooling on childhood development, but I am interested in the reciprocal of that question. How should educators interpret and implement the findings of your paper? Are there ways we can use what you have shown us about childhood development to create a more robust early education system?

[LFShan]

Thank you for sharing. I think the explore-exploit dilemma during childhood development is very interesting. The question I have is that the transformation from more exploring to more exploiting happens when a child grows. Could it be possible that the child's move toward exploiting is because he or she becomes more risk-averse since he or she encounters more and more events that punish explorations?

[skanthan95]

Thank you for speaking at our workshop, and for the fascinating paper! The reinforcement learning task study was particularly interesting to me, and reminded me of Jostein Gaarder's Sophie's World. I'd like to know more about why children shift towards being more risk averse over time - beyond what we know about how neuroplasticity favors the young. It seems that favoring 'exploration' (vs. exploitation) has evolutionary benefits -- or, at least a more tempered balance between exploration and exploitation.

[alevi98]

Thanks for the paper! I had never heard of the exploit-explore dilemma, but now I'll be thinking about it every time I'm going out to eat with friends, but can't pick a restaurant. (Maybe it'll become especially relevant post-COVID.) Something I was thinking about with this notion that as we get older, we neurologically shift from explore to exploit: to what extent do societal expectations of property acquisition instill values that tends towards exploitation, rather than exploration? Could it be that, as we evolved into societies beholden to property, the most inclined to exploit as they grew older (rather than explore) received better medical care, better resource allocation, etc. thus lived longer, prospered, raised offspring more likely to thrive. This actually has me thinking deeply about biopower. I'm excited to hear more about the neurobiological basis tomorrow.

[egemenpamukcu]

Thank you for sharing your work! It was my first time reading about this explore-exploit tension and, while reading, I could not stop myself from thinking about the progressivism-conservatism divide in political ideology, which is also likely to be as old as our species. Do you think this ideological divide can be an extension (or an evolutionary byproduct as Dawkins calls it) of this explore-exploit tension that is formed to ensure our continuation? I thought of this because the ideological divide is also universally associated with young vs. old age and characteristics of progressive people (not limited to today's progressive politicians or public figures) embrace exploration while 'conservatives' are more concerned with exploitation using existing mechanisms. I know that there is some evidence suggesting people get more conservative as they get older (although there is also a significant cohort effect), just as they get more exploitative as you show in your study. I think it would be really interesting to look at these findings from a behavioral (perhaps developmental?) politics point of view.

[Bin-ary-Li]

Dear Professor Gopnik, first I want to say I am a great fan of your works. I was not able to meet with you during my stay at Professor Piantadosi's lab at Berkeley, but I did get to know one of your graduate students and your works come up a lot during our discussion.

A more general question I have is, what exactly do you have in mind when we talk about this marriage between reinforcement learning and developmental psychology. I know at some point Dr. Joshua Tenenbaum at MIT also talked about how we build better machine learning algorithms through studying children. But I am still unsure how exactly will the result from psychology be of use to machine learning scientists. Can you maybe give some comments on the potential directions to take?

[ydeng117]

Thank you for sharing your work. It is really a great idea to apply children's learning mechanisms for developing AI. Nonetheless, one difference between AI and a human child is that AI does not have genetics. So, my question is whether there exists a genetic heritage for human children which leads them to "naturally" engage in active learning? How would deep learning engineers design an algorithm to mimic such an innate impulse for AI?

[yiq029]

Thank you so much for sharing the interesting paper! Connecting children learning to AI is so interesting, but I am wondering that since children's learning process is very hard to measure and evaluate, how could you make sure that this is correctly quantitatively represented? Thank you.

[sabinahartnett]

Thank you for sharing this interesting work!!

In section 3, you write that, "In general, though, the value of exploration is directly related to the dimensionality and complexity of both the environment and the potential adaptations to that environment. The broader and more complex the possibilities are, the more exploration will be valuable" - is it the case that humans will always explore proportional to the opportunity to explore? Or is there a bell curve of optimal complexity for development? In the same way that adults will make rational decisions when faced with a comprehendible number of choices, but will begin to act irrationally in the face of too many choices, is there an optimal complexity environment for children?

[Qiuyu-Li]

Thanks for sharing. Actually, I’m curious about a sociological question: how modern society distorts the normal process of children’s transition from exploration to exploitation? For example, to win in the school entrance competition, many children in my country are pushed to learn math, English, among others, at a very early age. I’m quite interested to know if that will lead to long-term consequences in terms of one’s intelligence and creativity development.

[vinsonyz]

Hi Professor Gopnik, thank you for your presentation. Since I am not familiar with this context, I have a general question about this field. Although there are some similarities between AI and human child learning, are they intrinsically similar to each other?

[Yaweili19]

Thank you for sharing the research. I never thought that the learning process of young children could help solve a big balance in our computer process. It's a pity that I don't quite understand the reading materials, and I haven't figured out what kind of reference is there between explore-exploit and our research objects. Hope to find the answer from your speech.

[Leahjl]

Thanks for this interesting presentation! My question is about the learning methods in AI simulation, how does the algorithm mimic the child's learning process accurately?

[yutianlai]

Thanks for sharing! I'm wondering how you handle the "nature" part of children that might have no equivalent in AI?

[tianyueniu]

Thank you for sharing this interesting research. I found your argument of childhood being the solution to explore-exploit tension really inspiring. I don't fully understand how we can transform this finding to develop AI, and I look forward to learning more in your presentation.

[FranciscoRMendes]

I am curious to know if all aspects of learning can be quantified as you have attempted to do. One of the most important things we learn in school is actually how to interact with other humans , what do you feel about this sort of culture based learning?

[harryx113]

Thank you for your presentation. I think there is so much we have yet understood about children's learning process. What do you think the exploration-exploitation tradeoff behaviors suggest on neural processing? What criteria are the decision based upon and if there is anything to be incorporated in the modern deep learning paradigm?