04/09: Bainbridge

[shevajia]

Comment below with questions or thoughts about the reading for this week's workshop.

Please make your comments by Wednesday 11:59 PM, and upvote at least five of your peers' comments on Thursday prior to the workshop. You need to use 'thumbs-up' for your reactions to count towards 'top comments,' but you can use other emojis on top of the thumbs up.

[wanitchayap]

Thank you so much in advance for your presentation and for sharing your research with us!

It is very interesting how your results show that people tend to recall the lower regions of the stimuli more. As you already address this discrepancy between the lower region bias and saliency/meaning in the discussion, I agree that the bias may have something to do with the stimuli being sceneries. I am not very familiar with perception and memory, but could this relate to how the stimuli are the 2D representation of actually 3D sceneries? I notice that all of the stimuli shown in the paper are pictures taken in perspective angles instead of flat angles. Our brain may need to strategize differently when trying to memorize a 3D representation shown in a 2D stimulus. (I also noted that categorical drawings shown in the paper are flatter in angle than the rest of the drawings). I wonder if adding stimuli that are taken in flat angles would help. On the other hand, do you think that abstracting the stimuli away from 3D at all and keep everything 2D will be better (using drawings, scenery without obvious depth/dimension, etc.)? In addition, since there are more techniques that allow actual 3D representations (interactive 3D picture, VR, etc.), do you think that employing these techniques worth exploring in memory/perception research as 3D stimuli are closer to stimuli in the real world? I apologize in advance for my limit understanding of the topics and if these questions doesn't make much sense!

[nwrim]

Thank you for presenting great research! I think this is one of the most interesting ways of using AMT in psychological studies I had seen. I have some questions:

(1) Similar to you testing the memorability of physically salient (GBVS) and semantically meaningful (meaning map), I wonder if there is a memorability difference in objects that can be seen as canonical to the category (bed in a bedroom, for example) or very atypical to the category (classical example - a skull in an office) or do not really fit into both category (an object that can be found in the category, but not as canonical as the first category). Many classical memory studies suggest that there will be a difference, but I was wondering if that could be seen in your dataset.

(2) Was there any heterogeneity in the category of scenes (in-door vs out-door, typical vs atypical, etc)? Specifically, I am wondering if there is a difference between scenes where there is a more typical organization or scenes where there is high variability.

(3) A different set of participants draw the pictures from memory and draw the categorical pictures. But I feel like there could be some individual differences in what object that people think is canonical in their representation space. I guess priming effect (and etc) might be affecting the result a lot if we do both tasks on the same set of participants, but I am curious if there is a way to bypass this if this indeed is a problem.

(4) Although I really think this is an awesome memory task, I think using thousands of AMT workers to evaluate the performance might be a little expensive compared to the verbal recall or pictorial recognition task you mentioned in the article. Do you think this could be a barrier to other psychologists to test this procedure out for their own memory research? If so, do you think there is a way to make this kind of memory task a little inexpensive so it used in other memory research?

Thank you again!

[WMhYang]

Thank you in advance for sharing your interesting paper. I am curious where the additional objects in delayed recall, immediate recall, and image drawing come from (I apologize if this question does not make sense given my limited knowledge in this topic). To be specific, are the additional objects observed in category drawing of the same person? (i.e. they are from the canonical representations of the scene for that person.) Or, do the additional objects appear in other images that the person has to memory? (i.e. they are from another image but the person memorize wrongly.) Or, is there a tendency that images with more details are more likely to witness additional objects? (i.e. they are from complexity of the image.) Since the additional objects are rare, could we formally test these hypothese?

[bhargavvader]

Super interesting work, thank you. I have a question about the role of real time distractors in remembering - in the experimental setup, AMT workers are at home and looking at images on a screen. But when we experience sight in the real world, we experience it along side a multitude of other senses. Considering theories of embodied cognition (Varela, 1991) which suggest that cognition is very likely multi-modal, how do you suggest to account for that while constructing remembering tasks? Do you think if the same kind of experiment was conducted but with actual real world scenarios instead of images, the memory with context of recall and remember would improve (or decline!)?

[timqzhang]

Thank you for your paper. It is a great research on memory mechanisms via experiments. My first question considers the digit span task during the delayed recall experiment. I wonder if it is ok to just control the task by asking participants to recite number series, as there exists some other potential "cues" that could alter the effect of this delayed recall task, ie. if the place where the experiment undergoes is similar to the layout in the image, or some particular items in the image also appear in the experiment room. The participants would probably just notice that and use it to remember the image during the study phase. So is it well controlled for such a digit reciting task to test the delayed call?

[anqi-hu]

Thank you for sharing your work!

You mentioned that participants did not just remember the objects in lieu of the image, nor did they remember the inter-object relationships. Instead, they registered a "spatial map" of the image itself. Going off of @nwrim 's first point, I'm wondering if this observation has anything to do with the fact that all of the test images simply 'make sense' in real life, as opposed to using an array of randomly selected objects that were positioned in the same image but would otherwise make no sense. In other words, to what extent does the spatial map itself have an impact on one image being easy to remember and recall? In the case of randomly displayed objects, I'd imagine some items in a certain random image would definitely stand out and be registered more quickly and for a longer time than others. Will such tweaks to the current experiment result in different conclusions/ significance?

[chiayunc]

Thank you for sharing your wonderful work.

My question is related to the memorability of images, which you controlled for in the study. You mentioned that you determined memorable images from un-memorable images based on previous large scale studies. I was wondering, could what you find in this work somehow account for how the image people see affects the way they remember things? i.e. what makes an image memorable. Do you find any pattern in the result that might shed some light? Are there any common traits in the images that are well remembered? if so, can we inversely make the assumption that these traits are what makes an image memorable?

[zeyuxu1997]

Thanks for sharing your interesting and meaningful work. I have some questions about participants. I checked the appendix describing participants and find there are only description about gender and age, however, I think education and occupation might affect their ability to remember stuffs. Besides, the data shows that most of participants are between 20 to 30 years old, are they chosen on purpose? If so, why do you think they are good subjects for your research question? Will your conclusions change for younger or older people?

[Yilun0221]

Thanks for the presentation! I have one question: Does the selection of pictures to test people's memory have influence on the results of the research? I mean that maybe people's brain is more sensitve to some objects, colors, themes, etc, and this situation may affect the result of the research.

[AllisonXiong]

Thank you so much in advance for your inspiring methodology on invetigating visual memory! I'm really impressed by the way you managed to measure the quality of recalling on complex stimuli, and application of online crowd-sourcing in cognitive psychology study (though @nwrim raised a good point on the badget issue). Here are some questions that confuse me:

1) The verbal & visual representation of the stimuli. As for the Delayed Recall group, a digit span task was performed before the recall to "limit verbal maintenance", while participants in other groups did not receive such treatment. Do you think the better performance of the latter is caused partially by the edge of verbal maintainence of the images? 2) Though I found it necessary to investigate whether subjects rely heavily on the canonical representation of each category, the idea of asking people to draw with no clue other than category names may introduce interference such as cultural difference for comparison later, since the canonical representation can vary across individual from different (or even the same) cultural background. Asian families, for instance, tend not to have fireplaces in the living room. 3) Additionally, I don't think the average number of incorrect objetcs is the best measure for memory intrusion. In the recall of the desert scene, for instance, both drawings included cactus, suggesting the use/disturbance of canonical representationof the category, in contrast of the previous results.

Thanks again for presenting this awesome research to us!

[bjcliang-uchi]

My question echoes the previous ones such as those raised by @Yilun0221 and @chiayunc: it seems that human memory depends heavily on our prior knowledge; That is, we fit our visual memory to the things we have already known. For example, a neurosurgeon can certainly recall the structure of skulls better than ordinary people. How would you control such education/ cultural factors? Also, do you think that this understanding of human memory can be applied to computer vision development?

[lulululugagaga]

Your use of image-based metrics is really interesting. I'm very impressed by such innovative work. Similar with previous ones, as this research heavily relies on the result of recognition, I was wondering if the effect of prior memory can be properly controlled/reduced in these experiments. Also, why you set the age range in 20s and not expand to a wider age group? Thank you!

[linghui-wu]

Thank you for the brilliant work! I have to admit that the drawings in the paper really appeal to me. And I appreciate the discussion of my peers above, which really helps me comprehend the methodology of the paper. My question extends a little from the third point of @AllisonXiong. I would like to know the accurate definition or measurement of errors, or how various types of mistakes are differentiated. For example, in the recall of a bedroom picture, would a pillow with different colors be invariably considered as the incorrect object as a non-existing plant?

[ydeng117]

Thank you for your presentation. This study is very interesting! In urban geography and urban sociology, one method for studying urban planning and the urban landscape is to ask respondents to draw the image of the city. This method is quite similar to your study. My question is, how does the daily image that a person can constantly see, such as the image of a city, can be quantified using your method?

[hihowme]

Thanks a lot for your presentation. Your use of image is truly brilliant and inspiring. I am wondering how you decide the demographics of participants to be considered in this study?

[dhruvalb]

Thank you for your presentation! It is a really interesting methodology to quantify findings from drawings and allow free recall of a memory. I was curious, would the results of recalling a visual memory be impacted if the participants were asked to verbally share what they remember versus drawing it?

[tonofshell]

Thank you for sharing your work! Do you think visual memory recall might be more accurate when drawing more familiar scenes? For example, would someone who lived in Arizona draw a provided desert scene from memory more accurately than someone from Illinois?

[hanjiaxu]

Thank you for your presentation! I think that this is very interesting research. My question is that do you think if pictures with an emotional-salience to the subject would help them memorize better? Does that confound the main result? thank you!

[KenChenCompEcon]

Thanks so much for presenting the interesting work. I am wondering about the possibility of scoring bias posed by online reviewers. Is it possible that they tend to give out better scores for drawings of objects that are relatively more familiar? How to ensure their consistency in scoring across drawings of different objects? thank you!

[bakerwho]

Thank you for presenting.

This is a fascinating insight into human memory, and one of the most impressive uses of AMT I've seen in a long time. Considering the deep learning revolution's effect on image processing - specifically object recognition in neural networks - do you have any insights that might inform advances in the field?

I can think of one interesting insight that could potentially come from expanding on your work. Consider the contrast between canonical and Delayed Recall image drawings - what is the 'error measure' in comparing the two, and how best can additional information be encoded to inform a canonical description on the ways it should be changed?

[YuxinNg]

Thank you for the paper. I am really impressed to see how the memory can be quantified. Though I am not trained in this field, I know memory can be very tricky. So my question would be memory can be affected by the environment, will your findings hold if the situation is different. That is to say, what may be the limitation of this findings. And one thing I am particularly interested in is that memory may be affected greatly by age. I am wondering if your findings can be applied to people of different ages. Thanks!

[RuoyunTan]

Thank you for sharing your paper with us. I do not have a background in psychology but this experiment is really fascinating to me. I wonder if how people memorize things would vary with the object or scene they looked at. Would the results be different if people were to memorize indoor/outdoor scenes, regular/irregular shaped objects, or colorful/dull scenes differently?

[skanthan95]

Thank you for sharing your work with us! Here are my questions:

In the Delayed Recall test phase, participants were then asked to draw as many images from the study phase as they could remember with as much detail as possible, and were given as much time as they needed to complete the task.

(1) How might findings in the Delayed Recall condition have differed if participants had a limited amount of time to draw the image, as opposed to completing the digit task and then drawing the images with no time limit? Would it be worth adding a subcondition where participants had a limited amount of time in that respect?

To assess whether there was additional information contained within memories that could not be initially accessed, participants were then cued with a diagnostic object for each category (e.g., bed in a bedroom) and were allowed to draw a separate set of any new images they could not recall before.

(2) How were diagnostic objects operationalized? If there were multiple diagnostic objects in a given picture, was one randomly selected for the participant cue/what was the methodology behind the selection process?

[PAHADRIANUS]

Thank you for presenting this work of fascinating results built on innovative methods. The studies on visual memory are a lot more difficult to quantify and then to explain than those based on numbers and alphabets, and thus the field is much less well known. The first occurrence of the paper to me having read the abstract was that it must be challenging, even with online crowd-sourcing, to properly score and categorize the experiment results. The amount of effort you put into not only crafting the well controlled laboratory experiment is already impressive, but what's more impressive are the various extensions of the online scoring experiment you devised to have vast numbers of recruits to analyze the drawings. In this way, objective evaluation of the images, to a certain degree, was obtained. My question regarding the project is the following: whereas the crowd-sourcing results were quite sufficient for the observational studies, is it possible to design some models to describe the human behavior when recalling objects? For instance, a model that captures how we select the prioritized objects to memorize or a model that shows the factors on imagery memory decay?

[HaowenShang]

Thanks for your presentation! I am quite interesting on whether the characteristics of testing people have influence on the results of the research? For example, the age, gender, educational background. Whether young people have better memory than elder people? Thank you!

[MegicLF]

Thank you for presenting such inspiring work! I wonder how you identify the impact of ones' previous memories and personal experience to their recall, and how you did to reduce such influence. Besides, I am quite interested in the results for different categories of scenes. Is it possible that the recall of participants may have special characteristics towards different types of scenes? For example, is it possible that females may tend to remember the natural scene better than males? Thank you!

[boyangqu]

Thanks for your presentation in advance! The topic appears to be really interesting and applicable to real world scenarios. I wonder that does recent memories and remote memories have different effects? And would memories affect people at different ages differently?

[SoyBison]

Thanks for attending our workshop! I noticed that you tracked details that were missing over time in the recall drawings. On the other side of things, were there any scenarios where the subject would add things to the drawing? Memory fabrication can be an interesting phenomenon, since it's generally more prevalent in long-term recall, I'm interested to see if you saw anything like this even on the time scales that you're working with.

[sunying2018]

Thanks for your presentation! I am interested in the participants in this experiment. Considering the limited number of participants (eg. for delayed recall, there are 30 participants), how to eliminate the influence of other factors such as age, education background, whether they received memory training before to ensure the result is valid enough?

[chun-hu]

Thank you for your presentation! Recall that one of my previous researches looks at the effect of training on visual learning, I'm wondering if the presence of a stimulus/reward during the memory task will improve or suppress the performance on recall. That is, will the external cues serve as a reinforcement or a distraction to the memory?

[dongchengecon]

Thanks a lot for presentation! This is a really interesting topic. I am wondering if it is possible to generate some measurements for different images and try to predict people's memory performance based on the stochastic development of these measurements? The general assumption is that some similarities between consecutive images would improve the memory in those particular areas, while some distraction might cause memory loss.

[SiyuanPengMike]

Thanks a lot for your interesting and inspiring paper. Both of your methods and results are impressive. I have a quick question about the selection of those pictures. As we all know, human brains have a preference for specific colors, and there is a difference between left brain type and right brain type. Therefore, will the change in those pictures change your result as well? For example, if we change the brightness and saturation of those pictures, will it be helpful for people to memorize them? Thanks again for your inspiring research!

[minminfly68]

Thanks for the presentation. It is a very interesting and inspiring topic. Maybe I am not an expert in this field, but I am wondering the scoring bias from the online reviewers. Are there any character which might influence their decision (i.e. More familiarity leads to a higher score)? Can you elaborate more on this perspective?

[Leahjl]

Thank you for your presentation. It is very innovative to quantify the content of memory. My question is about the method you used to score the memory-based drawings. Is there any alternative way to evaluate the memory? It seems that online observer's evaluation can be subjective.

[harryx113]

Thank you for the presentation. I think conducting psychology experiments online might just be the new paradigm. How do you view the effect of attention/effort made by online observers? In a lab setting, researchers can be sure that the participants are free from distractions, but it is not easy to evaluate the efficiency of the information acquisition stage. What is your take on this?

[mingtao-gao]

Thank you for the presentation in advance! This paper appears to be very interesting and the algorithms used to model image perception (GBVS and Meaning Maps) are intriguing. It indeed provides many insights on the qualification of the content of free recall spacial memory. My question comes in the free recall memory task. The participants are asked to draw as many images as they could, and the paper focuses on the content of delayed recall images, but I wonder if there are any patterns in the images the participants are more likely to recall and provide more detailed information in their drawings. I think this research and its method can also be used to study what spacial characteristics are more memorable.

[rkcatipon]

Funnily enough, your research made me think of Virginia Woolf's To the Lighthouse-- but instead of a household recounting a single day in a modernist exploration of memory and temporality, we have thousands of AMT workers. What fascinates me about your work is how consistent the recall drawings were. How did you select the participants for the Image Drawing Experiment, what factors did you try to control for?

[yutianlai]

Thanks for sharing! I'm wondering how you would deal with selection bias in your research. Your participants can be different in terms of previous knowledge, or sensitivity to images (for instance, people with autism think more "visually" than others). How would you control these factors to make your results more reliable?

[ShanglunLi]

Thank you for providing such an interesting paper to read! My question is that does the appearance of the picture which used in the memory test affect the result of the experiment?

[di-Tong]

Thanks for sharing this interesting project! Could you explain a bit more on the broader implication of the detailed object and spatial information in memory?

[tianyueniu]

Thank you so much for the paper! Using computational methods to quantitatively evaluate memory is really inspiring. I find it really interesting that even in immediate drawing, people would still draw imaginary objects that's not presented in the picture. What do you think is a valid explanation for this kind of behavior?

[ShuyanHuang]

Thank you for presenting. In this experiment you divide the images into two groups with different memorability, based on a previous experiment. I think it would be interesting to study which features make some images more memorable (e.g. contrasting colors, fewer objects, etc.), using the methods and measures in this paper.

[heathercchen]

Thank you very for presenting! I am curious about your participant recruitment process. What kind of standard or criterion you used when recruiting participants?

[ChivLiu]

Thank you for the presentation! When we recall something, the memory we had was processed with our understanding and experiences. It is like we hardly describe a comprehensive dream after being awake for more than 30 minutes. I am wondering how instant recalling could improve the accuracy of the experience-enhanced memory and how drawing the images could affect people to further add their previous experiences to their recallings?

[liu431]

Thank you for the talk. The measured outcome of your study is recalling information. However, our brains also do more intellectual tasks from images, such as inferring content and telling stories. Do you think it could the possible extension of the study?

[ruixili]

Thanks for your presentation! I wonder how to test if the random sampling is correctly separating our treatment and control group? What's more, is it possible to break down the influence on different groups?

[fulinguo]

Thank you for your presentation! I am wondering if we could study the heterogeneities among people in memorizing things. If there are heterogeneities, what parameters did we estimate in the study? Do they represent the average value of all people? If the people who participate are different from those who do not participate in the test, how could we interpret the results? Thanks!

[TianxinZheng]

Thanks for your presentation. This is really interesting work. I'm wondering if the background of the participants in the Image Drawing Experiment would impact the results. If so, how do you control for that?

[nt546]

Thanks for presenting your work!

[Panyw97]

Thank you so much for your presentation! As you studied human recall memory and let the participants draw the images they remembered. I am considering that this may lead to some bias because people would have different preferences towards different realms. They may be impressed by the images that match their preferences. How do you deal with that bias?