Images, Art, & Video - Challenge Response

[jamesallenevans]

What image or audio data could be relevant to your course/thesis/life research question(s)? What intuitions do you have about the broad content patterns you would express across and between text, image, and/or audio data? Could pre-trained models allow you to unleash image or audio features that would allow you to parsimoniously test this to support your project?

[william-wei-zhu]

Data: image data of companies' office space (e.g. is it open floor or closed cubicles, dark vs light color, etc).

Intuition: a company's culture and productivity are affected by patterns in company office space, which can be identified via image data.

[jacyanthis]

I'm interested in any content related to AI, so these could be Google Images or other sources with tags for AI (or just present on the internet near the words "artificial intelligence"). One common usage of images is the header images of AI news stories. If I had those, we could do something like image sentiment analysis or rough categorization (e.g., many of the headers are abstract human heads with Tron-esque circuits/lighting, which may be easy to identify).

One intuition is that news stories with header images of real human photos will be more focused on social and ethical issues. Pre-trained models may have these coarse categories built in.

[ming-cui]

I would try to build a model that can predict human emotions using audio/music as a predictor. But this training process would need people's subjective ratings.

[k-partha]

Audio/Image data: Shared images/memes on social media. Intuition: Shared images contain deep insight into individual information preferences and behavioral traits. Image embeddings could be very useful!

[jcvotava]

Course research question: How does Marx's work evolve between young/mature writings? How can ML be used to yield hermeneutic insights into Marx's corpus?

Expand into larger research question: How does left-wing speech evolve over time? (1st international --> 2nd international --> ... --> movements today)

Spin-off on larger research question: how have protest or revolutionary movements changed over time/influenced each other etc.?

Use image files to address spin-off question: in the style of "Randall Collins meets machine learning" use images or videos of protests (or even battles? revolutions? not sure) to look at style/tactics/similarity

[jinfei1125]

Emojis and GIFs and videos talking about personal finance

Intuition: people begin to use emojis and GIFs to discuss their opinions instead of typing. I come across this problem when I try to scrape data from the WallStreetBets subreddit page--sometimes there are only images instead of texts.

Examples: https://www.reddit.com/r/wallstreetbets/top/

Models: image recognition?

[sabinahartnett]

Thinking about various personality types and the respective behavior of those users:

It would be useful to look at movies, tv shows or interviews with individuals within each personality type (at the bottom of each description for the 16 personality types on https://www.16personalities.com/ there are a number of celebrities listed) and see if there are differences in their headshots (paralleling the reading for this week: can we distinguish personality type based on appearance) or audio of their characters.

[RobertoBarrosoLuque]

Data: Audio data of representatives in the Mexican congress during policy debates.

Intuition: members of the party in control of a majority and the executive (right now Morena before PRI) have specific semantic styles, whoever is a member of the ruling party can be identified via this idiosyncratic styles.

[MOTOKU666]

Data: News images with topics on Immigrants.

By using the pre-trained label models, we may able to see what's the general themes for specific news on immigrants. In addition, I'm not sure whether it's possible to use blob detection to find out key areas in the pictures and do label predicting respectively. If it's possible, then we can have another dimension of understanding of the news focus.

[Bin-ary-Li]

• When I scraped the Sotheby's website, I happened to also collect the images of artworks as well. So that might be used directly to train a CNN to predict the price of artworks. (dataset is too big to post here)
• Intuition: Images might contain information relevant to art styles and aesthetics that related to artwork price
• Pre-train model (e.g., CNN for art style detecting) might be helpful for extracting features that might just be enough for designing a simple logistic regression model for the price prediction task.

[lilygrier]

I'm interested in using audio data from speeches to measure sentiment. We could use sound bytes from presidential speeches as our data. My intuition is that more fluctuations in sound may indicate greater affect.

[romanticmonkey]

Data: Chinese brush/ink paintings

Intuition: the artwork/artist identification would be much harder than with Western oil paintings. (Prior work on oil paintings has great result with transfer learning (VGG16).)

[hesongrun]

Data: Audio on earnings call

intuition: By using audio data, we may get the tone of company managers. These are novel and different data compared to traditional textual ones. In this way, we can gauge the sentiment of managers more accurately.

[egemenpamukcu]

Data: audio recordings of structured debates It would be interesting to see if audio recordings of debates and characteristics of the voice of a speaker can tell us something about the prospects of the speaker winning the debate. Intuition: feminine vs. masculine voices can impact the views of the audience.

[toecn]

Populism and political charisma have to do with the histrionic qualities of politicians. An analysis could be made with the campaign tv and radio ads and pictures.

[theoevans1]

For my thesis, I've been studying the politics of men's self-help content on YouTube. Data could include audio extracted from videos, as well as images like thumbnails and stills. My intuition is that the tone of the images and audio feel personal and friendly (maybe similar to social images on other platforms), which contributes to the parasocial relationships that viewers form with these creators.