4/15: James Evans

[ehuppert]

Comment below with questions or thoughts about the reading for this week's workshop.

Please make your comments by Wednesday 11:59 PM, and upvote at least five of your peers' comments on Thursday prior to the workshop. You need to use 'thumbs-up' for your reactions to count towards 'top comments,' but you can use other emojis on top of the thumbs up.

[JadeBenson]

Thank you for this fascinating research! This project has brought up a lot of philosophical questions for me about how we can actually leverage incommensurability to inspire scientific discovery (and their paradigm shifts). You mention that these recommendations "will not be beautiful" and will be outside the burden of knowledge - this is exactly what makes them exciting but also difficult to justify their pursuit (to yourself, colleagues, funding organizations). I'd be curious to hear your ideas on the future potential of these methods and how we can interpret, accept, and then act on "alien" findings, especially within our current complicated scientific landscape?

[mikepackard415]

Hi Professor Evans - thanks for sharing this work! I'm really glad we'll have a chance to discuss it. My question may also be somewhat philosophical: I'm wondering about the extent to which you see all scientific knowledge production as essentially a matter of combinatorics - combining existing ideas/materials/methods/goals in novel ways. This seems to be an underlying assumption of this research, but please correct me if I'm misinterpreting. If this is largely how we think about innovation, and if it then becomes the role of the scientist to define goals that can be solved this way so that they have something to publish, does that system create a blind spot in the kinds of questions science can be expected to answer?

Another approach to the same question might be: is this combinatoric view of innovation necessarily reductionist, and if so, what does that mean for the kinds of outcomes we can expect from science? As I'm sure you know, SFI often emphasizes the importance of systems thinking (especially complex systems) for dealing with global challenges, and systems thinking is essentially anti-reductionist. Excited to get your thoughts on this.

Also - for my classmates and anyone else reading this - I recommend listening to Professor Evans talk about this work on his episode of a podcast called Complexity, produced by the Santa Fe Institute. I believe it is their third-most-recent episode now. Here is a spotify link.

[nwrim]

A truly interesting paper, James! I must have missed Jamshid's lab meeting day due to my sleep schedule nowadays (I hope my body let me attend the talk). I don't necessarily agree with the point below, but after our brief talk yesterday, I decided to bring my pretty captious side out of me.

A reader might say: This paper is overstating the power of humans, thus inflating the power of including co-authorship in the model. This criticism could be most easily done in regard to the random walk, specifically the length of walk that is allowed. The algorithm that the paper uses allows a maximum of 20 steps. Since the walk was balanced between taking a step using material or author, it could be claimed that maybe most walks incorporates 7~8 walks of authors. This could be problematic since: 1) These many "steps" in forming collaboration might not be ecologically valid - this means that the research team is collaborating with "a person that a person that a person that a person that a person that a person that a person that a person I worked with know". In addition to the "intuitional" difficulty of this happening, if we consider locational, language, institutional, or personal (yes, I am aware there are co-authors that swear they will never work with each other again after a project) difficulties, then it makes even less sense. 2) Considering the small world hypothesis (or any fancy hypothesis of that kind), these many steps will actually connect almost all authors in a domain. Thus, this could be seen as pooling from the collaborative power of the entire scholarship space, rather than plausible co-authorship.

A reader might say: This paper is understating the power of humans, thus inflating the novelty of AI-generated suggestions. This criticism could also be most easily done with the hypergraph. Co-authorship is not the only way a researcher knows another researcher - there is a lot of locational, language, institutional, or personal serendipities that make a researcher get introduced to another researcher or another material. Thus, by only incorporating the co-authorship connection, the paper is overstating the novelness of the AI model - if we could incorporate "additional" hyperedges, then the effect might decrease by a large margin.

Again, I do not agree with both of these claims (and I am aware that they are saying the exact opposite using the exact same logic just reversed) totally. I would hate the reviewer if they brought up such (petty) objections. But this was me in full reviewer-two mode...

I look forward to the talk!

[anuraag94]

Thanks for sharing this interesting paper with us Professor Evans. Your current model generates novel candidates and combinations of materials and chemicals that are unlikely to be discovered without intervention.

I am curious whether you think that an alien AI similar to the one that you've described may be able to use metadata on social science papers to generate novel combinations of preexisting theories worthy of investigation by theorists. If not, what sorts of changes might have to be made to your model to make such an approach tractable?

Social science research typically has a long lifecycle where a novel theory may inspire years of research validating it, but is then mostly supplanted in the literature by a theory that reclaims popular support. If your approach can extend to social science, I think that alien AI can play an important role in interrupting this lifecycle for faster innovation.

[AlexPrizzy]

Thank you James for sharing this MEGA fascinating work! The symbiosis of AI and human researchers forecasts an exciting future for the scientific realm.

AI guided paths for research not allow for greater efficiency of individual researchers by reducing institutionalized and domain specific attention to certain channels of existing research trends by generating interdisciplinary translational hypothesis that would not have occurred without intervention. But may also lead to more robust idea generation which can accelerate scientific discovery.

My question is whether this symbiosis of AI and human knowledge will not only drive scientific research by enhancing existing scientists work but also creating more human scientists. As a discovery tuned education can see each student as a new idea generating science experiment, will this also generate a more numerous and broader range of humans who conduct scientific research?

[Jasmine97Huang]

Thank you for sharing this very exciting paper. Although it seemed intuitive that domain knowledge would greatly improve machine performances, the scale and perspectives of this paper is incredibly fascinating. Like @anuraag94, I am also curious about to what extent can the expert-assisted artificial intelligences facilitate development of sociological knowledge and understandings, which (arguably) rely more heavily on collective knowledge formation, and (also arguably) have different definition/scale of "discoveries" as natural sciences? Lastly, I am wondering if human expertise is already embedded into the content of the literatures, rendering the two models- one that accounts for researchers and one that doesn't- somewhat overlapping with each others? Why's the explicit inclusion of authorship have such an effect?

[bakerwho]

Thanks for sharing this paper with us - super exciting work! I'm interested in how the augmentation of the model with an intuitive yet previously ignored heuristic (physical proximity as likelihood of collaboration) makes such a marked difference.

This raises some interesting questions about what types of human intervention we need in the loop for machine discovery, or at a different scale, machine decision making. It almost seems like human beings can symbiotically spot machine blind spots, while being blind to what the machine might see in those blind spots (forgive the metaphor). What should be our strategy for operationalizing new intuitions via approximation?

[vinsonyz]

Hi Professor Evans, Thank you so much for sharing your work with us. My question is: what we can learn from your research and how we can apply your methodology in our study.

[Tanzi11]

Thank you for your presentation, Professor Evans. It is exciting that the data-driven AI models can suggest "alien" hypotheses without human intervention, but when you mention it is "promising," how feasible are they in terms of application?

[chiayunc]

Thank you for this fascinating paper. I see this work to be potentially a transformative application for the educational system. Basically, it would be a computational and systematic way for researchers to find unchartered territories, hacking the past to foresee a future. On the other hand, these models are also 'products' of our long-standing education, where education turns to knowledge and outputs knowlege. I am curious if you have come across anything particularly interesting in your findings. Did you see any unforeseeable predictions? do they have anything in common? do they show common limitations/ biases that reflect or could be linked to our current societal setup?

[Lynx-jr]

Hi Professor Evans! Thanks for this cool paper! I have a bit further thoughts related to the applications of this paper. In July 2019, a team from the University of Surrey filed two applications at the USPTO seeking patent protection for inventions autonomously developed by an AI system. The applications named the AI system as the inventor, while the USPTO has determined "the plain language of the patent laws as passed by the Congress and as interpreted by the courts" limits patent applications to only naming natural persons as inventors.

I personally agree that AI has vast potential (and am more determined from reading your paper), but such policies by the USPTO hinders was aimed to protect the rights of human inventors in some way, are you in favor of naming AI as inventors? How will a massive deployment of AI in science and engineering experiments shake the current patent application system and protection mechanism?

Again thanks for this enlightening paper! Although I will be participating asynchronously but still looking forward to your speech!

[lulululugagaga]

It's exciting to have Prof. Evans sharing his latest research with us! I really look forward to this speech. Thanks!

[MengChenC]

Professor Evans, thank you for sharing such an innovative research paper, it is really informative and inspiring. I have a question regarding the two components in your model, which balance between human availability and scientific plausibility. There is a mixing parameter β to control the behaviors of the AI model, while I am curious about how you decided the appropriate value of β for different research fields? And how can we employ test harness to decide β in such cases? Thank you.

[siruizhou]

Thank you for sharing! I'm interested in learning how you come up with this idea in the first place and if this alien AI philosophy can be used in other applications.

[ddlee19]

Excited to listen to your presentation, Dr. Evans. Thanks for your presentation.

[jsoll1]

Hi James! This seems to share a lot of overlap with current JDM research about the wisdom of small crowds. Have you considered implementing systems to account for shared information amongst a crowd of experts?

[tianyueniu]

Thank you so much for sharing your work Dr Evans! I look forward to your presentation.

[yutianlai]

Thank you Prof. Evans for sharing your latest research with us! I'm wondering how we can apply the research results in practice.

[ydeng117]

Thank you Professor Evans for presenting such an interesting paper! Correct me if I am wrong, the idea of using AI to avoid the crowd and generate "alien" hypotheses sounds similar to musicians using AIs to find blindspots for music innovations. It makes a lot of sense for artists to use AI to help them to find things that are never created before. However, it seems somewhat counterintuitive that scientists apply the same or similar strategy. How can we ensure the validity of those "alien" hypotheses produced by AI?

[ChivLiu]

Thank you for your presentation, Professor Evans! It is very interesting research! I understand that many AI algorithms such as the recommendation algorithms took personal information from millions of users to train. Would you think of this as an informatic "alien attack" from companies that we do not know who they are?

[linghui-wu]

Hi Professor Evans, thank you for bringing us such exciting work! I really enjoy how the multistep transition probability methods are incorporated into the study.

[ginxzheng]

Thank you for bringing this interesting paper Professor Evans! I think I would like to hear more about how you design the biased random walk algorithm to predict authorships. Thanks!

[qishenfu1]

Hi Prof. Evans, thank you very much for sharing your work! In your paper, you mentioned a lot of wonderful predictions about human advances in medicine and science. I am curious that do these models also accurately predict the advances in humanities, literature, social sciences, etc.? Thank you!

[Anqi-Zhou]

Thanks so much for your sharing, Prof. Evans! The work is so exciting! Why would the AI model predict so accurate towards the hypothesis or the expertness? What's the essential difference between AI models and the common used machine learning models? It seems AI model can be widely used, what exact fields or circumstances would you recommend us to use AI models?

[yierrr]

Thanks so much for the research, Prof Evans! I’m also curious about the application of this research in real life. Thank you!

[a-bosko]

Hi Professor Evans,

Thank you for presenting in this week’s workshop! It was really interesting to read about how to improve AI prediction of future human discoveries and interventions. It was very interesting to learn about the promising “alien” hypotheses.

In the discussion, it is mentioned that the approach used can also be adapted to identify biases that limit productive exploration and improve human prediction by focusing on discovery. How can we apply these concepts to our education system, in order to promote productive exploration?

Thank you, and I look forward to your presentation!

[chrismaurice0]

I am excited to hear you speak on your research Professor Evans! Specifically, I am interested in hearing your perspective on the role ethics/ ethical decision making should take as AI takes a more prominent role in research. As future academics and researchers, should we sacrifice ethics for technological advancement or should ethics always be at the forefront of our methods?

[goldengua]

Thanks for your research! I have the same question as anuraag94.

[hesongrun]

Hi Professor Evans, Thank you so much for sharing your work with us. How do you think we can apply this methodology in economics research?

[bowen-w-zheng]

Hi Professor Evans, thank you for sharing this very interesting work. You mentioned that the AI system generates novel hypotheses that are usually outside the scope of human-generated ones. I am wondering if there is an ethical concern related to this project. I think some places in the hypothesis space are left unexplored because of collective ethical concerns, and the novelty of the AI-generated hypotheses might partially come from exploring those spaces. Fascinating work, thanks!

[TwoCentimetre]

Hi professor, does that mean this research can somewhat predict human's thoughts? I mean since you can predict the next step of discoveries, you must be able to peek into people's thoughts.

[wanitchayap]

Hi Prof. Evans, thank you for sharing your work! Could you talk more about ethical implications of this research?

[Qiuyu-Li]

Thank you for bringing us this wonderful presentation! I’m very interested in this topic and looking forward to hearing from you about your views towards “alien” findings. I’m curious about to what extends you trust the complicated computational models, or just see it as an expedient for lack of data and other human inspiration? Thank you!

[YuxinNg]

Thank you for sharing your work! It is an interesting topic. Like @wanitchayap, I also hope that you could talk more about the ethical implication. And, I am hoping that you could provide more real cases. Thank you!

[adarshmathew]

I've been looking forward to this paper for a while! Primarily because I was curious about your exploration process and the use of hypergraphs. I have two questions, both motivated by my own research interests:

1. What would the data collection & processing tasks look like if I were to try and adopt this approach to understand the exploration-vs-exploitation trade-off in, say, movie or plot scripts? Because your method is an advance in understanding how the creation of derivative works can be automated to a high degree. (Movie scripts is a toy example, I want to sic this on op-eds.)
2. Would it be accurate to say that executing a Markov process over a hypergraph is a fuzzier version of representing Knowledge Graphs? You are not restrained by the the geometry of nodes and edges, replacing them with broad sets, within with elements have some kind of unspecified relationship.

Thank you!

[ghost]

Could you please explain alien artificial intelligence in more detail?

[WMhYang]

Hi Prof. Evans, thank you very much for sharing your work again regarding the science of science! I was interested in the concept of human contribution, but kind of curious whether we could model it as a unique set of influence instead of incorporating it in a hypergraph combined with the topics or key words of the papers. I am looking forward to your talk tomorrow!

[skanthan95]

Looking forward to your presentation, Dr. Evans! i wonder if this method could be used to predict what unsolved past events might have been.

[Yilun0221]

Thanks for the presentation and look forward to tomorrow's talk! My concern is that whether fields with objective criteria are easier to apply AI?

[YaoYao121]

Hi, Prof.Evans, thank you for bringing such an interesting and innovative research! Because I am not very good at theoretical aspects of AI techniques, I am more curious about the impact of such an application on human society. I think if now the application of AI remains in the step that "AI hypotheses are designed to substitute for human experts, failing to complement them for punctuated collective advance", this might not be problematic. If AI not only could substitue for human experts, but only could play an important role in pushing human advance, what else remaining for our human beings to do, please? No offense, thanks once again, this really is a fantastic topic!

[FranciscoRMendes]

Hi Professor - I look forward to your presentation! I wonder if we can use this to look backwards to some extent.

[MegicLF]

Thanks for sharing, Prof. Evans! I’m curious about what limitations the complicated computational models you used may have, and how you expect to improve the models.

[sabinahartnett]

Excited to hear your presentation Professor Evans! I would like to reiterate a lot of the previously posted questions so I'll like those and just ask a simple practical one that I couldn't help but wonder while reading:

How might the implementation of these alien artificial intelligences change the way research authorship and citation (and the whole research ecosystem that depends so heavily on it) are determined? Does the author of an algorithm which suggests a research path get foundational authorship in that research? How does this change our understanding of ingenuity? (similar to @mikepackard415 s question regarding a 'combinatoric view of innovation')

[yiq029]

Thank you for your fascinating paper. I am also curious about the ethical aspect of AI and to which extent can AI be used in today's studies.

[yongfeilu]

Thank you very much for your presentation! My puzzle is how you can apply AI to forge social science theories. To which degree do you think these ML methods work?

[william-wei-zhu]

Hi Professor Evans, thank you very much for sharing your work. I find the second finding of your paper to be really interesting. It reminds of AlphaGo vs Lee Sedol: Because of reinforcement learning, a computer can play some brilliant move that no human player could ever thought of. My question is about interpretability: once AI makes an alien discovery, can domain experts always interpret the rationale behind the discovery?

Looking forward to the talk!

[anqi-hu]

Thank you for sharing your work with us. What might be some of the ethical concerns with this AI awareness of human expertise?

[egemenpamukcu]

Hi Professor, I would really like to hear some more about your thought process on coming up with these creative methods to answer the question you are interested in. I felt like, in a way, combination and comparison of several methods somehow made the interpretation of your findings much more clearer. I was wondering if you have a thought template or a framework for coming up with the optimal and most suitable research desig for the problem at hand.

[YijingZhang-98]

Thanks for your sharing! I am so impressed by the fascinating topic, as well as the graphs used in the paper. Those graphs help me to grab the main idea quickly. So I have a non-academic question, that is what tools can be used to make such an amazing graph like Fig. 7? Thanks!

[Dxu1]

Thank you for sharing this exciting paper! I am interested in hearing your thoughts on how AI could help not only scientists, but also potentially policy makers or people outside research by providing "alien" perspectives.