Integrate w/ Semantic Scholar TLDR, embeddings

[adelavega]

Semantic Scholar is a platform/api for semantic based browing of academic literature.

We should integrate w/ them.

One thing we can do that would be helpful is retrieve the "TLDR" for a paper (AI generated summary of a paper that is shorter than the abstract). This could help in sorting through relevant papers. We could also potentially prepopulate this in the database for all papers.

example:

In [17]: import requests
    ...: import json
    ...: 
    ...: # API endpoint URL
    ...: url = 'https://api.semanticscholar.org/graph/v1/paper/PMID:'
    ...: 
    ...: # Replace PUBMED_ID with the actual PubMed ID of the paper
    ...: pubmed_id = '36040302'
    ...: url = url + pubmed_id
    ...: 
    ...: url = url + '?fields=tldr'
    ...: 
    ...: # API request headers
    ...: headers = {'Accept': 'application/json'}
    ...: 
    ...: # Make the API request
    ...: response = requests.get(url, headers=headers)
    ...: 
    ...: # Parse the JSON response and extract the TLDR summary
    ...: data = json.loads(response.text)
    ...: tldr = data['tldr']
    ...: 
    ...: # Print the TLDR summary
    ...: print('TLDR summary: ', tldr)

TLDR summary:  {'model': 'tldr@v2.0.0', 'text': 'Neuroscout is presented, an end-to-end platform for analysis of naturalistic fMRI data designed to facilitate the adoption of robust and generalizable research practices and makes it possible to overcome modeling challenges commonly arising in naturalistic analysis and to easily scale analyses within and across datasets, democratizing generalizable fMRI research.'}

The field openAccessPdf also has a direct link to the open access PDF if it exists. So that might be a cool thing to add directly as a link.

[adelavega]

https://api.semanticscholar.org/api-docs/graph#tag/Paper-Data

[adelavega]

Semantic Scholar also exposes embeddings for each paper from this mode: https://github.com/allenai/specter

This could help w/ some automated meta-analysis tasks.