Customizable altmetrics for study pages

[prototyperspective]

Altmetrics are useful scientometrics that go beyond measuring e.g. a study's inbound citations and impact factors.

After the Altmetric tile has been added to pages about studies (#1724) I suggest developing your/our own altmetrics which could be made so that it's configurable by the user to give different weights to things like, for example, number of mentions on Twitter, selected assessment of societal importance of topic/goal, advocacy by experts/institutions (on Twitter or elsewhere), number of views/reads, size of audience of study, assessment by experts (incl features in specific collections), number of news outlets features in general, number of features in large/reputable news outlets and various other potential metrics. It's not an authoritative metric of objective truth or anything similar but simply interesting relevant data one could choose to use / consider / evaluate further (or not).

• Making it transparently configurable by the user would also show how the score was calculated. Also Altmetric is missing a lot of data/mentions such as sometimes when a paper is linked via it's DOI rather the plain URL etc.
• The scores or parts of it (like number of news articles about it) could also be added to tables that show many studies such as the pages about topics.
• This may require creating new crawlers of social media sites (or relevant data-forwarding of existing crawlers and indexing). It doesn't necessarily need to crawl the live-websites but could use data-dumps and incremental dumps. It don't know how Altmetric gets all its data but researching this could help to create this.

It may also enable researchers to more easily locate comments and responses from peers and other scientists and researchers like via a number of and links to comments of sciencemediacentre.org, pubpeer, retractionwatch, reddit (including reddit posts of news articles about the study not just direct links to the study itself) & e.g. nondownvoted phys.org comments.
Note that very often such comments can be counterproductive or unconstructive. It may be useful to somehow scan for criticism/concerns and open questions (not mere discussion or expressions of approval/support/appreciation etc) as well as constructive criticism/valuable questions in specific if possible. I think this is best separated into a distinct issue.

This would basically be a long-term milestone with lots of tasks due to the need for crawlers, scrapers or other ways to get this data. It's not easy to build this. This issue is about building a very imperfect preliminary prototype of it using the most easily accessible data (probably some of the social media sites allow you to use an API to easily retrieve links to and metadata on posts that link to a study).

[prototyperspective]

Here is a short overview introduction to existing altmetrics. This is also relevant to #1724. The page "Altmetric.com Sources" has info about some of its limitations such as "Text mining mostly for articles with DOIs" and "Direct link to a scholarly output" as method for Source:Reddit (rather than also scanning for posts of news articles that link to the study or even that are about the study but don't link to it).

Once Scholia-custom, preferrably user-customizable, altmetrics have been built this could also be used for criteria for showing items on the Scholia Watchlist (#1733).

[prototyperspective]

Maybe this is useful for how this could be built.

Having the altmetrics score algorithms transparent and configurable is a large benefit over the current Altmetrics.com. For example, you could tweak and customize the scoring parameters to how you think the societal impact can best be evaluated. For this one could factor in various things that Altmetrics.com doesn't seem to consider (does it?) like in principle:

• number of tweets × number of likes of the tweets × number of reddit posts × number of reddit upvotes × number of reddit comments with a link to the study × same for news articles that are about the study = 20 % of the score
• positive citations and expert feedback = 22 % of the score
• pageviews = 1-10 % of the score
• etc (while currently many of these things like number tweet likes or reddit comments linking to the study or reddit posts linking to an associated news article are not considered and the algorithm is completely intransparent)

One could also look into how various datasets are created and the use the datasets as well as how they retrieve their data (updates) to monitor for links to studies and links to news reports about (linked) studies.
One example is the "TWikiL - The Twitter Wikipedia Link Dataset" which contains all Wikipedia links posted on Twitter between 2006 and 2021 (instead of Wikipedia links study links would be monitored like Altmetrics does and Wikipedia-links may be useful too). It seems to be using this crawler.