arxiv-sanity lite: tag arxiv papers of interest get recommendations of similar papers in a nice UI using SVMs over tfidf feature vectors based on paper abstracts.
Here are 50 parameters that could be used for ranking and evaluating research papers in an AI peer review system:
Relevance to the field
Originality and novelty
Clarity of research objectives
Soundness of methodology
Rigor of experimental design
Quality of data collection
Appropriateness of sample size
Robustness of statistical analysis
Validity of conclusions drawn
Adequacy of literature review
Contribution to existing knowledge
Potential for real-world application
Societal impact and significance
Ethical considerations addressed
Reproducibility of results
Scalability of proposed solutions
Generalizability of findings
Interdisciplinary relevance
Clarity of writing and presentation
Logical flow and organization
Adherence to formatting guidelines
Grammar and linguistic quality
Appropriate use of figures and tables
Sufficient explanation of technical terms
Balanced and unbiased reporting
Acknowledgment of limitations
Discussion of future research directions
Practical implications discussed
Theoretical contributions made
Creativity and innovation
Depth of analysis
Breadth of scope
Attention to detail
Integration of multiple perspectives
Engagement with counterarguments
Persuasiveness of arguments
Coherence of narrative
Effective use of citations
Quality of sources cited
Appropriate level of technical detail
Accessibility to non-specialist readers
Potential for generating further research
Timeliness and relevance of topic
Alignment with journal scope and aims
Adherence to ethical research practices
Disclosure of conflicts of interest
Effectiveness of abstract in summarizing key points
Appropriateness of keywords chosen
Suitability for target audience
Overall impact and significance of the work
Based on this a user can filter for any particular parameter and AI system can give 1 to 10 rating 1 being lowest 10 being highest and we can add up for all 50 parameters and pick greatest papers .
Here we can use different models to rate these 50 parameters and then take average or may be weighed average considering number of model parameters and then select best research papers.
For this we can use API on the website like openai or local ollama type llm and then users can use their own llm to rank research papers.
Please implement this if possible if not make me a contributor and and I will make a pr to implement this. This is purely for the name of 🔭 science.
Here are 50 parameters that could be used for ranking and evaluating research papers in an AI peer review system:
Based on this a user can filter for any particular parameter and AI system can give 1 to 10 rating 1 being lowest 10 being highest and we can add up for all 50 parameters and pick greatest papers .
Here we can use different models to rate these 50 parameters and then take average or may be weighed average considering number of model parameters and then select best research papers.
For this we can use API on the website like openai or local ollama type llm and then users can use their own llm to rank research papers.
Please implement this if possible if not make me a contributor and and I will make a pr to implement this. This is purely for the name of 🔭 science.