DP 94: Aristotle0x - Decentralized Peer Journal

[ghost]

Name - Aristotle0x

Purpose

Currently the peer reviewed journal process is fraught with inequality and inefficiency due to fraudulent paper submissions, perverse incentives, and expensive paywalls barring access to newly published papers. This proposal is designed to solve these and many other issues with academic, scientific, government, and professional journals in order to ensure incentives are in-line for each party, improve the content of published research, and ensure instant access to newly completed research without needing to pay for a subscription. These efforts would help drive future scientific progress by reducing cost and incentivising quality submissions.

Description

The traditional peer review process is designed to be a form of self-regulation where one or more people of similar competence evaluate the work of the producer i.e. author(s). By having those qualified in the profession able to provide expert feedback to the author(s), a feedback loop is created that allows the author(s) to refine their work. In turn, this assures that the final work maintains a standard of quality, improves the performance of the author(s), and provides credibility from trusted names in the field.

In the current environment this process is controlled by many separate journals that focus on different specialties across the spectrum of fields. This process requires coordination of the journal to ensure proper referees are selected and then an editor ensures the works selected for publication. The reward for those doing the review is minimal (typically just a resume booster) and the editor of the journal is typically working to ensure the reviewers are a match for the topic and able to do quality work. After publication the journal charges universities and individuals exorbitant fees for access to the works that help fuel innovation. These high fees are charged even after public and/or government funding was used to do the research in the first place. This often leads author(s) to host a second version of their paper on a personal site ensuring their work is accessible to others who can’t afford to pay the journal’s fees.

Often, funding decisions are partially based off of how many papers a researcher has published. This has resulted in fraudulent and randomly generated papers designed to subvert this process by passing themselves off as real articles. This dilutes the quality of the research being done and diverts valuable resources away from legitimate endeavors instead of trying to pad stats or generate revenue.

This project aims to break down these barriers by allowing open access to the research created by the author(s) while simultaneously rewarding quality content with monetary rewards as well as a reputation score based on activities. The platform’s submission and review process will be transparent, allowing others to review previous correspondence and interactions with the author(s) and reviewer(s). Along with these features, it would be paramount to ensure that incentives are properly aligned to increase research productivity by reducing overhead of the review process, reducing the cost of referencing other works, reducing fraudulent activities, and rewarding those that do the best work.

How it would work

At a high level, this project can be thought of as a hybrid social platform (DApp) based on reputation combined with a typical journal model. Each user (author) would need an account that can be attributed to their work(s). If there were multiple authors on the submission they too could be tied into the submission. After submission of the research, the platform would identify those that are deemed "experts" in the field and randomize reviewers (referees). Once the referees have been selected they will have a predefined number of days (this will be configurable) to submit suggestions and revisions, at which point the author will then be given a specified number of days to make modifications. After a reviewer and author are in agreement that the work is completed to satisfaction, both parties must sign off to submit the final version to the platform. At this stage all reviewers have agreed with the author and the paper can then be considered "publishable" and available to anyone who wants access to it.

To lower the barrier to entry of this platform it will be vital to ensure author(s) can easily interact with the platform. Ideally this would include a method of account creation and account funding outside of exchanges by providing an interface as easy to work with as Google or Amazon (the holy grail of DApp adoption). In general, those submitting to this platform would not be the most technically savvy since their area of focus is unlikely to be the use of computers (our differences make us better).

To help reduce fraud, the author will be required to make a non-trivial escrow payment, held by the platform (blockchain) to indicated the author’s intent on going through with the entire process. Once specific goals or thresholds are met this initial escrow can be distributed to the referees once their work is complete to provide incentive (payment) for their part of the process. When a paper completes the review process, the author(s) would eventually get a reward and would also be entitled to any proceeds the paper may garner (tips/donations/web traffic).

The platform would also build a reputation system for those involved in the process by being able to score each account based on success of each "published" paper. Those who submit content and those who review/referee content would all improve their reputation based upon the success of the papers they submit or review. Once a paper is "published" it can be released as an article on the platform or even grouped into a monthly publication that the journal produces for easier readability.

Topic Overview (In Progress - Whitepaper in Progress)

• Problems Solved
  • Transparent Review Process
  • See who the reviewer's are
  • See reviewer(s) history and rating
  • See reviewer(s) area(s) of expertise
  • Reward author(s) and reviewer(s)
  • Highly rated and highly referenced submissions will increase personal reputation and potential payouts.
  • Open Access to each Paper (ideally stored on the blockchain)
  • Open to anyone who wants to reference it for free
  • Remove paywall, especially in the medical field where most grants are taxpayer funded.
  • Open paper submission to anyone (lower barrier to entry)
  • Encourage reproducing work presented in other papers
  • This ensures results previously published are valid and can be reproduced in another (similar) setting (or can't be reproduced)
  • Encourage publication of expected or negative results
  • Research that has an expected or negative result is typically not published since it is not deemed important. This platform would allow and encourage these findings with reputation and reference rewards.
  • Reduce submission of Spam
  • Escrow payment possible to signal intent to publish
  • Utilize Knowledge of the crowds
  • District0x users can help determine usefulness and popularity while reviewers will be expected to maintain the quality of submissions.
• Journal Process
  • Editor(s)
  • Governing Body (District0x or experts)?
  • Publish election and validation
• Areas of Peer Review Focus
  • Professional
  • Potential issues may arise here if the topic is not allowed to be shared publicly
  • Scholarly/Research
  • Government
  • Medical/Clinical
  • General
  • An open area for those looking for review of their paper that are not necessarily looking to be published
• Publishing Process
  • Author(s) submit proposal
  • Escrow payment to help to avoid/reduce spam
    • Should a governing body or District0x token holders be able to vote on papers that may need escrow help due to its importance?
  • Reviewers Selected
  • Initially, this will be a large hurdle to overcome to ensure reviewee's are qualified in the field they are covering.
  • An editor or experts in the field (perhaps the author(s)) could recommend potential reviewer(s)
  • Review Process
  • Checkpoints will be required to ensure that expectations and timelines are met (the time can be dynamic and enforced by a smart contract)
  • Should be transparent with critique and/or suggestions provided when possible
  • Publish Phase
  • Once a predetermined number (3?) of reviewers accepts the paper it can be deemed as "published" on the platform
  • At this phase all entities will start generating/gaining reputation scores
  • Payout
  • The initial escrow will be distributed between the reviewers and author(s) (This amount will be variable and dependant on the decision of requiring an escrow to start the review process)
  • The payout will be handled by a smart contract on the blockchain and that will close out the publish process for the paper submitted
• User Experience and Visualization - i.e. Front End
  • Streamline experience required
  • Account creation and management
  • Deposit/Withdrawal needs to be seamless for the author(s)
  • Explore potential integration with the Open Science Framework
  • Open source front end for presenting and managing the publication process
• Data Storage and System Logic/Functionality - i.e. Back End
  • Infrastructure
  • Utilize a blockchain (Ethereum) to store papers, entity, and reputation data (Augur/d0xINFRA)
  • Architecture
  • Each paper, author, reviewer, etc. needs to be identified and a "profile" created around them.
  • Each entity will have an activity and reputation score associated to them.
  • Each paper needs to have bibliography/references linkable with reputation score
    • This should help ensure references are reputable and the paper itself holds higher value
• Reputation (TO DO)
• Potential Revenue Streams
  • Publish a "best of quarter" journal or journals with a specific topic (Example: AI research)
  • While all research will be accessible for free a quarterly journal could be published that features the best articles or the most impactful.
  • District0x token holders could help vote for the best articles on the platform
  • This could be sent to those willing to subscribe to the quarterly digest as well as funding via selling ads in the digest.
  • Philanthropy and/or donations
  • It is not unusual for research to be funded by donations and this could be extended to the journal process
  • Ad based revenue (not preferred)
  • The web platform that serves the papers could have select ad's

PS: Please vote by thumbs up if you like the proposal

[Alexand0x]

Excellent proposal. It'd be a long road to disintermediate academic peer reviews as sources of truth, but I think it's a worthwhile venture. How would you envision this district looks? What would users do? What would tokenholders do?

Also, please DM me @alexander on our slack to discuss this if you get the chance.

[Alexand0x]

This needs a better name - something more titular and less descriptive!

[Andersmholmgren]

Super excited to see this. I've been thinking for a while that we need to get science on the blockchain. Hard to think of anything more valuable to humanity and more important to be owned by all.

Some things I think would be nice to see:

1. Tackle the tendency to only publish positive results. Some sort of pre registration of research that would be referenced when publishing, coupled with reputational penalties for not publishing pre registered research might work

2. Serve as a highly referenceable source for scientific knowledge. Would be great if one day wikipedia, lunyr etc heavily referenced this information.

The last point is a bit nebulous and I'm not sure if this is the right place for it but here is what I'm thinking.

Could this include some structured data that captures relationships between this research and other research and to scientific concepts? Could it help some future dapp / search engine to answer questions like:

1. Does XYZ (e.g. some aspect of climate change, or homeopathy) have scientific support and if so how strong? Some measure of peer reviewed journals coupled with corresponding reputations that support or oppose a concept.

If we could achieve something like this it may help fight pseudoscience and alternative facts. Who knows maybe help us get to a Post Post Truth world ;-)

[oso-network]

Interesting. Sine few months, we are also working on a similar project called Open Science Organization (OSO) https://github.com/open-science-org to create an open and democratic scientific ecosystem. We have pointed out two main pain points in the current scientific research ecosystem: funding and publication. We have realized that, solving the funding problem is much harder and will take some time until people will fully adopt blockchain technology and is more tricky due to it's cross-border, cross-organization nature. It is wiser to solve the publication problem first. We are also considering using either district0x (creating a district for open and paid peer-review and free publication access) or Aragon protocol. The most interesting part is that this proposal was made at the same day when we made our white paper public https://github.com/open-science-org/wiki/blob/master/OSO_white_paper.pdf on Aug 15. We are really happy that other people are also working on a very important problem.

[longpd]

Cliff Lynch and I talked about something this last year. He was very supportive of the idea. It would be nice to coordinate these various efforts to get the (open-science.org, alexand0x, etc.), and reach out to others who might have something to contribute, e.g., the Center for Open Science @ UVA.

Great to see the proposal and you're on the right track! If I can help pleas ket me know.

Phil Long, CiNO & Assoc. Provost University of Texas at Austin phil.long@austin.utexas.edu

[oso-network]

Yes, the like-minded forces should definitely collaborate! I am happy that the mind share dedicated on solving this problem is increasing.

[ghost]

Wow, very excited to see all the feedback! I will expand this proposal with how the district will function and what it is hoping to achieve shortly.

@Alexand0x Agreed, would be a long road for mass adoption but if other projects mentioned in the comments above could band together and share a common goal it would be interesting to see what could be accomplished. Certainly open to a name change, have been brainstorming a bit and think Aristotle0x is a good name, certainly open to ideas/suggestions.

@open-science-org This is awesome! Didn't realize there was another project working on the same topic as I had been looking for something in this space since there are clearly gaps in the current ecosystem and wanted to get the idea out there once I found out about District0x, glad I did! Just read your whitepaper and you have a much more formal/robust proposal and I'm impressed with the level of detail put into this. Would love to see some collaboration on this topic and am encouraged to see many others interested in tackling this problem!

@Andersmholmgren Very good points, I think another bullet point to add to that list would be to encourage papers that reproduce others work and provide a rewards/reputation mechanism that encourages this and is transparent to everyone involved (similar to @open-science-org proposal of IPIS).

[ghost]

@longpd Curious to hear what you and Cliff Lynch identified as the core issues to solve? It would certainly be exciting to see a joint effort with the Center for Open Science and the Open Science Framework as a possible front end solution to complement back end framework the blockchain provides.

[oso-network]

@basiccrypto I was also pleasantly surprised to see a similar project. Thanks for initiating this district. We should certainly collaborate!

My understanding is that the problem of funding is significantly more challenging than the problem of publication. As I said above, the funding problem is tricky because of it's cross-border, cross-organization nature and associated business interests in case of private funding. It will take a good amount of time before all the parties can adopt the new decentralized tech. Due to this reason, I think we should first tackle the publication problem. If we can create a successful decentralized model for scientific peer-review and publishing, it will help us to solve the funding problem too because we can make a persuasive case out of it.

To create a decentralized publishing model (peer review + publication), creating a district in district0x platform seems like an easier start. However, we are worried that the limitation of d0xINFRA framework might constrain the publishing model. This reddit post of mine is on this topic https://www.reddit.com/r/district0x/comments/6uu0wx/a_district_in_district0x_vs_a_dao_based_on_aragon/

What I am thinking is to propose multiple decentralized publishing models and then the pick the best ( socially, technically, and economically) by voting. district0x platform is a perfect fit for that.

-Gajen

[Marianodp]

very nice proposal

[ghost]

@oso-network have you joined the Slack channel? My username is the same as here, otherwise I will email you.

A discussion around the decentralized publishing models would be a good one. I am assuming this means the blockchain technologies involved as well as the methodology/mechanics of the publishing model.

Your Reddit post asks how a district created on District0x is different from a DAO created on Aragon and the answers in the thread seem pretty good, i.e. similar to Word Press but for the blockchain. Since a district inherits the development stack of Ethereum and Aragon it will certainly allow for a much quicker deployment (as you point out). I believe District0x is the proper place for this project due to the fact that they allow for customization with its modular design. The "auxiliary modules" section of the District0x white paper tends to indicate that advanced customization is achievable (something that would have to be done regardless of where its deployed) and 3rd parties may also contribute to this process and potentially charge for using the module. Should probably reach out to an engineer or extended team for clarification on the possibilities that can be deployed.

Perhaps a bigger concern about the District0x model is the fact that most (if not all) publishers and reviewers will not own any DNT tokens initially (could be a problem for many projects). So getting them to vote on district issues will be tough since they will be the ones using the platform as their backbone for sharing thought.

[oso-network]

@basiccrypto I agree with most of your analysis. I just sent you a DM in slack. Looking further to discuss more about decentralized publishing models.

[KadeMorton]

Just chiming in, I've also been working on a decentralised publishing platform for research with a small team of volunteers called Aletheia, you can check us out at https://github.com/aletheia-foundation

[ghost]

We are pleased to partner with Open Science Org to begin working on the decentralized peer review topic. Please follow and comment at the following locations.

Website: http://oso.network GitHub: https://github.com/open-science-org Whitepaper: https://github.com/open-science-org/wiki

[Bradymck]

@basiccrypto Oh very cool to see! I'm interested to see IPIS get started. Almost sounds like the ideas in proposal number #11 could be a good collaboration effort. I will comment on their proposal as well so they have visibility on yours.

[uvizhe]

@basiccrypto, hello from #11! Here are some thoughts regarding funding and our ideas synergy.

What seems to me as one of the funding problems here is that publication escrow is composed from participant's funds. So it's like a zero-sum game and thus no good. So to amend it I suggest the same mechanics as I proposed for #11. Add a community of people interested in a research in general and let them vote with money (tokens).

Further developing this idea I propose two things:

• Since non-academic people may lack of a particular knowledge they can vote less rational then in case when they are aware of science fields of research and consequences of studies. Therefore there should be some kind of guidance for them. Maybe a periodical discussing possible studies and their impact on humanity.
• There should be possible for non-academic people to propose new studies. Some people just out of academia for some reasons but are aware of what's going on. With the help of professionals they possibly can initiate a perfect research backing it with good amounts of money (since it's their own idea).
  You can even think of outsourcing research labs for small companies.

As @Bradymck mentioned this proposal shares some common ground with my #11. Specifically they both may use the same modules for handling user submissions and voting/funding workflow. So I'm ready to participate.

@oso-network can you DM me on Slack please? Same username as here.

[Andersmholmgren]

FYI came across another effort http://www.blockchainforscience.com/ the other day

[oso-network]

@uvizhe @Bradymck Please accept my apologies for my very late reply. I am more readily available via email. Please feel free to reach me at gjk0112358@gmail.com.

@uvizhe You raised a very important point on why an ill-informed vote can be detrimental. When we are aggregating the votes from the members, we are basically performing a weighted average of them. From the perspective of 'ensemble learning' concept from machine learning, the decision of a member classifier has to be weighted properly to reflect its accuracy; see https://en.wikipedia.org/wiki/Ensemble_learning. Taking this analogy to voting in a particular domain, the vote of an expert member will have more weight than the vote of an average member (assuming that the expert members tend to make better decisions in that domain). I have formally laid out these concepts in my original white paper where I talk about expert value (e-value), voting value (v-value), and voting function (v-function); see sections 3.2 and 3.3 at https://github.com/open-science-org/wiki/blob/master/OSO_white_paper.pdf.

We are actively increasing our community; see http://www.oso.network/#team and we are very eager to collaborate with like-minded folks who believe in true democratic society in general and "open, democratic, and efficient scientific ecosystem" in particular. I think DP-11: Idea sharing and crowdsourcing hub https://github.com/district0x/district-proposals/issues/11 and Interplanetary Idea System (IPIS) https://github.com/open-science-org/IPIS of the OSO project have lots of overlap and we can certainly collaborate.

Please feel free to reach me at gjk0112358@gmail.com. @himalayajung

[fengkiej]

Hey, I just stumbled upon the similar concept to this last few days. But it is kind of different in several aspects. Tell me what you guys thought on my concept. There are two tokens to be used in the district: DNT and journalToken (can be renamed to anything)

1. DNT can be staked to the district, it serves two purposes: to govern the district and to earn or mint journalToken.

2. New journalToken will be minted by hosting paper in IPFS. This will involve a smart contract to manage token issuance, how the file are stored, and file redundancy. Only DNT stakers can host files.

3. Reviewer will be incentivized in journalToken, they must submit a review paper and a verdict (pass / not pass), they will be competing to be in top 3 and each incentivized proportionally. DNT stakes will have right to vote which peer review are deemed are winning the competition.

4. Submission of paper will need journalToken. This works like Ethereum gas, larger the journalToken paid, faster it will be reviewed. More are competing (more review to the paper) the higher paper's rating in the district.

5. The journalToken will be distributed among top 3 reviewer proportionally (first place > second place > third place) and the voters who voted winning reviewer.

6. (optional) journalToken will be paid to access the journal.

[uvizhe]

FYI here is another project of the kind I stumbled upon today: https://orvium.io/

[Bradymck]

Hey thanks for the share @uvizhe. I guess this project is more so geared towards the scientific research community though I do think the model could be fairly generalized for any content.

[himalayajung]

Thank you @uvizhe and @fengkiej . It is good to see other non-profit projects trying to improve scientific research using internet3 tech. Open Science Organization (OSO) is always open to collaboration and it would be great to synergize. Please feel free to reach out to us at contact@oso.network. This is our github page https://github.com/open-science-org

And yes, personally I tend to agree with @Bradymck

[aleebrahim]

I strongly support the proposal to change the dilemma in peer review process.

[Bradymck]

@aleebrahim I agree, this is a great idea. Have you voted for a proposal yet?

vote.district0x.io

[Bradymck]

Hey there,

I'm not sure if you're aware but we recently launched a new bounty to migrate your proposals to the actual District Registry: https://registry.district0x.io/

We're replacing the old voting app with the registry. Let me know if you need help but I would love to see you migrate this over so you can claim your DNT.

It does take a 10,000 DNT submit to submit your proposal but this gives you an extra 2000 you can stake in the registry beyond the deposit amount.

Hit me up on Telegram or Discord if you need help or have questions.

Telegram: https://t.me/district0x/75217 Discord: https://discord.gg/P9RQejv

PS, please excuse the canned response. I am encouraging everyone here to start migrating so they can claim their 12000 DNT.