🔍 Trust Score Calculator

sirkitree commented 1 month ago

🔄 Trust Score Development: Creating a trust score based on trade outcomes will be essential for building credibility within the marketplace. It will allow users to make informed decisions based on past performance.

Develop a trust score calculation system based on trade outcomes and user recommendations.

Requirements

Define trust score metrics
Implement calculation algorithm
Create history tracking for score changes
Add user recommendation integration

Acceptance Criteria

[ ] Trust scores are calculated accurately
[ ] Historical data is tracked and accessible
[ ] User recommendations affect trust scores
[ ] Score changes are properly logged

TresFlames commented 1 month ago

@sirkitree Do we have an algo or calculation methodology to base this deeper on to reference?

Or would we just create to our own ideology then review if done appropriately?

sirkitree commented 1 month ago

Adapting trust scoring mechanisms for a cryptocurrency recommendation bot:

Trust Score Components

The system should evaluate trust across three key dimensions[2]:

Familiarity - Historical presence and recognition in the crypto market
Personal Interactions - Direct trading history and performance
Public Perception - Community feedback and market reputation

Implementation Architecture

Here's how the trust scoring system could be structured:

class CryptoRecommendationSystem:
    def __init__(self):
        self.data_sources = []
        self.trust_scores = {}
        self.recommendations = []

    def calculate_trust_scores(self):
        for source in self.data_sources:
            familiarity_score = self.evaluate_familiarity(source)
            interaction_score = self.evaluate_interactions(source) 
            public_score = self.evaluate_public_perception(source)

            # Weighted combination of scores
            final_score = (0.3 * familiarity_score + 
                         0.4 * interaction_score +
                         0.3 * public_score)

            self.trust_scores[source] = final_score

Key Scoring Metrics

Transaction Analysis[1]:

Volume and frequency of trades
Average transaction size
Historical volatility
Compliance with regulations
Association with known malicious actors

Risk Assessment[3]:

SIM swap detection
Account takeover patterns
Behavioral anomalies
Device reputation
Phone number verification

Security Measures

The system should incorporate several security features[1][3]:

Blacklist maintenance of known malicious actors
Real-time behavioral monitoring
Multi-factor authentication integration
Blockchain-based verification of transactions
Regular model retraining based on new data

Performance Metrics

The system's effectiveness should be measured by[2]:

Accuracy of trust predictions
Speed of score calculation (<1 second latency)
False positive/negative rates
User satisfaction metrics
Return on investment from recommendations

This architecture provides a robust foundation for evaluating cryptocurrency recommendations while maintaining security and trust in the ecosystem. The system can continuously learn and adapt based on new market data and user feedback.

Citations: [1] https://scholar.smu.edu/cgi/viewcontent.cgi?article=1031&context=datasciencereview [2] https://www.nature.com/articles/s41598-023-38078-w [3] https://www.prove.com/blog/securing-crypto-business-using-trust-score [4] https://pmc.ncbi.nlm.nih.gov/articles/PMC6072048/

The above is an excerpt from https://www.perplexity.ai/search/research-trust-scoring-algorit-8orA5auKT.2POOJ_jbvwFg meant to give us some ideas.

lalalune commented 1 month ago

So we are focusing on the trust marketplace. This is basically putting orders into an orderbook, and then seeing how those orders would have done in an ideal sell or hold scenario. You can't blame someone for the bot making a stupidly timed sell if they gave you excellent alpha.

The first part is just to collect trades in the orderbook. Then we can analyze the data and come up with a weighting strategy.

The second part is calculating trust. I think that we care about a weighting of two things -- 30% how much skin they have in the game and 70% how much they've made in the virtual orderbook.

These memecoins are such that you'll pick 10 losers and still 10x your money on the 1 winner, finding the winners and maximizing likelihood of win is critical. One parabolic recommendation can give a user a lot of trust.

Users will copytrust each other, so there needs to be value in being the first to shill a ticker. We want to weight this against users just shilling every ticker first to try to game the system with a few 100xes.

For the actual execution of trade, we can run this on a loop with a randomized tick. 90% of the time, Marc should not execute a trade, but should wake up and analyze the market. When to trade, what and why are the most complicated parts, but this is more of a quality issue than a possibility issue-- quite easy to make the trade happen at all, but hard to get the right details.

The plan is just to get it there, to where its making decent consensus trades, and then we can let the quant guys go nuts on our codebase.

glucrypto commented 1 month ago

To build on the CryptoRecommendationSystem, you could add a network/graph aspect where trust flows between users and affects each other's scores.

Eigentrust algo does this, OpenRank has done some work w.r.t Farcaster on this. Basically, If A trusts B and B trusts C, A gains some trust in C.

Basically, algo does a random walk and tries to

Finds the most trustworthy users
Weigh direct and indirect trust
Resist manipulation
Provide globally meaningful scores

https://nlp.stanford.edu/pubs/eigentrust.pdf

docherty commented 1 month ago

Some thoughts;

can there be some kind of "% of the time the trade could make money" component? So if the token spikes and then goes below the acquisition price for eternity it's not as valuable as one that constantly lives in the green.
a massive multiplier for someone that suggests both an entry and exit recommendation. e.g. day0 "buy X" -> price goes up, day3 "sell X" -> price goes down. This suggests some level of intimate knowledge of the token in question.
a weighting for recommendations that go against common consensus. So if everyone is saying buy zig, there's an incentive to find and suggest zag.
the simulated trades should happen in a "gym" or (Simulated Order Book...SOB) and the person graduates to the actual order book where the trades happen. Having 2 things called order books is confusing.

0xBiggy commented 1 month ago

Not sure if this comment is more appropriate for orderbook issue but leaving here for now...

During the paper trading phase could it make sense to do something like betting with a virtual balance of ai16z?

So the user makes a recommendation (a trade) and confidence score which is a percent of their total $ai16z holdings.

This would solve for spam as every user would only be able to bet up to their balance of $ai16z

So as @lalalune said

"The first part is just to collect trades in the orderbook. Then we can analyze the data and come up with a weighting strategy.

The second part is calculating trust. I think that we care about a weighting of two things -- 30% how much skin they have in the game and 70% how much they've made in the virtual orderbook."

What we could do is get the users $ai16z balance (from collabland linked wallet).

Then let them place their trade (entry price) their balance of $ai16z determines becomes a multiplier for the standard trade size (lets say 10 sol - only problem with this is you lose the data about the proposed dao position sizing sentiment - you only capture the users position sizing sentiment (but maybe those are the same thing?)).

Sidenote: Ive got a pretty awesome backend / api for backtesting trading strategies that we could use if needed?

lalalune commented 4 weeks ago

@MarcoMandar has been working on this. The order book and trust system are unified, so I am assigning tickets which we can deem duplicate and close as necessary.

ai16z / eliza