search

Tribler / tribler

Privacy enhanced BitTorrent client with P2P content discovery
https://www.tribler.org
GNU General Public License v3.0
4.77k stars 443 forks source link

Bitcoin-Intelligent-Life - Blockchain engineering class project #4039

Closed synctext closed 2 years ago

synctext commented 5 years ago

Project : Bitcoin-Intelligent-Life

Self-replicating Bitcoin-based entities using deep re-enforcement learning

You will create a key prototype that advances the state-of-the-art of self-replicating software, autonomy, and artificial intelligence. Your mission is not to terminate all human life.

image

Bitcoin is an essential technology for digital life: you can buy servers. In prior work TUDelft has created CloudOmate. With CloudOmate you can buy servers with Bitcoin automatically. CloudOmate gives any computer the power to own money, replicate freely, no human can access that server, and no human can take that money away. It becomes an autonomous virtual entity. With cloudomate can even enhance it's own privacy and buy VPN networking autonomously.

Next step is adding intelligence and learning to this autonomous virtual entity. You will combine buying servers with Bitcoins with the another peak-hype technology of today: AI. You will create a basic intelligent entity, capable of self-replication and learning capability. For the learning side you can use deep re-enforcement learning engine in Python. See basic tutorial

Possible sprints for the 10 weeks:

Outcome: running code and Github readme.md (no thick final report that nobody really reads).

Warning: this is a scientifically most challenging assignment (recommended for 'Cum Laude' level students) Supervised by: Dr. Johan Pouwelse

synctext commented 5 years ago

Kick-off meeting:

devos50 commented 5 years ago

Please note that the 2017 plebnet project has been continued by another group this year. See the last few posts of this ticket: https://github.com/Tribler/tribler/issues/2925

Report: https://repository.tudelft.nl/islandora/object/uuid%3A614efc19-e6a9-453f-8084-2e2c54ddd93f

They made this visualization: 42031541-16e66e28-7ac6-11e8-8ddd-c2db87de599f

synctext commented 5 years ago

Meeting minutes:

Next sprint ideas:

andrebreis commented 5 years ago

We are using this Plebnet repository

rwblokzijl commented 5 years ago

Timeline of the PlebNet and Cloudomate Projects

This post serves to give an overview of the history of the PlebNet and Cloudomate projects. It aims to give a high level overview of the project groups that have worked on the project and their contributions. 5 teams have worked on this project before we (team 6) got started.

Team 1: TENNET (Skynet 2.0)

This is the inception of the project. The project started in april 2016 as the bachelor end project of:

In this first project a miminum working project was created. Skynet 2.0 added:

After this project there was not working version yet as money making was not properly implemented.

Documentation:

Team 2: PlebNet

The PlebNet project started in May 2017 as the bachelor end project of:

This project improved on Skynet 2.0 by:

Documentation:

Note that Team 3 and 4 worked parallel.

The project improved upon Cloudomate by:

The project improved upon PlebNet by:

Documentation:

Team 4: Growing primitive blockchain life

Starting November 2017 as well, the PlebNet Project was improved upon by:

Note that Team 3 and 4 worked parallel.

The project improved upon Cloudomate by:

The project improved upon PlebNet by:

Documentation:

Team 5:

Starting April 2018 the PlebNet project was improved upon as the bachelor end project of:

The project improved upon Cloudomate by:

The project improved upon PlebNet by:

Documentation:

Team 6:

Starting November 2018 the Plebnet project is being improved by:

Documentation:

synctext commented 5 years ago

Impressive overview! 25 developers...

synctext commented 5 years ago

Meeting minutes. Current status: simple learning and reproduction operational. Created wealth of 20GBytes in 2 days and creating market orderbook entries.

Continuous procurement bot buys from the self-replicating bots. This is still under construction. Next problem "derive efficient representations of the environment" for Q-tables. When each step in learning takes 1 month, you learn very slowly. Possible solution: robot learning gossip for parallel learning.

Upcoming sprint: agents will have a character class which determines: appetite for risk, long-term versus short term thinking, stubbornness versus flexible. Q-tables in general and CloudOmate in general. CI?

synctext commented 5 years ago

Meeting minutes: Current status: initial Q-table code created, multiple market strategies, and partial fulfillment of orderbook entries.

Upcoming sprint goal: end-to-end test

Idea of collective robot learning seems viable. First mechanism is message exchange between robots, assuming full trust, complete honesty and no attackers. Use IPv8 tutorial to publish your selected public-key, announce using our DHT, and receive incoming messages. Remove IRC server based mechanism in Plebnet. Use messages in Protocol buffers serialisation. Example demo messages to implement: "full Q-table sharing", keep it simple and atomic. Only share after you learned something and "edited it".

To call it "deep reinforcement learning" we need a neural network and historical performance data, yield of market strategies, and hardware performance (intro). Future sprint could use a testnet and orderbook replay, see our prior work with limit orders. Do the bots need to crawl the IPv8 market community to build transaction history database of last 30days? Please avoid tensorflow or other gigantic libraries, keep it simple and maintainable for easy bot-install (like here).

synctext commented 5 years ago

Meeting notes:

synctext commented 5 years ago

Thoughts; please use as basis for your readme.md:

Harapuia commented 5 years ago

I'm still monitoring this issue by the by, If you have any questions or want to yell at me for what we did to cloudomate feel free to shoot a message.

synctext commented 5 years ago

Meeting minutes:

synctext commented 5 years ago

VPN shopping list: "All providers listed here are outside the US, use encryption, accept Bitcoin, support OpenVPN and have a no logging policy".

synctext commented 5 years ago

Next week: end-to-end replication tests (and screen capture of cmdline demonstration on Youtube)?

MattSkala commented 5 years ago
synctext commented 5 years ago

Meeting note:

synctext commented 5 years ago

@mateicristea88 and @JayDew are thinking of becoming "team 7" for their honor track at TUDelft.

synctext commented 5 years ago

@mateicristea88 and @JayDew. Some semi-random suggested reading for broadening your scientific knowledge base (note, you've asked for a challenge..):

(honor project brainstorm: Facebook alternative requires ultra-reliable data storage. A self-replicating agent that uses agent gossip to offer bullet-proof encrypted personal data hosting under control of a swarm of AI agents {learning performance demand versus performance offered)

JayDew commented 5 years ago

@mateicristea88 and @JayDew. Some semi-random suggested reading for broadening your scientific knowledge base (note, you've asked for a challenge..):

* 1998: [Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria](http://www.opim.wharton.upenn.edu/~sok/papers/e/erev-roth-AER-1998.pdf)

* [Reinforcement Learning Dynamics in Social Dilemmas](https://pdfs.semanticscholar.org/6517/ac9d6d3da823b258128c520bbf41fed20200.pdf)

* [SybilBelief: A Semi-Supervised Learning Approach for Structure-Based Sybil Detection](https://arxiv.org/abs/1312.5035)

* [IoT security techniques based on machine learning](https://arxiv.org/pdf/1801.06275)

* [The latent community model for detecting sybil attacks in social networks](http://cmj4.web.rice.edu/NSF/paper.pdf)

* [An Adaptive Autonomous Bidding Agent](https://arxiv.org/pdf/1106.0678.pdf)

(honor project brainstorm: Facebook alternative requires ultra-reliable data storage. A self-replicating agent that uses agent gossip to offer bullet-proof encrypted personal data hosting under control of a swarm of AI agents {learning performance demand versus performance offered)

@mateicristea88 and @JayDew.

1998: Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria: Very interesting paper about game theory and equilibrium, also well written, would enjoy more articles like it. Reinforcement Learning Dynamics in Social Dilemmas: Good follow up to Roth and Erev, slightly too teoretic, less applied. SybilBelief: A Semi-Supervised Learning Approach for Structure-Based Sybil Detection: didn't have time to read IoT security techniques based on machine learning: Didn't enjoy it, too many references to other papers, not a lot of original ideas, just running some benchmarks and comparing them The latent community model for detecting sybil attacks in social networks: Also very interesting, had a bit of trouble with the more advanced statistics (multivariate distributions and bayesian inferences). Cool idea, reading it was pretty enjoyable. An Adaptive Autonomous Bidding Agent: easy to read, cool, could be used pretty directly to create a tribler bidding agent.

We would also want to inform that we are on track with the Honors program, having achieved all 60 ECTs and an average of 8.

synctext commented 5 years ago

Great to hear the bsc study is also going above-nominal. As a next step, please try to find 5 next-step papers in the AI sub-topics that triggered your interest when reading these papers.

synctext commented 5 years ago

Distributed machine learning:

Plus some links that are relevant for a possible distributed-Facebook based on a swarm of AI agents:

synctext commented 5 years ago

reposting the basic trustchain paper: https://pure.tudelft.nl/portal/files/47579639/paper.pdf 1968 classic: http://pages.mtu.edu/~asmayer/rural_sustain/governance/Hardin%201968.pdf reputation 2002: https://cseweb.ucsd.edu/groups/csag/html/teaching/cse225s04/Reading%20List/E-bay-Empirical-BodegaBay.pdf 2005: http://www.econ.ucla.edu/sboard/teaching/tech/dellarocas_2005.pdf https://dl.acm.org/citation.cfm?id=3236008

MattSkala commented 5 years ago

Accountless, API-driven VPS hosting for Bitcoin:

synctext commented 4 years ago

idea. Fundamental AI research as the key focus of you honors program. Gossip Learning as a Decentralized Alternative to Federated Learning from 2013 and also this one from 2019: http://www.inf.u-szeged.hu/~jelasity/cikkek/dais19a.pdf

ToDo: understand, review, implement, and evaluate for usefulness. What is the performance, scalability, robustness to errors, and attacks through false gossip?

mateicristea88 commented 4 years ago
synctext commented 4 years ago

Great having the SVM basics up and running. Always focus on the end-2-end architecture, not polish individual components. Read about making the possible step to an illustrative use-case:

Concrete next step: 1) publish code 2) convergence diagram, 3) make cooler: real sockets, 100 machines, etc. 49) use DNA dataset

@MattSkala Do you like the idea of a proof-of-principle that you can transfer your DNA (or synthetic generated data) through Bluetooth for an hour (slow), then run SVM on them, meet others phones, and convergence slowly the model? "A social AI network", "Internet-Free AI". (collaborate with @mateicristea88 @JayDew or focus on your own project; also fine)

synctext commented 4 years ago

Welcome new honor students! @dandreescu @ioanasv Please also read the starting material from 4th of July before the next meeting. We then can see if you like this stuff or put you on another direction. Directions: economic-side, trustchain-core, hardware/embedded, streaming, social/trust/reputations, security, social networks, gossip AI, etc.

synctext commented 4 years ago

Suggested reading:

synctext commented 4 years ago

And another member of the lab :-) @MateiAnton Two suggested directions for reading:

synctext commented 4 years ago

@mateicristea88 @JayDew Trustchain by requirement should be usable for your Robot-Gossip project. If it fails, we need to learn from Trustchain1 and design Trustchain2...!

@MattSkala is working on Native Java work getting trustchain up and running. He is working on getting code ready for the Blockchain Engineering master course. https://github.com/Tribler/tribler/issues/4863#issuecomment-554309477

As honor students you have not yet worked with Trustchain, please dive into it. So a practical sprint these weeks. If you browsed the Android code and workings a bit, please organise a chat with Matouš to sync with his work and roadmap for coming 2 months.

For now the "Decentral AI" could be a completely isolated button in the app. Similar to how we did "Scan Passport" in the Play store app "Trustchain". For demo purposes, that it simply runs a demo script. Simple thing, like calculating human DNA variation metrics.

Draft API:

synctext commented 4 years ago

@MateiAnton Please read https://www.ifi.uzh.ch/ce/publications/Accounting_Mechanisms.pdf and others please: https://github.com/Tribler/tribler/issues/4039#issuecomment-516052453 Also robot economy work might be interesting: https://github.com/Tribler/tribler/issues/4039#issuecomment-443536911

synctext commented 4 years ago

Update meeting:

mateicristea88 commented 4 years ago

Update:

link to repository: https://github.com/mateicristea88/trustchain-android

MattSkala commented 4 years ago

@mateicristea88 In case you are trying to use trustchain-android for communication, please switch to kotlin-ipv8 which should be much more reliable and easier to use.

mateicristea88 commented 4 years ago

Update:

synctext commented 4 years ago

Sprint update in the age of Corona.

mateicristea88 commented 4 years ago

Update:

-got code into superchain-kotlin-app

-not very clean however, we still have some java(not kotlin) code -also, for some values it deosn't converge -pushed to local fork: https://github.com/mateicristea88/trustchain-superapp

synctext commented 4 years ago

Sprint progress:

synctext commented 4 years ago

This is now officially a hot topic! Best overview paper I've seen: Adversary-resilient Distributed and Decentralized Statistical Inference and Machine Learning. It is hot because of the entire special issue publication with lots of similar articles. That great overview paper includes Recent Advances Under the Byzantine Threat Model: image

@mateicristea88 Please do a PR of your distributed AI kernel to the master branch of the superapp. We're doing a release this week and have journalist to talk about it.

synctext commented 4 years ago

Honor project update @mateicristea88 @JayDew. ToDo determine direction for May 2021 wrap-up, required to produce a scientific paper. We now have @awrgold @Tim-W on board. They can build upon your library, deploy it to real users, provide feedback on usefulness, and you can see the live gossip learning network. @awrgold is working toward a DAO-based investment agent with ML and market history datasets.@Tim-W can use your library for music discovery & recommendation.

State of the art: Automating the end-to-end lifecycle of Machine Learning applications. The distributed case remains unsolved. The tough open problem is in security. First deploy, tweak your distributed-ai-kernel to perfection, and then try to attack yourself and think of defences? Fascinating literature on Sybil attack for model poisoning. "Mitigating Sybils in Federated Learning Poisoning". Note that Sybil attacks are real, especially in social media with 4.6 million fake friendship requests. Use a [0,1] trustworthiness estimator of strangers you meet to fix this?

synctext commented 4 years ago

@mateicristea88 @JayDew Superapp is moving forward rapidly, please keep up ;-) Deployed on Google Play store. May 2021 wrap-up, define concrete goals. Possible phasing