search

Tribler / tribler

Privacy enhanced BitTorrent client with P2P content discovery
https://www.tribler.org
GNU General Public License v3.0
4.77k stars 443 forks source link

autonomous self-replicating code #2925

Closed synctext closed 5 days ago

synctext commented 7 years ago

Goal: creating healthy exit-node network through cybercurrency.

Previously a TUDelft thesis project created an initial implementation of the autonomous self-replicating code concept.

next step: go live

Key features:

Planning:

Include in the report a detailed analysis of why the previous code could not be re-used and why clean-slate implementation was started.

Jaapp- commented 7 years ago

Project

VPS configuratie kiezen via evolutie

VPS kopen

VPS instantieren

Reputatie minen

Reputatie verkopen

Tests

Jaapp- commented 7 years ago

Main repository https://github.com/rjwvandenberg/PlebNet

Jaapp- commented 7 years ago

The priority is to get the end-to-end system architecture operational instead of putting intelligence in each or every module.

synctext commented 7 years ago

docker as run everywhere execution model, from-the-source, other ?

synctext commented 7 years ago

https://www.reddit.com/r/Bitcoin/comments/50zapi/whats_the_best_vps_hosting_service_that_accepts/ https://www.google.nl/search?q=vps+bitcoin+review Perhaps include pricing info within the pip package API http://www.2sync.co/vps/taiwan/ ? Enabling computers to reason about computing power and bandwidth.

rjwvandenberg commented 7 years ago

Sprint 2 will focus on creating the robust open compute api, aptly named Cloudomate. Initial System Architecture Agent Creation VPS Registration & Payment

Agents may communicate with each other through decentralized mailing systems. We implemented a sophisticated mailing system, with agents from Delft, Singapore and New York sending each other greet messages: Agents sending greetings Messages arrived within seconds after joining the network. Further work may fall outside the project requirements, so this project will be put on notice. Dispersy_communication branch

We performed a data transfer test between VPS s hosted by DigitalOcean. Data rates around the globe varied between 8.2MB/s and 44MB/s. A 1TB bandwidth cap may be consumed within days. This may cause problems if the agent gets shutdown without being able to instantiate new agents. We must take care to review hoster policies and select suitable hosters.

In sprint 1 we reviewed the previous BEP project on autonomous self-replicating code and applied their methods. Especially selenium may be too resource heavy to run on a cheap VPS. Other problems included too tightly coupled modules. A detailed report will follow.

Using docker will mean one standard project configuration and a docker install install script for each major os (this probably already exists). If configured properly this would come at a negligible cost http://domino.research.ibm.com/library/cyberdig.nsf/papers/0929052195DD819C85257D2300681E7B/$File/rc25482.pdf

synctext commented 7 years ago

ToDo:

image image image

MitchellHop commented 7 years ago

cloudomatepip output output2

devos50 commented 7 years ago

Amazing! 👍

synctext commented 7 years ago

very impressive!

Jaapp- commented 7 years ago

report.pdf

synctext commented 7 years ago

Expand chapter 1, also add picture of, for instance, 1) buy server 2) buy cloud server 3) buy VPS with Bitcoin.

Experiment with GIT based timing of 3 providers. Start till (Github URL) first success with real Bitcoin transaction (blockchain-browser). Table with VPS provider, minutes of development, LoC, code coverage, etc?

Performance analysis. Please add timing (time-to-buy-and-Hello-world) and performance graphs (speedtest, etc.).

Measure: Trustchain record creation over time.

rjwvandenberg commented 7 years ago

dockerbuild Tribler exitnode including decentralized market is now available as a docker image. Built directly from the master branch of plebnet. https://hub.docker.com/r/plebnet/tribler_market/ Exitnode not enabled by default, and still being tested.

Jaapp- commented 7 years ago

report.pdf

synctext commented 7 years ago

Please focus on report writing. For instance, chapter 6, Actual money trials

ToDo: prepare for 100-500 Euro exit node capacity buying trial. Understanding DMCA impact.

synctext commented 7 years ago

new report ?

Jaapp- commented 7 years ago

report.pdf Report now on our own issue.

Jaapp- commented 7 years ago

The first time we're seeing our decentralized autonomous entity (DAE) reproduce and trying to earn money on our market to fork more children. 2017-06-29-170136_905x764_scrot

rjwvandenberg commented 7 years ago

In the end the server selection for cloudomate proved insufficient. Operating autonomously will require a very strict cherrypicking based on DMCA policies and registration procedures. Even then the internal setup of servers was shown to differ from what may be expected, requiring additional step after server activation. Out of the current 8 providers none have been reliable enough for an autonomous system.

In the final test we encountered some very specific errors, to correctly handle these additional code improvements to cloudomate and PlebNet are required. Especially the interaction between the two programs will have to be more rigid.

For tribler integration, the plebnet_plugin used by twisted did not incorporate the tunnel_helper instance directly, this will have to be done to avoid running a seperate twistd instance for market and tunnel_helper.

PlebMail was succesfully used to monitor the status of the Plebs: final_contribution 1

A final graph of the upload of some of the servers at this moment. (only a few remain running) performance

PlebNet repository Cloudomate repository tribler fork for PlebNet

rjwvandenberg commented 7 years ago

For the next month plebnet will run as exitnodes on the following servers that remain: linevast, pulseservers, abelohost, hidemyhost, underground, aminserve, flokinet and offshorededi These last 6 were bought without making use of cloudomate and if they stay online for the next month, they may be suitable for use with plebnet. Near real time updates on the status may be seen on the following site: plebnet live status

synctext commented 7 years ago

impressive !

Dmole commented 7 years ago

Was adding a VPN layer for VPSs with incompatible DMCA policies considered? http://compare.wikia.com/wiki/Vpn_Provider

devos50 commented 7 years ago

@Dmole actually, we considered that a few weeks ago and it's on our wishlist to implement that in one of the coming releases 👍

synctext commented 6 years ago

Next: going live in Tribler We are ready for the next steps this year with the maturing of our token economy and self-replicating code. The initial step is to provide advanced CloudOmate to ordinary users of Tribler, who are not experience traders, market makers, or command-line Linux experts. With the integration of the Bitcoin wallet we can expose buying servers to our users. Target the following for a new V7.3 or V7.4 version. It is a killer feature for press attention.

"buy mining hardware" button in Tribler. Next screen showing Earn tokens automatically. Use your Bitcoin or Ethereum wallet to buy the following (step 1 of 3):

devos50 commented 6 years ago

@thijmensjf @codesalad @vwigmore this ticket is all yours for the coming ten weeks! 👍

devos50 commented 6 years ago

Literature to start with:

devos50 commented 6 years ago

Overall goal: Setup, deploy and optionally evaluate an end-to-end Botnet with self-replicating servers. Use the existing Cloudomate software to buy (and monitor) new servers, use PlebNet to install Tribler and deploy an exit node (optionally with VPN protection). Finally, sell earned bandwidth tokens on our decentralized market.

First sprint:

codesalad commented 6 years ago

Initial long-term plan (plan van aanpak): week 1 - 2 (see first sprint):

week 3 - 4:

week 5 - 6 (depends on progress):

week 7 - 8:

week 9 - 10:

thijmensjf commented 6 years ago

Sprint 1 Logs

Tribler

Cloudomate

PlebNet

Monitoring

devos50 commented 6 years ago

Current progress:

Next sprint:

Idea:

vwigmore commented 6 years ago

Current progress

synctext commented 6 years ago
thijmensjf commented 6 years ago

The current version of the report: Bachelor_Project_2018_BotNet.pdf

vwigmore commented 6 years ago

Added coverage report building for plebnet on jenkins screenshot from 2018-05-22 13-03-31

devos50 commented 6 years ago

Status at the beginning of week 6:

TODO next sprint:

codesalad commented 6 years ago

Our Sprint 3: sprint3

Report: Bachelor_Project_2018_BotNet.pdf

We mainly focussed on preparing for SIG, Jenkins and proxmox end-to-end testing this sprint. We'll add more to the report this week as proxmox integration is done and jenkins, marketplace are prioritized in sprint 4.

vwigmore commented 6 years ago

First upload to SIG complete with a total coverage of 76% coverage

vwigmore commented 6 years ago

The current version of the report: Bachelor_Project_2018_BotNet (4).pdf

codesalad commented 6 years ago

Proxmox

Listing options with Cloudomate cloudomate purchase

Purchasing container with Cloudomate cloudomate purchase 2 We're using testnet bitcoins here, along with the Bitpay testnet API.

An invoice is created using the Bitpay API cloudomate purchase 4_edit_0

Accessing server cloudomate purchase 3 You can also create and destroy the container through the /manage "API". Containers are created and destroyed with ansible scripts (we don't use snapshots because the proxmox machine's filesystem doesn't allow that). This allows us to test things such as installing the server and monitoring on a clean container, making sure the install scripts are not missing anything.

codesalad commented 6 years ago

sprint4

Most of the items were finished, installing VPN is going to be a task for this sprint (5)

devos50 commented 6 years ago

Sprint 5 is finished. Report of this sprint will be posted on this ticket today.

Issues:

What's working right now:

Sprint 6 goals:

thijmensjf commented 6 years ago

Sprint 5 is completed and for the next sprint we will focus on end-to-end testing by buying multiple servers and installing plebnet and make a full cycle. The report should get a higher priority and will be expanded in Sprint 6 as well.

image

The current version of the report can be found here

codesalad commented 6 years ago

Naive bidding bot created The bot looks for http://localhost:8085/market/asks items and places a bid against these (works for both TBTC and BTC, depending on Tribler setup).

Currently, sometimes transactions show up as a failed transaction (but price and quantity are transferred). Because asks are not removed when transactions are not complete, the bot may place multiple bids. A lower timeout for asks could be set to relieve this.

12/06/2018 18:25:20:Coin type is: TBTC
12/06/2018 18:25:20:Starting new HTTP connection (1): localhost
12/06/2018 18:25:20:http://localhost:8085 "GET /market/asks HTTP/1.1" 200 1528
12/06/2018 18:25:20:[Ask]: 0.0001131 TBTC 500.0 MB
12/06/2018 18:25:20:[Making a bid]:  0.0001131 TBTC 500.0 MB
12/06/2018 18:25:20:Starting new HTTP connection (1): localhost
12/06/2018 18:25:20:http://localhost:8085 "PUT /market/bids HTTP/1.1" 200 None
12/06/2018 18:25:20:{u'created': True}
12/06/2018 18:25:30:[Ask]: 0.0001131 TBTC 500.0 MB
12/06/2018 18:25:30:[Making a bid]:  0.0001131 TBTC 500.0 MB
12/06/2018 18:25:30:Starting new HTTP connection (1): localhost
12/06/2018 18:25:31:http://localhost:8085 "PUT /market/bids HTTP/1.1" 200 None
12/06/2018 18:25:31:{u'created': True}
thijmensjf commented 6 years ago

The current version of our report can be found here

thijmensjf commented 6 years ago

Currently (re)writing our report, as it has to be handed over comming Saturday.

Implemented some features regarding data collection to expand the monitoring options. Every ten minutes the data (uploaded and downloaded MBs, MB/BTC/TBTC balance etc.) from the agents is stored and this used to automatically create graphs. fig__deltas_for_mb_balance fig__deltas_for_uploaded fig__values_for_mb_balance

For now these figures do not tell that much, as the time span is short (~24 hours), but during the demo we will have collected data for 2 weeks. This should be usable to gain an insight in the effectiveness of PlebNet.

thijmensjf commented 6 years ago

And we have created a logo for PlebNet, as it was lacking one: logo

synctext commented 6 years ago

Your promo slides presented here upcoming Friday?

vwigmore commented 6 years ago

Final version of the report as send per email Bachelor_Project_BotNet.pdf

codesalad commented 6 years ago

@synctext This is one of the gifs we've prepared. A "real-time" network viewing tool for PlebNet

demo_vision

I will send you the rest via e-mail.

codesalad commented 6 years ago

As requested, I'm uploading the slides/images/videos for presentation here as well.

Slide 1:

logos

Slide 2

Slide 3: