Fairness and freedom for artists: Towards a Robot Economy for the Music (Streaming) Industry

[Tim-W]

(Placeholder issue for msc thesis)

Problem statement

(Musicians/artists/creators) receive low compensation for publishing their content. * [Typical cut of revenue:](https://www.theguardian.com/technology/2015/apr/03/how-much-musicians-make-spotify-itunes-youtube) (distributor: Spotify/iTunes/Google Play/...) Signed records: 25% distributor 55% label 20% artist Unsigned records: 40% distributor 60% artist * Smaller artists get low revenue, due to the "pro rata" model applied by e.g. Spotify. A [user-centric](https://www.rollingstone.com/music/music-features/should-spotify-change-the-way-it-pays-artists-763986/) model should be more fair (economics/business, less CS) * [The distributor could censor tracks](https://www.springfieldnewssun.com/entertainment/spotify-removes-kelly-music-from-playlists-under-new-hateful-content-policy/9NIgUD8qJBd2tAYmVhn5hK/) * [Platform lock-in](https://www.google.com/search?q=Google+Rules%3A+The+History+and+Future+of+Copyright+Under+the+Influence+of+Google) **Existing and upcoming solutions** Blockchain-based solutions exist for music streaming services. At least two start-ups attempted. * [OPUS Audio](https://opus.audio/) "Helps artist generate the revenue they deserve". It uses [IPFS](https://ipfs.io/) for distributed file storage, but actually uses a back-up off-chain database for performance purposes. Strategy for IPFS nodes: OPUS deploys **its own** nodes around the globe for low latency. OPUS uses Ethereum smart contracts. Using smart contracts, "[consumers] can rest assured almost 100% of the payments are delivered to the artists transparently, immutably and without intermediaries taking large chunks of revenue”. * [Musicoin project](https://musicoin.org/) encourages **independent artists** to register and publish their work on its own blockchain-based platform. Very interesting: consumers have a free service without ads, while artists get paid (in $MUSIC) Uses earn-per-play smart contracts based on preset fees each time a song gets played (for >s seconds). Consumers can also tip artists (micropayments). Uses mining in the form of bitcoin. * [Mycelia project](http://myceliaformusic.org/) is not a streaming service on its own but does provide "creative passport" technology: a digital container with authorization of work, acknowledgements, payment mechanisms and metadata. **Shortcomings of these solutions** * Musicoin pay-per-play system depends entirely on investors and the price of its non-stable currency. If its currency becomes [worthless](https://coinmarketcap.com/currencies/musicoin/) then the artists also receive low revenue. * OPUS deploys nodes itself and consumer pay a (small) fee for this infrastructure (different from Tribler/torrents) **The dream: a good decentralized streaming system** * A (torrent) streaming technology as fast as spotify, based on a p2p file sharing system (tribler? IPFS?). Challenge: seeders for content with small popularity. To think about: are bandwidth tokens enough incentive? * Proof of authorship on the blockchain Tribler/tribler#4144 * (Near) 100% of fees from consumers go directly to the artists they listen to. * Payouts with stablecoin (USDT/EURS) * Tipping system/micropayments * Effective sharing and recommendation systems **Completed reading list** * [How the blockchain is disrupting the art economy as we know it (2017)](https://www.forbes.com/sites/rogeraitken/2017/08/17/how-the-blockchain-is-disrupting-the-art-economy-as-we-know-it/#53c12c9f74fe) * de Vos, M., & Pouwelse, J. (2018). A Blockchain-based Micro-Economy of Bandwidth Tokens. CompSys 2018. * [Blockchain is changing how media and entertainment companies compete](https://sloanreview.mit.edu/article/blockchain-is-changing-how-media-and-entertainment-companies-compete/) * Zhang, B. (2018). Credit Mining: An Incentive and Boosting System in a Peer-to-Peer File-sharing Network.

[synctext]

Brainstorm on direction:

• pre-fetching algorithm/engine for uninterrupted audio streaming
• organic growth of music taste and neighbour discovery
• crowdsourcing of music playlists (magnet links, fight spam, build trust)
• Eurotoken foundation for artist donations
• flawless streaming of magnets (latency and stutter)

Work from now 15 years ago Wi-Fi Walkman: A wireless handhold that shares and recommend music on peer-to-peer networks

Upcoming sprint: document related_work.tex in thesis wording?

[Tim-W]

Looked up more decentralized music streaming services:

1. Audius (85% profit to artists, uses ads, other profit to developers of music app)
2. BitSong (unreleased) (no stablecoin)
3. Musicoin (No stablecoin)
4. Resonate (Profit distribution is a "co-op": Musicians 30% Listeners 25% Workers 15% Rainy day fund 20% Owners of “supporter shares” 10%)
5. eMusic (Seems to be most mature, 60 tracks for $20/month. No stablecoin. They sell their eMC token for 0.35$ each, but could up this price whenever they want, pays artists/content distributors in eMC)

IMO services only using non-stablecoins can be disregarded. They build a platform and launch their own token/coin which hopes to gain value (hard to say for future).

Conclusion: None of these applications pay near-100% of profits to artists in "real" money (eur/usd)

[Tim-W]

(Work in progress) related_work.tex

[Tim-W]

Updated the tex with a summary of current decentralized audio streaming services, and decentralized content delivery networks.

Also, possible future directions:

• "content discovery and metadata sharing using gossiping protocols (torrents and/or blockchain)"
• "incentivizing seeders (bandwidth token?)"
• "paying seeders and artists through fast smart contract system"

• Encrypted audio files, key unlocked via smart contract ?

[synctext]

Feedback on progress:

• are you providing an alternative to Big Tech and the centralisation of our economy?
• what is a better problem description for your thesis?
• Can this be (in principle) generalised beyond audio?
• "pre-fetching algorithm/engine for uninterrupted audio streaming"
  • radio non-interactive model
  • invest MBytes into an audio asset
  • Only when downloaded 100% we score user satisfaction
  • Downloads of 99% or less resulted in waste of user resources
  • Having no download to play for a user results in significant user anger
  • assign a factor to usage of caching space and speculative bandwidth usage
  • flavors: greedy, frugal, or unlimited resource usage for true uninterrupted audio streaming
• Optionally filter potential magnet links with existing collaborative filtering algorithms
• completely other direction is "follow the money"
  • artist fairness, frictionless Bitcoin donations, "Tipping system/micropayments"
  • build upon the DAO
  • pay Bitcoins for pull request which extend The DAO (e.g. enslave developers)
  • crowdsourcing, text files of magnet links mashups, music discovery, bluetooth local discovery
  • proof-of-authorship, artist passport
  • 17 student: https://github.com/Tribler/tribler/issues/4863#issuecomment-588331663
  • content discovery ongoing work

[Tim-W]

Reading:

• Kreitz, G., & Niemela, F. (2010, August). Spotify--large scale, low latency, P2P music-on-demand streaming. In 2010 IEEE Tenth International Conference on Peer-to-Peer Computing (P2P) (pp. 1-10). IEEE.
• Y. Huang, T. Z. Fu, D.-M. Chiu, J. C. Lui, and C. Huang, “Challenges, design and analysis of a large-scale P2P-VoD system,” SIGCOMM Comput. Commun. Rev., vol. 38, no. 4, pp. 375–388, 2008.
• Zhang, B., Kreitz, G., Isaksson, M., Ubillos, J., Urdaneta, G., Pouwelse, J. A., & Epema, D. (2013, April). Understanding user behavior in spotify. In 2013 Proceedings IEEE INFOCOM (pp. 220-224). IEEE.

[Tim-W]

From my perspective, depending on time constraints/progress I could try system 1 or system 2 (using a DAO): System 1: fairness for artists, any files Challenges:

1. Latency/speed of (donation) payments, synced with torrent downloads
2. Proof of ownership/artist passports
3. Permissioned access to publish content
4. Sharing and discovery of content, metadata storage
5. Incentivizing seeders (bandwidth tokens? seeders receive stablecoins?)

System 2: fairness for artists, music streaming Additional challenges:

1. Low latency/stutter/seeking audio files
2. Prediction of next played audio file, pre-fetching algorithm
3. Music discovery/recommendations
4. Pay-per-stream instead of pay-per-file?

[Tim-W]

Change in relation to my most recent comment: The system: A music streaming and/or file sharing DAO where users can donate to artists. (near) 100% of those donations are received (near) instantly by the artist. Artists can prove which content is theirs so that the correct artist receives the donation.

[synctext]

cool! btw Both Bitcoin and Libtorrent have now been integrated in DelftDAO. That saves you lots of engineering.

[synctext]

@Tim-W online for sprint update?

[Tim-W]

Sorry, I have not been able to make much progress in the recent weeks.. (see my email) I will give an update before the coming Tuesday

[Tim-W]

@synctext Sprint update: Trying to define a better problem statement: Problem statement updated, split up into 4 subproblems:

• As a user of any good paid audio streaming service, large share of my money goes to the platform instead of my favourite artists.
• Torrenting the music while giving donations in parallel to artists is a solution. However this is a bad user experience as there exists no seamless streaming and exploring system for audio torrents.
• I need to look up how to donate for each of my fav artist which is cumbersome.
• I need to be sure that when I make a donation, it actually goes to the right artist/label/rec holder.

any good paid audio streaming service refers to any of the spotify/google play/itunes that provide seamless experience and a large amount of content. A possible (dreamy) solution:

• A seamless music streaming experience without a middleman taking (nearly any of) my money. I want to donate directly to my fav. artists. As a listener I want these functionalities combined in a single app.

Progress Read most of Mitchell Olsthoorn's FBase thesis Read The DAO whitepaper Trustchain superapp: Inspected docs, code, ran the app. Kotlin-ipv8: Inspected docs, code. Plans Coming days: fork superapp, create a DAO, upload&share audio/music metadata with peers on an IPv8 app. Coming week: Find more papers about fast music (torrent) streaming. Then: describe the "dreamy" solution in a more technical system, describe what the "MusicDAO" should be able to do, and predict feasibility for thesis.

[Tim-W]

Forked IPv8 superapp, wrote some code, added my own very simple app to it.

• Created custom ipv8 community
• Iterative peer discovery
• "Discover music content": iterative scanning
• Very simple "discovery" messages on overlay with authenticated communication
• Followed TrustChainTutorial to get hands on with trustchain interactions. Content on trustchain: "Artist" publishes a track with a track id, name, magnet link. Others can validate whether the content is a duplicate (simple identity theft resilience)

[Tim-W]

@synctext Can you link me to the project for bitcoin/libtorrent integration in DelftDAO?

[synctext]

Great stuff! Feel free to re-use the pro-artist storyline I wrote April 2019:

Many people have tried to create a micro-economy for Internet privacy using coins, credits, and
karma for onion routing. Various incentive mechanism have been proposed to prevent the tragedy
of the commons, such as, GoldStar, PAR, BRAIDS, LIRA, TEARS, and TorCoin
[ARS08,DWB10,GRF14,JHK10,JJS13,JMS14,RMS04]. However, the problem remains unsolved.
Our prior work: In prior work we created “Self-replicating autonomous AI agents”. Our
self-replication toolkit called “​ CloudOMate​ ” is now mainstream open source software 2 . CloudOMate
offers a permissionless open compute API which provides an automated way to buy VPS instances
and VPN services from multiple providers. Our work differs from buying cloud hardware by credit
card. Our agents are truly autonomous and humans are unable to access their money. This
architecture offers unique security properties, theft of credit card numbers or passwords are no
longer possible. Based on this toolkit we created a live ecosystem. Our self-replicating agents can
own electronic money (Bitcoin, Ethereum), buy server space in the cloud without any human
assistance, offer compute services on a decentral market, have a primitive learning ability, and
can convert income into replicating themselves [RSP19].

Method and approach:We will study the right conditions for a decentralised digital economy
focused on music to flourish and realise it. Our alternative for the music industry is dubbed the
“Maximum Love Music” (MLM) community. Previously, we’ve grown a Youtube-like community
with ​ currently 35.000 daily active users​ , called Tribler. This MLM community will focus on
trust, marketplaces, and feature real money. MLM is academically pure and disruptive; it only
exists in code (e.g., “DAO” [UDA17]); all middleman functionality is conducted by open source
software for free. All money directly goes to the artists. Musicians can release music, sell concert
tickets, and receive payments without ​ any of this money going to record labels, producers,
bankers, credit card companies, or tech companies.
MicroEcon is deliberately provocative. The irony is that companies, like Spotify, use secret AI
algorithms extensively. We aim to disrupt the “Tech Elite” by offering a transparent non-profit AI
alternative, based on CloudOmate. The tension and emotion we consciously trigger lies in our
conflicted desire to create superhuman contraptions in capacity, but subhuman in status.
Creative element: We will create a live micro-economy with a self-sustaining society of
agents. All decisions about self-replication, money, tokens, and long-term survival will be taken
by autonomous intelligence. By applying end-to-end reinforcement learning we will use a single
goal which will drive the behaviour of agents: survival.
Our provocative approach is designed to stimulate volunteer contributions and mirror successful
projects like Linux, Ethereum, and Wikipedia.

We did CloudOMate. A full 19 master students are actively working on Delft-DAO towards a single codebase. Including Libtorrent integration and Bitcoin MultiSig scripting.

[synctext]

Suggested thesis sprint goals:

• Build upon existing Delft-DAO
• use Bitcoin wallet (Testnet)
• Libtorrent seeding of Creative Commons music
• Trustchain records to publish music Magnet links and link it to Bitcoin donation wallet
• play and donate button for music
• Proof-of-Principle for MusicDAO
• Do pull request of MusicDAO to SuperApp of @MattSkala of basic prototype in coming weeks
• iteratively improve for thesis
• build-up expertise and devise a clean architecture of what you already prototyped

Scientific issues:

• artist passport (this is My music and My wallet)
• music discovery and recommendation
• seamless, friction-free experience which has zero-maintenance and requires no central servers
• disruptive business model for artists (self-funded, self-publishing, self-managed)
• architecture for direct donations
• security: spoofing prevention, pollution of recommender, etc

[synctext]

The first Robot economy

Maximum ambition claim, you replace an entire industry by software. For the specific case of music you remove all middleman from the value chain. All economic activity into a single architecture. This means creating creating an operational end-to-end prototype. Emphasis can be in various directions:

• Private streaming; Tor-like privacy on smartphones; approaches that fully scales; build upon Tribler/tribler#4827
• Flawless streaming; speculative downloads, swarm health monitoring, fetch in parallel and only play if downloaded fully correctly
• User focus; improve user interface; search existing player code
• Business model; donations, artist revenue; talk to actual artists for a field trial and publicity
• AI; connect to CloudOMate
• Recommendation engine; operational playlist exchange without servers; spam flighting and pollution prevention; Trustchain for building web-of-trust
• Delft-DAO; creating a sustainable robot economy; get paid a bounty for extension of your app; create an autonomous micro-economy for this superapp; Bitcoin for new features.

[Tim-W]

Coming 4-ish weeks:

• Create a thesis.tex
• Prototyping simple app that can do:
  • Store magnet links + metadata on trustchain for tracks
  • Discover tracks
  • Play tracks using libtorrent
• Problem description (+introduction)

[synctext]

content: Jamendo + these 29,818 Creative commons tracks

[Tim-W]

Progress of code: https://github.com/Tim-W/trustchain-superapp/commits/master

0. DHT peers are continuously searched in background
1. The user can select an audio file from his Android
2. Audio file is converted to trackerless torrent, magnet link generated
3. Proposal block generated to trustchain including 1 song: {track ID, author, magnet link}
4. Block is self-signed
5. Other peers can fetch signed blocks (in video it is the same person), grab magnet link, start torrent metadata fetch
6. Use libtorrent to discover DHT peers, start buffering torrent pieces
7. When torrent pieces > X are loaded, exit "buffering" state. Play button then clickable
8. Keep downloading the torrent file. Seed after completion

Uses https://github.com/frostwire/frostwire-jlibtorrent and https://github.com/TorrentStream/TorrentStream-Android . TODO: seeking, track selection

Caveats:

1. Cannot demonstrate uploading and streaming from 2 peers as only 1 peer is not enough for torrent streaming. Will be very slow obv. In the video, the magnet link used for downloading is a static one with ~50 seeders.
2. Video is sped-up, "establishing peer time" is ~10s for magnet link described above

[synctext]

nice! please rebase superapp with latest upstream additions + use a fixed creative commons content linked for demo purposes.

[Tim-W]

:+1: Good point about the creative commons content, also I already rebased the superapp with latest upstream additions.

Thesis document updates will be from now on here: https://github.com/Tim-W/master-thesis I extended the related work and abstract here (document.pdf) mainly as a guideline for myself

Also, still thinking about the title. Perhaps "A music app/system fully owned and run by its users" or something more in the DAO direction (sustainable/future-proof app?).

Edit 30/04: Started working on introduction. see thesis git

[synctext]

Direct link to binary .PDF of master thesis a music industry without any industry

Thesis draft comments:

• Solid progress with your prototyping: Libtorrent, streaming, Trustchain-based music discovery
• "A music app fully owned and run by its users", focus on a collective. Alternatives welcome with more bite.
• "This thesis investigates the feasibility and usability of an audio streaming service without a central distributor."; too modest and hesitant language. You will design a system and present a proof-of-principle where all money goes to artists. full stop
• shift in business model: online payments and integrated tickets purchase for live performances
• Read prior thesis work as preparation:
  • Trustworthy code execution
  • Sybil-proof accounting with Trustchain
  • Trading and markets
  • Consensus that scales
• Please do an upstream PR coming days. Before 11st May have a stable Creative Commons music playback (http://bt.etree.org/index.php?sort=seeders).
• Upcoming sprint goal?
  • integrate with MultiSig Bitcoin wallet
  • Artists can create a signed playlist with wallet and magnet links
  • any user can discovery this music and reward artist directly
  • most rewarded artists get top of frontpage (e.g. Like and Followers culture)

[Tim-W]

Short update on recent work: Worked on the app to make it more stable for a very simple proof of concept and to prepare it for a PR to the superapp repo. Also I'll try to share an APK soon once downloading/playing runs more smoothly.

• Use creative commons magnet links
• Each block is now a Release (album/playlist/EP/...) with 1 or multiple audio files. Once the magnet link metadata is fetched, the audio tracks are shown in a scrollable list. Each track can be selected independently to download & play.
• Created simple seeking & buffering behaviour: once you seek the song, it prioritizes the according torrent pieces to those minutes
• Created a view where you can see the torrent pieces. Each torrent piece turns blue when it is downloaded. (for debuggin & buffering testing)

[synctext]

Impressive progress again..! Please do a first PR today or tomorrow at the latest for the Superapp

[synctext]

robot economy for music

comments on thesis progress:

• repeating thesis direction: a music industry without any industry
• repeating: the first robot economy
• "2. Problem Description" focus: The robots are coming. Big Tech is dominating our online world. We attempt to create the first real alternative for Big Tech. We focus on the music industry due to the low level of innovation, monopoly formation, significant friction in this industry, copyright orthodoxy, market failure to control prices, and its overall readiness for digital displacement.
• a music collective is less scientific then a first robot micro-economy with real users, real money, democratic rule and full self-governance. e.g. DAO-based
• platform lockin: Youtube is too large, you have to be there as an artist and play by their monopoly rules
• luxury communism feedback: sounds nice dream, but can it also deliver within a non-trivial setting. Can this underpin the music industry of the future?
• too much activism, too little science: "breaking the tyranny of central control"..
• pragmatic artist naming claim: first claim wins, TOFU, note vulnerability for name squatting

Exploring scientific directions:

• "3. System architecture and design" We explore using swarm intelligence as the new organisational for alternatives to central Big Tech. Our design uses acadamic pure forms of self-organisations without any central point for coordination, control or governance. Specifically, music is discovered by gossip between neighbors. The emergent effect is that within seconds keyword searches return relevant results and zero-server AI content recommendations are keep users engaged.
  • Business model focus: we show a real functional micro-economy with 100% going to artists.
  • App focus: self-publishing and artist passport.

Upcoming sprints:

• wrap up streaming, buffering and seek. Create a stable basis for next sprints, ideally no need to re-visit the Magnet-Level and Libtorrent.
• first re-write of Chapter 2. and start with "3. System architecture and design" chapter
• Open question, re-usage of TUDelft broad knowledge of music recommendation
  • single artist playlist
  • crowdsourced recommendation playlist

[Tim-W]

Short update on this week's progress: Focussed on stability and debugging progress per file.

• Removed the torrentstream-android dependency layer for now and only use JLibtorrent, as torrentstream-android was buggy
• Fix several issues with seeking, pausing/playing and switching tracks
• Handle file priorities: Once a file is selected to play, it gets a high priority to download
• Created a progress bar per-file
• Created animated GIF, see below

[synctext]

Please include subscription concept in thesis, if you agree with this: https://stratechery.com/2020/dithering-and-the-open-web/ open + pay-for. Playlist donations in Bitcoin in prototype? We need to replace Big Tech storyline

[Tim-W]

Updated fork with upstream changes: app-debug.apk (The creative commons magnet link used for the example contains flac files from bt.etree with size 10-20 MBs, usually high-quality mp3 songs would be 5-10MBs)

[Tim-W]

Improvements on UI partially done (some components used from https://github.com/devendrakushwah/Refresh-Music-Player) . Fast forward/backward created.

To do:

• handle track titles, artist names etc if they are available.
• Some improvements on margins and colours.
• Make one more screen with album/playlist overview, select album->view tracks
• Possibly also: make shuffle/repeat buttons work.

Related work? Free Music Archive/Tribe of Noise Free Music Archive to develop new online business models based on open standards, supporting access to royalty free music.

[Tim-W]

Progress update: a lot of work on the UI and functionality

Video 1: Publishing and browsing playlists

Video 2: Playing songs

Note: when opening a playlist, the metadata is found using magnet link, which can take some time to fetch.

Progress

• Implement ExoPlayer mediaplayer for stability in listening to tracks while they are downloading (which is used in major apps like youtube)
• Prioritize torrent pieces at start of selected track to buffer and play it asap. See video 1 where the user does not have to wait for full download before it can be played.
• Browse through different playlists
• Keep playlists in memory using trustchain persistant local database
• Discover playlists: when a playlist tagged in the form of a trustchain block with type "publish_release" is found, it is shown in overview

app-debug.apk (This debug app currently uses "Royalty Free Background Music" magnet link for the "Load example" button, it has around 40 seeders at the time of writing)

Possible to do list next few days

• Thesis: Extend the problem description section, make it a coherent story
• Code cleanup -> create a pull request to Tribler/Superapp
• Enable FLAC support in ExoPlayer
• Read song name/artist etc. metadata per track
• Create a seed box for CC music?

[synctext]

https://cointelegraph.com/news/blockchain-to-disrupt-music-industry-and-make-it-change-tune/amp

[synctext]

• Making solid progress!
• Opening experience:
  • 10 albums
  • direct ability to scroll
  • 1-click playback
  • Creative Commons cover art (hard-coded, demo only)?
• :heavy_plus_sign: button in lower-right corner for magnet link addition of album
• Always show in player "loading" with magnet link swarm-name
• Future sprint idea:
  • keyword search
  • Vadim channel support for millions of magnet links IPv8 community
  • seedbox with all Creative Commons/Public Domain stuff available
  • tough 4GByte testcase to stresss-test all code with 1000+ Create Commons songs in 1 swarm
  • Podcasts with coverart: https://player.fm/podcasts/Creative-Commons
  • minimal recommendation engine, collaborative filtering
  • content indexing, downloading, and adding to a channel
  • content is freely available, but stuck on a website, requires account creation
  • One Website, 1 channel duplicate
  • https://freemusicarchive.org/genre/Rock_Opera
  • https://archive.org/details/audio
  • https://developer.jamendo.com/v3.0/docs#introduction-xmldump
  • CCtrax royalty free section
  • https://hearthis.at/creative-commons/
  • http://ccmixter.org/
  • many
  • many
  • many
  • more
  • Create a web-scrape and automated download link harvester if and only if Creative Commons?

[synctext]

(ToDiscuss) Start a seeding website called "CreativeCommonsSeeding.org"? (ToDiscuss) Later sprint of 2 weeks for Creative Commons content ingestion and metadata ingestion:

• Scrapely is a library for extracting structured data from HTML pages.
• Scrapy, a fast high-level web crawling & scraping framework for Python.
• Web crawling framework based on asyncio.
• PyQuery-based scraping micro-framework. Supports Python 2.x and 3.x.
• (too fancy I believe) Cola: high-level distributed crawling framework
• (fork of original) Beautiful Soup is a Python library designed for quick turnaround projects like screen-scraping.
• Lots of Python-based others
• AutoExtract Spiders are a easy to use tool, built on top of ScrapingHub's AI Enabled Automatic Data Extraction and designed for e-commerce and article data extraction at scale.
• Document for others how to build an importer for Creative Commons content

# https://github.com/gaojiuli/gain#usage
class MySpider(Spider):
    start_url = 'https://mydramatime.com/europe-and-us-drama/'
    concurrency = 5
    headers = {'User-Agent': 'Google Spider'}
    parsers = [
               XPathParser('//span[@class="category-name"]/a/@href'),
               XPathParser('//div[contains(@class, "pagination")]/ul/li/a[contains(@href, "page")]/@href'),
               XPathParser('//div[@class="mini-left"]//div[contains(@class, "mini-title")]/a/@href', Post)
              ]
    proxy = 'https://localhost:1234'

MySpider.run()

Note, Kotlin would be for Android integration. Python for not excluding possible Tribler integration (Channels with semantic scraper plugins for Creative Commons content + daily website sync)

[Tim-W]

Updates:

• Fixed listening to own tracks: When you submit a Release, you can play it immediately. Torrent file is saved on cache so that on next app boot the tracks can be played fast.
• Created ContentSeeder class, which, in the background, selects all torrents of which the torrent metadata is already fetched from the magnet, are all seeding. With a maximum of 10 torrents for now (lead for future experiment?)
• Show selected song description
• Pre load tracks in the background. Current selected track has highest priority to download.
• FAB button added which allows for either selecting a tracks from local folder or for pasting a magnet. All in one form/interface. See video below.

Video showing: Uploading local tracks and listening to them plays them immediately.

Created pull request for trustchain-superapp https://github.com/Tribler/trustchain-superapp/pull/42

[synctext]

Just had a chat with the honour students which are improving their distributed AI kernel till May 2021. For your robot economy it is much more scientific to focus on this line of work then doing large-scale scraping. I will ask one of our scientific developers to build a Creative Commons content library + seedbox. Free for you to use :-)

Could you at some point spend 1 day exploring the distributed AI kernel and form an opinion on the current state of usability for music discovery? I have no idea how mature this stuff really is. https://github.com/Tribler/tribler/issues/4039#issuecomment-645882761

[Tim-W]

I took a look at the code and docs for distributed-ai kernel it this afternoon. It seems to be a good base for local machine learning and linear regression/evolutionary algorithms kind of stuff, but does not have any interaction with IPv8 (if I'm not mistaken?). It only spawns multiple machines locally in the JVM which interact, but does not use actual IPv8 nodes. So that will need to be implemented. Also, I don't think linear regression is close to the simple collaborative filtering problem that I'm planning to build as recommendation algorithm?

In other news, finished the PR for superapp and improved pre-loading and caching of tracks and releases.

app-debug.zip

[synctext]

Still immature. Cool, yes, the honour students are now tasked with integrating their machine learning with IPv8. It now is still distributed in a simple way, already leaderless.

Also, I don't think linear regression is close to the simple collaborative filtering problem that I'm planning to build as recommendation algorithm?

Back in 2006 I co-authored work with simple collaborative filtering. Hopefully you can go beyond that type of work. The essentials:

What can you think of that screams "robot economy" in music context? Was also looking into using gossip-based machine for content discovery. Million Playlists Songs Dataset with Musicbrainz/Acousticbrainz integration, friends of mine (co-founded that stuff back in tha day). Music recommendation is a whole new field with lots of scientists, few easy to use libraries, some datasets, many one-shot experiments and disconnected code repos; overview.

Model-based approach within collaborative filtering looks promising for possible future sprints. https://github.com/robi56/Deep-Learning-for-Recommendation-Systems#papers

No need to do scraping, solved! 917 GiB and 343 days of Creative Commons-licensed audio from 106,574 tracks from 16,341 artists and 14,854 albums, arranged in a hierarchical taxonomy of 161 genres.

[Tim-W]

Short term:

• Expand problem description. I think this will be more along the lines of "Power in the creators industry is in the hands of Big Tech companies. Their software services are data-hungry, closed-source, proprietary and the companies form oligopolies. Solution: Replace a company by software. Bring the data = power in the hands of its users and make the software transparent."

Future sprint: Integrate with CCommons tracks or all 2000 Tribler channels. Keyword search and browsing experience.

Overall: different paths/directions to take:

• Investment in artists: invest bitcoins in some artist and take profit from them https://www.maecenas.co/
• Donations to artists
• Sell live concert tickets, while using the MusicDAO as a platform for displaying your music
• Collaborative filtering, machine learning
• Vote for updates: integrate luxury communism concept/code

Title ideas (just some ideas for parts of titles, combine different of these statements from below to build one full title)

• Fairness for artists: robot economy for music without any middleman
• Distributing of the power of Big Tech to artists and their listeners
• Replacing a company by software / Running software-as-a-company
• Removing (greedy) intermediaries in software (takes my approach to a more abstract level)
• Decentralizing power in the music software industry (focuses on the power that Big Tech has, data = power, so distribute the data)
• An attack-resilient and sensor-free music sharing DAO
• A self-organizing community of listeners supporting their artists without intermediaries
• A running implementation of the DAO principle of a music streaming service
• A decentralized, transparent and attack-resilient music streaming service

ML/AI integration ideas

• Music playlist recommendations: collaborative filtering, latent factor decomposition, deep learning
• Subscription concept: combine and pay-out bitcoins from listeners to artists based on clicks/upvotes/listens in a transparent way
• Swarm intelligence: build and implement an algorithm to search for healthy swarms, preload and cache content

[synctext]

a 'Swarm Hunter AI', your personal music seeking agent. Nice idea!

[Tim-W]

Integration with crypto-payments (an overview where I ramble about some rough ideas I have)

Subscription concept rough ideas This whole idea might not work because of the distributed nature of the system that I built. Of course the user can ignore the application layer and go straight to the libtorrent interactions.

Implementation 1: Single, independent robot/agent executing cron jobs. Its pricing, internal algorithm, and functions can be changed by voting (luxury communism approach). All users pay a monthly fee -> this fee is stored in a wallet that only the agent has access to. Then, every day/week/month the money on this wallet is distributed to the artists based on amount of plays per artist (or based on some other heuristic).

Pros: The transaction message complexity is low: one transaction per time unit from the user, and x transactions per time unit, where x is the amount of artists that need to be paid. The robot controls the money flow and is robust: does not go offline when users leave the system. Privacy: the amount of plays is anonymized. Cons: Requires strong security, difficult to engineer? The robot needs fuel/a VPS (CloudOMate may be a solution?). Also this is a single point of failure. The robot needs to be attack-resilient, strong against spam attacks, and needs to have an anti-cheat system. Which is probably difficult to build. Trust in the robot with user data (amount of plays) is required.

Implementation 2: a peer-to-peer subscription payment system. Every user stores money on a wallet. Every time unit, this wallet is reduced by a fee, which is split into p2p transactions directly to the artists, based on the user plays of the last time unit up-to current time unit.

Pros: Keeps the fully distributed nature of the system. Trivial to extend to other direct payment options and donation functions. Possibility to have 'personal interaction' this way with artists, if the user wants to. Cons: It is very difficult to engineer a system that is on the one hand fully distributed and on the other hand has a paywall against content delivery (? needs research). Transaction data can be scraped by anyone to observe statistics of amount of plays per user (unless transactions are anonymous). Message complexity of transactions is higher than in previously mentioned system.

Live concert tickets As an alternative, live concert tickets could be purchased directly from the artist/label/organization without an intermediary like TicketMaster taking commission fee (TicketMaster: 10% fee for buyers, 15% fee for sellers). This can be extended trivially to different subjects, like preview album releases, merchandise, physical records, ...

Invest in artists A very different approach is giving the users the ability to invest bitcoins in artists, who can receive a return on investment based on the 'pricing' on these artists, with a small commission fee. Artists supported based on 'voting power' integrates well with DAO principle.

Conclusion: I think the first implementation of subscription system is the most interesting to implement, for the reason of novelty and alignment with 'robot economy'. But I'll have to do more research on what exists today. Core to all these ideas: builds on the assumption that the underlying identity system works well to identify artists.

[synctext]

Agreed. Your idea: a peer-to-peer subscription payment system has the least amount of "thesis risk", could become popular and sounds like fun for future students to expand upon. There is even an algorithms component. We formulate the Artist Income Division problem as follows, how to divide the monthly donation, given the song playback history, payouts from prior months, and fraction of each song that has been played. Trustchain contains your validated artistic genesis ID, Bitcoin wallet, and verified playlist. This is very "Robot Economy"!

### Songs from https://soundcloud.com/royaltyfreemusic-nocopyrightmusic/sets/creative-commons-music
1.  100%:  Ikson — Last Summer
2.  3%:    MBB — Destination
3.  100%:  Arc North — Arc North - Ivory
4.  100%:  IanFever — Ian Fever & Almi - Autumn
5.  78%:   Rameses B — Rameses B - Bae Bae
6.  2%:    Nasko — Definite

[Tim-W]

First draft of problem description (2 pages): problem-description-draft1.pdf The basics are there, can be expanded a lot quite easily. It needs a stronger introduction why exactly the music industry is how it is now (an oligarchy of 3 streaming services). Could not find exact reasons why this oligarchy is currently the case, but rather only that it is the case. But if I go down that path it may become a financial/economics research on market shares :thinking: What is your opinion?

[synctext]

good stuff! GDPR etc I would leave out. Just copy from economic works and focus on 'monopoly pricing power' and lack of new disruptive innovations. More robot economy: can we replace a monopoly-based unfair industry with code? Find citations like this: https://neweconomics.org/uploads/files/Digital-Power-Players_181113_155656.pdf

[Tim-W]

Alright, thank you :+1: read the paper. Mainly it is useful because of its description of 'monopsomy' that fits my context. In other news, I'm planning to implement GigaChannel, as it currently is described in https://github.com/Tribler/tribler/blob/devel/src/tribler-core/tribler_core/modules/metadata_store/community/gigachannel_community.py in Kotlin. Is that a next logical step? @synctext (As we discussed in this ticket 18 days ago:

• Vadim channel support for millions of magnet links IPv8 community
• seedbox with all Creative Commons/Public Domain stuff available)

@ichorid @MattSkala For implementing GigaChannelCommunity, as it exists in devel currently, in Kotlin, are there resources/docs available? Or should it be satisfactory to just look at the code at gigachannel_community.py and convert it, by subclassing the Community.kt from kotlin-ipv8? What are your thoughts? Thanks! :)

[ichorid]

@Tim-W , it depends on why you need GigaChannel community in your app? What do you expect it to do? How are you going to interface with it? Anyways, it is better to use the RemoteQuery community that is more recent and cleanly written.

[Tim-W]

That seems useful :+1: I'll checkout the RemoteQuery code. What I want to accomplish is: browsing channels and keyword search behaviors in my Android app (the app is described in this ticket)

[ichorid]

RemoteQuery community was designed just for those two purposes: browsing channels on remote machines and doing remote queries :+1: Your biggest challenge will be downloading, processing and storing (indexing) the contents of the channels. You'll have to use SQLite directly. Transferring data from the channel torrent's on-disk format should be pretty straightforward, as the stuff is just concatenated and compressed channel entries. Try figuring it out from my code. If you'll be able to, that means my code is good enough :rofl:

[Tim-W]

@synctext I'll work on some improvements for the problem description based on your feedback. Also, I started investigating the RemoteQueryCommunity code and ported some parts of it to Kotlin, to be able to do keyword search and browse channels on Android. Is it a good next step to keep working on this RemoteQueryCommunity port to Kotlin for the coming days/sprint?

From now on this will be the static link to my msc thesis report/dissertation: https://github.com/Tim-W/msc-thesis/blob/master/report.pdf

[synctext]

yes, good direction and solid target: operational keyword search on the wild deployed community.