search

lightning-sprinkle / master-thesis

Automatic Universal In-Browser Payments: an alternative model to web advertising
3 stars 0 forks source link

Meeting Notes / Project Diary #1

Open landgenoot opened 5 years ago

landgenoot commented 5 years ago

https://hackernoon.com/datx-a-blockchain-powered-decentralized-advertising-ecosystem-71fb944b02ea

https://www.petsymposium.org/2011/papers/hotpets11-final3Bilenko.pdf http://adresearch.mpi-sws.org/privad-performance.pdf

Click fraud

landgenoot commented 5 years ago

Profile will be stored on the client side. This makes it possible to gather more relevant details. How to transfer the profile to the advertiser without revealing too much details. This makes it possible to deliver ads which are more relevant, so that they will make more revenue.

Ways to

Exposé,

landgenoot commented 5 years ago

image

landgenoot commented 4 years ago

https://www.adex.network Is not really decentral, but uses the Ethereum network for transparency. Implements microtransactions. Business model not clear?

https://adbank.network/ Features almost everything discussed. Business model seems to be the ICO. Should be live by the end of the year. Total supply of tokens around 5,5 million EUR The problem with this system is that it is not academic. All software seems to be open source, but the research that is done to come up with this product stays proprietary. This makes it hard to extend or improve the system.

landgenoot commented 4 years ago

image

landgenoot commented 4 years ago

https://stackoverflow.com/a/57698377

landgenoot commented 4 years ago

image

landgenoot commented 4 years ago

Research proposal as expected. Graduate seminar at 6.11 Pagerank algorithm for reputation

Presentation:

https://www.snet.tu-berlin.de/fileadmin/fg220/theses/master-snet_169.potx Payment methods is too much Evaluation: Simulate real life scenario Maybe time schedule

landgenoot commented 4 years ago

Feedback on slides:

Problem & statement

Define definitions Explain rolmodel Explain why current model is not effective New opportunities crypto Why unpaid content Kernproblem

Motivation

Explain problem with current systems,

Evaluation

Add performance

Related work

Maybe crypto paper

landgenoot commented 4 years ago

photo_2019-10-22_16-01-01

landgenoot commented 4 years ago

Some feedback from Prof. Dr. Küpper:

30 minute agressive monologue about:

Current considerations:

Questions to Philip:

landgenoot commented 4 years ago

Related longread: https://thecorrespondent.com/100/the-new-dot-com-bubble-is-here-its-called-online-advertising/13228924500-22d5fd24

landgenoot commented 4 years ago

20191114_135759

landgenoot commented 4 years ago
landgenoot commented 4 years ago

photo_2019-12-02_14-47-33

landgenoot commented 4 years ago

Discuss with Philip the possiblities of the Lightning protocoll. Some tests with it turned out to be really successful.

The problem is that there is no web based implementation of the lightning protocoll (yet). So for now we will use the node-launcher project which will run a lightning node on the computer of the user. This client is accessible via a REST API and features a lightning wallet.

In this new approach, we will stick to the existing infrastructure, but our javascript library will talk to the node-launcher instead.

This makes it possible to build a fully working prototype, however, we need to drop the zeroconf idea and turn it into a 0.5 conf prototype.

landgenoot commented 4 years ago

Did some further research on the Lightning protocol. Problem with the protocol is that it is not possible to send any amount to a wallet directly: an invoice should be issued first. This will make the process a lot more complicated, and results in a couple of extra request.

Current decision. Generate the invoice on the fly using the macaron of the publisher. https://www.npmjs.com/package/@lntools/invoice

landgenoot commented 4 years ago

https://github.com/lightningnetwork/lnd/pull/3795

Invoiceless payments.

landgenoot commented 4 years ago

Coming feature to LND

Accounting-based macaroons, which can make an instance of lnd act almost like a bank for apps: for example, an app that pays to consume APIs whose budget is limited to the money it receives by providing an API/service

landgenoot commented 4 years ago

Discussion points Philip:

landgenoot commented 4 years ago

The fact that this prototype will work with the bitcoin mainnet is a major advantage. Will setup multiple VirtualBoxes to demonstrate the system.

Plugin or website -> none of them. Decided that we will built upon the node-launcher system and just expose an arbitrary http port in order to communicate with the publisher.

image

landgenoot commented 4 years ago

image

Google Scholar just scrapes PDF's on github for some reason.

image Not sure why Google thinks it is a PhD-thesis.

landgenoot commented 4 years ago

First payment with --keysend succesful. Next step will be the implementation of the webservice ezgif-3-d6d3665af2a7

./lncli sendpayment --keysend --dest=027d2456f6d4aaf27873b68b7717c8137aaa8043d687a2113b916a5016e9a880e9 --amt=10 --final_cltv_delta=40
landgenoot commented 4 years ago

Problem I

Discussed the problem with keysend and gRPC. We can use the standard CLI as a fallback.

Possible solution to prevent batch domainname fraud could be the mandatory use of real SSL certificates (non let's encrypt).

landgenoot commented 4 years ago

Further research on certificates: There is a difference between domain-, organization and extended certificates.

Domain is like, Letsencrypt only validates that someone is the owner of the domain name. Organization Validated SSL Certificates require that the organization is registered in some governmental database, which is perfect for this cause.

Certificate validation can be performed easily by doing a cURL request from Python.

landgenoot commented 4 years ago

Problem I with gRPC solved. Turned out that the preimage can be created by generating 32 random bytes and the payment_hash is just a hash of the preimage.

Now a simple flask web server is running with one endpoint that supports requesting 10 satoshis

landgenoot commented 4 years ago

Questions to Philip:

  1. What kind of license should I use for the project?
  2. Copyright is TUB, right?
landgenoot commented 4 years ago

iightning-sprinkle-server is now working with the adjusted reward system.

Reward system

Set a max_hourly_amount, e.g. 40 satoshis. The first publisher that requests a payment gets 25% of 40, so 10 satoshis (€ 0.00078591). 30 satoshis are left in the bucket, so the next publisher gets 25% of 30, which is 7 satoshis (€0.00055013). And so on.

The bucket is filled every minute with the max_hourly_amount/60 till max_hourly_amount is reached.

This system makes it impossible to steal all your coins, but only up to the max_hourly_amount. Another possible security measure might be to limit the payment requests to 1 per hour per domain name. However, it is fairly easy to register a large amount of domain names in order to attack this. Futhermore, the organization SSL certificate requirement can be implemented as discussed here.

Localhost and port number

The webservice in bind to 127.0.0.1 in order to make sure it can only be accessed from the local machine and is bound to port number 283733 which is randomized by me typing blind on the number row.

Next steps

The next step will be to implement the publisher-library, which will be written in vanilla JS, in order to keep it small and it won't affect the page speed that much.

landgenoot commented 4 years ago

Problem II

Current problem has to do with mixed content. We are not allowed to request our http service, if the publisher domain is on https.

There might be a couple of possible workarounds:

  1. Let the user thrust our self signed certificate. (not user friendly)
  2. Re-use the certificate from lnd which is already trusted (user should have trusted the lightning cert)
  3. Register a domainname which points to 127.0.0.1 and publish the private key, so that every instance of the service can use the same certificate (really hacky)
  4. Only support http for publishers for now (not really 2020-proof)
  5. Generate a certificate and install it to the users' computer (is this possible without sudo rights?)
  6. Thanks to this article about zoom, who is using a similiar trick. You can still load an image from localhost and mixed content. We can reply a status by using several image dimensions. (hacky, but user friendly)

I have verified the trick from number 6 on github pages, which works, so that we don't have to setup a separate server and everything can stay on github.

landgenoot commented 4 years ago

Attack vector I

Right now the system relies on an image that is requested on localhost. However, someone might spam an embedded image of the url all over the internet in order to get paid themselves, not the publisher.

Possible solution A

Require the publisher to register a subdomain containing the pubkey, e.g. spiegel.de has to register: 027d2456f6d4aaf27873b68b7717c8137aaa8043d687a2113b916a5016e9a880e9.spiegel.de as a subdomain, which should be used in order to make the request. By using this hacky workaround, it is not possible to embed an image on a public forum that requests the payment.

Possible solution B

Add a TXT record to the DNS which contains the pubkey. Therefore an empty request is enough and Python can find out the pubkey using a Domain Information Groper. This limits the authorization to request a payment to domain-owners.

A single TXT record has a limit of 255-bytes or characters in a single string

DNS records can easily be fetched using: http://www.dnspython.org/examples.html

landgenoot commented 4 years ago

In order to do some further testing, i created https://publisher.landgenoot.com with the following DNS records:

publisher.landgenoot.com. 3600  IN  A   185.199.108.153
publisher.landgenoot.com. 3600  IN  A   185.199.109.153
publisher.landgenoot.com. 3600  IN  A   185.199.110.153
publisher.landgenoot.com. 3600  IN  A   185.199.111.153
publisher.landgenoot.com. 3600  IN  TXT "lnd-pubkey=027d2456f6d4aaf27873b68b7717c8137aaa8043d687a2113b916a5016e9a880e9"

However, Github is complaining that I should use a CNAME instead, but a CNAME cannot be combined with a TXT record: https://serverfault.com/questions/834320/adding-both-cname-and-txt-dns-records-for-one-subdomain

Sorry, @github

landgenoot commented 4 years ago

Feedback Philip:

landgenoot commented 4 years ago

http://localhost:28373/request-payment/027d2456f6d4aaf27873b68b7717c8137aaa8043d687a2113b916a5016e9a880e9

landgenoot commented 4 years ago

Problem III

Referer is not sent if the publisher's website is accessed over https. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer

Edit: solved: https://developer.mozilla.org/en-US/docs/Web/API/HTMLImageElement/referrerPolicy

landgenoot commented 4 years ago

Just created the certificate checker. Turns out that the type of certificate is stored in the Certificate Policies extension.

If there is a '2.23.140.1.2.2' or '2.23.140.1.1' property present, we are dealing with an OV or EV certificate.

This is now embedded in the isOrganization('https://www.example.com') function, which also supports SNI.

landgenoot commented 4 years ago

Last night I was thinking about the approval of the payment.

Right now: I would go for something like this:

landgenoot commented 4 years ago

Attack vector II

Clickjacking: Iframes are vulnerable to clickjacking.

Possible solution. Show the message from Python with QT, however this makes the experience less seamless.

landgenoot commented 4 years ago

Just requested a real Google Adsense account in order to create a working example.

It seems to be possible to hide adsense in a later stage: https://stackoverflow.com/questions/22410370/remove-adsense-on-mobile

landgenoot commented 4 years ago

Now we are communicating via an iframe, it might also be possible to communicate via messages, which makes the hack from Problem II not needed anymore.

myIframe.contentWindow.postMessage('hello', '*');
window.onmessage = function(e){
    if (e.data == 'hello') {
        alert('It works!');
    }
};
landgenoot commented 4 years ago

Setup ads on the example page. However somehow my browser is also blocking adsense with my adblocker disabled.

In the VM it works perfectly. Never been so happy to see an ad. Fresh VM only some traffic to Github.com, so that must be a lonely nerd looking for a girlfriend... :/

image

Next step is checking if we can disable the ads after pageload.

landgenoot commented 4 years ago 
$('.adsbygoogle').remove()

Seems to do the trick

Next step is to setup an example publisher website. Considering Wordpress or just some static HTML.

landgenoot commented 4 years ago

Stumbled upon a nice template, but it was not licensed. Asked the owner for permission: https://github.com/leogodoyllg/News-Website-Template/issues/1

Later on, I found the original template, which has a MIT license: https://github.com/imshashikantdev/WTFNews

Will go with the WTFNews template, thanks @imshashikantdev

landgenoot commented 4 years ago

Packed homepage full with ads, thanks @google

https://publisher.landgenoot.com

image

landgenoot commented 4 years ago

Problem IV

Chrome does block mixed content, which means we cannot load an iframe with localhost embedded. Everything works fine in Firefox

Possible solutions:

  1. Don't support chrome for now. Lame
  2. Load a new window (like authorizing a paypal payment)
  3. Use QT for the permission question
  4. Add self signed SSL cert to chrome

Will go for the new window now. Seems to work fine.

function requestPermisison() {
    const y = window.top.outerHeight / 2 + window.top.screenY - ( 500 / 2);
    const x = window.top.outerWidth / 2 + window.top.screenX - ( 400 / 2);
    return window.open('http://localhost:28373/request-permission', 'Lightning-Sprinkle', 'toolbar=no, location=no, directories=no, status=no, menubar=no, scrollbars=no, resizable=no, copyhistory=no, noreferrer=no, width='+400+', height='+500+', top='+y+', left='+x);
}

and

<meta name="referrer" content="origin">

In order to find out the hostname of the publisher.

Next steps:

landgenoot commented 4 years ago

https://github.com/lightning-sprinkle/lightning-app

Seems to be much more easy to use, because it does use neutrino as a bitcoin node, so that a real btc node is not needed anymore, if I understand it correctly.

Let's see if we can translate everything that we have in Python to JS and bundle it nice to an appimage and .exe

landgenoot commented 4 years ago

Wil check if we can run an express app inside the lightning-app. If yes, will discuss it will @philip-raschke next week.

Then it will just be a matter of translating the flask application to express.

Edit: Koa runs perfectly inside electron. Now checking if it still works when packed into an appimage Edit2: It works perfectly packed. So this gives us a great advantage

landgenoot commented 4 years ago

Right now, I will focus on the the JS lib for the publisher. So that we have a working prototype. Maybe find some time to work on the koa version the rest of the week.

landgenoot commented 4 years ago

The system in action:

ezgif-3-d9f66db9cae9

landgenoot commented 4 years ago

https://gist.github.com/bellbind/f65f78a35bbbd4917a8ae4a6b18c5012

Electron supports apps in tray, so this makes it even better.

landgenoot commented 4 years ago

image Now managed to setup my own electron tray application and translated the cert.py and dns.py libs to js. Everything TDD, which is nice, I think.

The tray application now also spins up an lnd service, but does not interact with it.

landgenoot commented 4 years ago

i still think the electron tray application would be a cool proof of concept. However, is is a lot of work because you need to implement a complete wallet. Even things like setting it up an presenting the recovery passphrase are required to run a lightning node.

One possible solution would be to just fork the lightning-app and bake it all into that app. But that means that you need to run an extra app.

Right now I am exploring the possibilities to kind of link the tray application to the lightning-app so that they can work both simultaneously with the same wallet.