search

OdyseeTeam / chainquery

Chainquery parses and syncs the LBRY blockchain data into structured SQL
https://lbry.tech
MIT License
2.26k stars 42 forks source link

Put subscribe count into ChainQuery #127

Closed melroy89 closed 4 years ago

melroy89 commented 5 years ago

Hi,

It would be nice if the number of subscribers (eg. subscribe_count) are put into the claim table, so I can order by the nr. of subscribers instead of relying on 3rd party API's like https://api.lbry.com/subscription/sub_count...

Meaning, this would be possible in the future:

SELECT name, claim_id FROM claim WHERE claim_type = 2 ORDER BY subscribe_count ASC LIMIT 10

Regards, Melroy van den Berg

eggplantbren commented 5 years ago

I doubt subscriptions will ever be on-chain. What use would there be?

IMHO a better solution would be to set up a MySQL database that exposes the API data in a more convenient way.

EDIT: After chatting with danger89, turns out that's what he meant - have the API stuff also stored in the same DB.

melroy89 commented 5 years ago

I doubt subscriptions will ever be on-chain. What use would there be?

So I could do the query above, that is why.

Another reason would be to base the trending also on views (both support/tips and views) if view count is also added to chain query!

kauffj commented 5 years ago

@danger89 I think we all agree that it would be awesome if this info was somehow on chain.

The problem is there is a negative incentive for a user to spend their currency to write data about their views and subscriptions.

kauffj commented 5 years ago

I'm going to close this. I'm very interested in this being possible but it seems basically impossible for the incentives to exist (as well as a bunch of other problems with accuracy, etc.).

melroy89 commented 5 years ago

Too bad you think about it that way.

I personally like more transparency, open-source and decentralized. The data is already in the blockchain, but hard to fetch it. On other services (uhm eg. YouTube) you can already easily see the number of subscribers/views, which is transparent for both the content creator as well as the users.

tiger5226 commented 5 years ago

Anything we have on the blockchain can be put into chainquery. Unfortunately, views are not on the blockchain. The blockchain is used as a catalog for the metadata of claims. The data of the claim is stored on the p2p network called lbrynet. The actual tracking of the views ( if data sharing is enabled ) comes from the app itself. The is open source software but it leverages internal apis.

What is interesting and maybe applicable is that we have a new comments feature that is decentralized and also unique per user. A mechanism like that could be used to publish views and subscribers to the data network. @kauffj what do you think? I am not entirely in the loop on the comments feature but I would assume these mechanisms exist.

kauffj commented 5 years ago

I've re-opened this because I think @danger89 misunderstood my point and/or I did not make my point clearly.

We love transparency and being open-source. We're not avoiding putting view data in the blockchain because we don't want to for some reason, we're not putting it there because we don't think it's possible to put it there.

YouTube can track view counts with near perfect fidelity because every view accesses a server they own and control.

Consider a view on a file published to LBRY for free. The user resolves that URL on a wallet server that we do not run and then accesses the data from peers that also are not us.

How can we force knowledge of this view, or force the user to write it to the blockchain? Even if we strongly encouraged it, why would a user want to spend money to record this view on the blockchain, when the content is free?

The best answer we have atm is:

Does this make sense @danger89 ?

eggplantbren commented 5 years ago

I have a question, a comment, and a cliché. That was the cliché. The question is: if a user turns off analytics sharing in the app for privacy reasons, do their views and subscriptions no longer count in the numbers I get from the API? The comment is: I think all Melroy really wants is a way to query the views/subs/other data available through the API but using MySQL. Could that data be put in a MySQL DB?

On Wed, Sep 11, 2019 at 9:56 AM Jeremy Kauffman notifications@github.com wrote:

I've re-opened this because I think @danger89 https://github.com/danger89 misunderstood my point and/or I did not make my point clearly.

We love transparency and being open-source. We're not avoiding putting view data in the blockchain because we don't want to for some reason, we're not putting it there because we don't think it's possible to put it there.

YouTube can track view counts with near perfect fidelity because every view accesses a server they own and control.

Consider a view on a file published to LBRY for free. The user resolves that URL on a wallet server that we do not run and then accesses the data from peers that also are not us.

How can we force knowledge of this view, or force the user to write it tot he blockchain? Even if we strongly encouraged it, why would a user want to spend money to record this view on the blockchain, when the content is free?

The best answer we have atm is:

  • Call home to one of our servers for users who opt-in to tracking
  • Attempt to monitor some portion of the data network and then extrapolate views from the portion of network traffic we observe (e.g. if we observe 5% of network traffic then multiply views by 20)

Does this make sense @danger89 https://github.com/danger89 ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lbryio/chainquery/issues/127?email_source=notifications&email_token=AAMBKOX27A3JXWQOW27F5WLQJAJYRA5CNFSM4IUU6PA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6MTYQA#issuecomment-530136128, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMBKOUSEK6BFJWTRYWCKS3QJAJYRANCNFSM4IUU6PAQ .

-- Dr Brendon J. Brewer Department of Statistics, The University of Auckland, New Zealand Ph: +64 27 500 1336 Web: https://www.brendonbrewer.com/

tiger5226 commented 5 years ago

Yes, it's possible to have it in a database. There is work involved in doing that though and I am not sure this is a high priority thing since it's already available through the API. One could write a script to run through every claim_id and gather this data into a database.

For the question, if a user turns off analytics then that's correct, none of that data is captured.