search

TeamNewPipe / NewPipeExtractor

NewPipe's core library for extracting data from streaming sites
GNU General Public License v3.0
1.4k stars 420 forks source link

D.tube support #420

Open Serkan-devel opened 6 years ago

Serkan-devel commented 6 years ago

D.tube is a decentralized video streaming platform, backed by the IPFS Network and the STEEM Blockchain.

IPFS is used for torrent-style hosting while STEEM is used for decentralized account management, comments, ratings and more. Everything there is open source but didn't get the recognition which it deserves

theScrabi commented 6 years ago

Yes pls. I am currently working on a tutorial about how to write own services for the NewPipeExtractor. This way everyone with simple java and webexperience can write support for any streaming service. Including d.tube :D

trymeouteh commented 6 years ago

D.Tube and Bitchute in newpip would be great. I know DTube and Bitchute are working on their own apps but to haven then all in one app would be amazing.

theScrabi commented 6 years ago

I'm currently working on a tutorial for 3. party developer to add support for certain services. It's not yet ready but if you want to give it a shot take a look here :D https://teamnewpipe.github.io/tutorial/

FlorianSteenbuck commented 6 years ago

Working on a D.Tube Implementation and some better URL parsing. Not finished yet, but if I get my mail account recovered and a approved steemit account I can create a pull request next week. Its a little bit tricky because I think the user should select which ipfs gateways they want to use. Maybe if we get the d.tube implementation their could be a possibility to more easily implement other steemit services, because dtube strongly depends on the steemit RPC Api and the asksteemit rest Api. And other service like Dlive also strongly depending on the steemit RPC Api.

theScrabi commented 6 years ago

Did you try to parse the dtube website or did you use the APi?

For now I think its ok if the user can't switch the gateway. However you are right, it was better if he could. We need to find a way to pass user defined parameter to the service. This way we could also allow services which need support for login for example spodify.

FlorianSteenbuck commented 6 years ago

The Website is a Javascript Client for videos that get published with DTube on steemit.

Steemit looks somehow like ipfs with a gateway (api.steemit.com), that every steemit based application is using, but with a extra frontend with crftoken that is required by the steemit.com/api/v1, but even this frontend using the api (api.steemit.com).

You could use rhino and a Virtual DOM envirment to parse the website or parse the steemit frontend with crftokens on it. And that is not what I have done.

Their is no api key or client id like in SoundCloud. I just replace the JavaScript logic from DTube with Java logic from NewPipe. Anyway the third party asksteemit.com DTube using for searching and for suggestions. And the "official"/application api (api.steemit.com) for the everything else. Their is no "no script client" where I can parse it from. The APIs that are in usage got no request visible request limits. And if they got in future their can tricked by Just using identical request or parsing the website. DTube only got a settings in json format, translations in json, one weird looking sockjs/info?cb=xxxxx endpoint that just response a empty json object, all what is needed to setup a static html website + JavaScript + emb.d.tube for everything that is needed for the player, snap1.d.tube for saving the thumbnails just a ipfs gateway, dmca.d.tube for handle dmca claims return as api a object with one key ("dmca") and and int value 0 or 1. The only thing I am using from this endpoints are the settings and with that the ipfs gateways.

They claim to save videos but I never seen that also they not using a extra gateway for display the video its just the official ipfs gateway. So I think it is quit important for users to setup own ipfs gateways that do not delete their videos and do not get censored.

A Steemit Applications are united by just airdrops of power in the Steemit system, that get controled by its devs. A Steemit Application is somehow centralized over the steemit plattform.

Serkan-devel commented 6 years ago

Where is actually the tutorial for implementing it?

TobiGr commented 6 years ago

The tutorial has been renamed to documentation: https://teamnewpipe.github.io/documentation

FlorianSteenbuck commented 6 years ago

Short Answer yes I am using the APIs but the app also using this APIs like I do.

Its not exactly the same way, but this is only because I want to write it in a light way with less requests.

theScrabi commented 6 years ago

Short Answer yes I am using the APIs but the app also using this APIs like I do.

muy bien

FlorianSteenbuck commented 6 years ago

My Current Pushed Version of the Implementation: https://github.com/FlorianSteenbuck/NewPipeExtractorDTube I got some trouble with splitting the whole project in mutiple projects and connect them over jitpack. It only needs a Dynamic Settings Implementation, some extending of the current downloader (dtube using head requests and post requests with body), the subscribtion import that is public and is the same request like the one that shows https://steemit.com/@floriandev/followed content. Uploading and Perform requires some keys.

The parsing part of my code also needs some fixes.

Also I have worked on some other thing and got some problems AFK in my life. One of this projects is a NewPipe Tracker that is based on checksums so the user never get injected data from peers. The other one is a complete python clone of the NewPipeExtractor. Both are basically not finished and are things that I need to work on, while fixing problems with my current life and doing even more stuff in the direction of IRC Protocols und Javascript (and DOM) Fixing things.

Somehow I need to clone myself for doing all this stuff.

theScrabi commented 6 years ago

@FlorianSteenbuck awesome, however you might want to hold back you're changes a bit, there where huge changes on the SearchEngine which will be deplored with the next version of the extractor. SearchEngine basically got removed in favor of a new SearchExtractor.

Somehow I need to clone myself for doing all this stuff.

Be careful not burn out.

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Serkan-devel commented 5 years ago

Why does this bot exist? Closing issues don't automatically resolve it

wuniversales commented 5 years ago

Great idea!!!

CypherpunkSamurai commented 3 years ago

@TobiGr @theScrabi Is it possible to provide a updated service template? I searched github and tested a few extractors i found. All of them are outdated and conflict with current NewPipeExtractor code.

I can extract DTube, Dailymotion API, but i'm not very skilled with large Java projects. It would be very helpfull if you guys can provide a sample service template :)