Add PeerTube service to NewPipe

ghost commented 6 years ago

[bounty 300€]

Hi, Is it possible to add PeerTube to NewPipe ? ;)

https://github.com/Chocobozzz/PeerTube Regards,

theScrabi commented 6 years ago

Yes it is, if you can add almost any service to NewPipe. If you want to give it a shot, take a look at our documentation. However doc is currently on work, so you might not find much yet.

FlorianSteenbuck commented 6 years ago

Absolute possible. PeerTube, is like DTube, but better, all data come from one api (even the default access key). Their is no preloading. And their is a only one path on their api /videos. The only problem is the optional login feature and the torrent protocol and even this can be skipped because their is also a direct video link (Yeah PeerTube is soo much wow p2p).

okias commented 6 years ago

(supportive post) as a long time user of NewPipe, few weeks ago I found SoundCloud support has been added. As I noticed that Blender channel has been blocked on Youtube and Blender guys started testing PeerTube - so, NewPipe support sounds really great!

ghost commented 6 years ago

Blender guys started testing PeerTube

For the record:

@ blender_org

Update: we are testing #PeerTube on one of our servers. Go to https://video.blender.org . More info: https://www.blender.org/media-exposure/youtube-blocks-blender-videos-worldwide/ #b3d https://pic.twitter.com/kBRKd6mip3

7:18 AM - 19 Jun 2018

ghost commented 6 years ago

Also, in latest youtube-dl added some improvements in it's PeerTube extractor with example for https://video.blender.org/*

https://github.com/rg3/youtube-dl/releases/tag/2018.06.19
- https://github.com/rg3/youtube-dl/issues/16733
- https://github.com/rg3/youtube-dl/commit/8b4b400aef83b233502ece7321ee84f6ab9e213e

dscottboggs commented 6 years ago

Is anyone working on this? I'd like to provide some assistance if any is needed.

theScrabi commented 6 years ago

@FlorianSteenbuck said he was working on d.tube support, maybe he can give feedback on adding peertube support as well.

theScrabi commented 6 years ago

however please be aware that I will soon push these changes: https://github.com/TeamNewPipe/NewPipeExtractor/pull/94

theScrabi commented 6 years ago

By the way I may need help with #94. The changes on the extractor are more or less done, the only thing that is missing now is that the forntend needs to be made compatible with these.

FlorianSteenbuck commented 6 years ago

The best way of implementing this thing would be over the settings that going to be added with the DTube support. Therefore we need a mandatory attribute to a single setting attribute and we need to open up the add ServiceList for add custom sources, because everyone can host a own peertube instance.

Peertube Client

Their is no preloading on the peertube client. So no html parsing is required. Their is like DTube no private api key and no hard coded configs. So no javascript regex search or parsing required for get this working. Everything is working over a public API. Their is no other way then using this API, because the Javscript also using this.

Peertube API Endpoints

{host}/api/v1/config/

here is everything that the web client using. We only need the instance endpoint for the name of the Service and to show the prioritized kiosk. Maybe we could check the Server version for compatibility.

{
  "instance": {
    "name": "FramaTube",
    "shortDescription": "Framasoft instance",
    "defaultClientRoute": "/videos/recently-added",
    "defaultNSFWPolicy": "do_not_list",
    "customizations": {
      "javascript": "",
      "css": ""
    },
   "serverVersion": "1.0.0-beta.9"
   ...
}

Everthing else could be a cool feature, like licensing in the future.

{host}/api/v1/videos/{query-for-all-videos}

Everything for query all videos (kiosk). Recently Videos - {host}/api/v1/videos/?start=0&count=8&sort=-publishedAt Trending Videos - {host}/api/v1/videos/?start=0&count=12&sort=-views Local Videos - {host}/api/v1/videos/?start=0&count=12&sort=-publishedAt&filter=local Search For Video - {host}/api/v1/videos/search?start=0&count=12&sort=-publishedAt&search=Search+For+Video find out by yourself why the count is so different and how they calc this this way. This is the response to requests on this endpoint:

{
  "total": 1349,
  "data": [
    {
      "id": 1468,
      "uuid": "839345ed-26b5-4503-99b1-2cd196356019",
      "name": "03.08 Open source et brevets ",
      "category": {
        "id": 13,
        "label": "Education"
      },
      "licence": {
        "id": 2,
        "label": "Attribution - Share Alike"
      },
      "language": {
        "id": "fr",
        "label": "French"
      },
      "privacy": {
        "id": 1,
        "label": "Public"
      },
      "nsfw": false,
      "description": "Le brevet est un titre de propriété industrielle qui confère à son titulaire un monopole d’exploitation - c’est-à-dire un droit d’interdire toute exploitation par des tiers - temporaire (pendant au maximum 20 ans) - sur le territoire d’un État don...",
      "isLocal": true,
      "duration": 180,
      "views": 85,
      "likes": 1,
      "dislikes": 0,
      "thumbnailPath": "/static/thumbnails/839345ed-26b5-4503-99b1-2cd196356019.jpg",
      "previewPath": "/static/previews/839345ed-26b5-4503-99b1-2cd196356019.jpg",
      "embedPath": "/videos/embed/839345ed-26b5-4503-99b1-2cd196356019",
      "createdAt": "2018-05-24T13:00:32.780Z",
      "updatedAt": "2018-07-04T10:11:39.076Z",
      "publishedAt": "2018-05-24T13:02:20.518Z",
      "account": {
        "id": 3,
        "uuid": "0a635cad-dcdb-443f-b600-6c38ffaffe1f",
        "name": "framasoft",
        "displayName": "framasoft",
        "url": "https://framatube.org/accounts/framasoft",
        "host": "framatube.org",
        "avatar": {
          "path": "/static/avatars/f73876f5-1d45-4f8a-942a-d3d5d5ac5dc1.png",
          "createdAt": "2018-05-24T12:48:06.202Z",
          "updatedAt": "2018-05-24T12:48:06.202Z"
        }
      },
      "channel": {
        "id": 54,
        "uuid": "e4985792-98ca-49be-a1aa-bceecd1c8051",
        "name": "e4985792-98ca-49be-a1aa-bceecd1c8051",
        "displayName": "Mooc Comprendre l'Open Source",
        "url": "https://framatube.org/video-channels/e4985792-98ca-49be-a1aa-bceecd1c8051",
        "host": "framatube.org",
        "avatar": null
      }
    }
    ...
  ]
}

{host}/api/v1/videos/{uuid}

All video infos contains mutiple sources in torrent, magnet link and direct link format. And all other data that is required by NewPipe (no donation links, so searching in description is required). Some data are outsourced to other urls.

to be continued

Their is a lot of more stuff I could explain to the endpoints feel free to ask.

The Mystery of Bencode

Bencode is a combination of the delimiter : and the surrounding that starts with d,l or i and ends with e. If data is surrounded you should not use the : delimiter. This going to end up in that only string and bytes using the delimiter. Like in json there is no way to got a int as key in a dict. Everything is written char on char so its quit heavy to read complex or long bencode by yourself.

i is used for int

i1e

= 1

l is used for list

li1ei2ei3ee

= [1, 2, 3]

string and bytes are handled this way:

7:NewPipe

= "NewPipe" the 7 is the length of the string. 1 char = 1 byte, so it really do not matter if it is a string or bytearray. (But do not use encoding on this if you do not know that it is a string, this could end up in a mess)

d is used for dict

d7:NewPipeli1ei2ei3e4:tadae4:thisd2:is7:awesomeee

= {'this': {'is': 'awesome'}, 'NewPipe': [1, 2, 3, 'tada']}

.torrent Files

I can not read .torrent files but this python codes helps a lot it take the torrent file and convert it into json. The encoding is the bencode. yield ouput is handled like a loop iterate so decode_item is going to be call the next mutiple times if needed (for dict and list). Their are mutiple implementation in mutiple languages out their for bencode without and with regex.

import re

def tokenize(text, match=re.compile("([idel])|(\d+):|(-?\d+)").match):
    i = 0
    while i < len(text):
        m = match(text, i)
        s = m.group(m.lastindex)
        i = m.end()
        if m.lastindex == 2:
            yield "s"
            yield text[i:i+int(s)]
            i = i + int(s)
        else:
            yield s

def decode_item(next, token):
    if token == "i":
        # integer: "i" value "e"
        data = int(next())
        if next() != "e":
            raise ValueError
    elif token == "s":
        # string: "s" value (virtual tokens)
        data = next()
    elif token == "l" or token == "d":
        # container: "l" (or "d") values "e"
        data = []
        tok = next()
        while tok != "e":
            data.append(decode_item(next, tok))
            tok = next()
        if token == "d":
            data = dict(zip(data[0::2], data[1::2]))
    else:
        raise ValueError
    return data

def decode(text):
    try:
        src = tokenize(text)
        data = decode_item(src.next, src.next())
        for token in src: # look for more tokens
            raise SyntaxError("trailing junk")
    except (AttributeError, ValueError, StopIteration):
        raise SyntaxError("syntax error")
    return data

data = open("torrent.torrent", "rb").read()

torrent = decode(data)

print torrent

taken from http://effbot.org/zone/bencode.htm

But we got another problem with this code. The enpoint pieces seems to have a own thing to parse:

{
  "info": {
    "length": 7250102,
    "piece length": 16384,
    "name": "The Daily Dweebs 720p.mp4",
    "pieces": "\xb6\xef#\xd7\xee\xf83\x0f\x1a\xb3\xbdY\xf5\x88\x16\x96\xc0\xdfc\xf4\xe9]\xfe\x9a\xf6\x937\xca\xf5\xa4\x80\xde\x90\xfc\xc3\xe7\x8d\xe7\x96I\xbbuG \xeb\xa1\xfdf\x1a\x84\x8f\xf5F0F\x01\xb5\x00DB\xd3No\xf3\xd0\x95U\xecdZ\x1fx\x91ug\xf9\x8e\t\xf6\xfa\xe5\xfb\x03\xa2\xe8\xf7\xb5\x9d\x00\x98^\nH\xfc\x8eD\xb32<\x96\x89D\xc7\x0f\xe0a$V\xa2\x87\xb9\x0e=f\xe5\xd6\\\x15h;-\x10\x8aQC\xcf\xec\x03\xe4\xa7s\x98n\xe1\xea\xadI0]\xfeL$;\xf0 \xddE>\xcd\x12\xb0\xbd\xf5Yg\x1c\xc8&E\x86\xe4\x89\x9d\x82\xef\x01\xfb|\xac\x9c$\x8b\xd0s\\+/x!\xa1S\x82V.\xb3\x89Q\xf5\'\xfb\xa4\xa5\xe8\tY3\x1ax\xfe\xac\xe3\xb3b)\xef\xc0\x8b\xe6\x00\x91\xc9\x16\xf7\x1fc\x88+\xbf\xcb\xf4\x88\xc0jIv\x93U\x89\xa0\x80\xd5]\xf8GOM\x8c\x18>\xe4\xf2\x81\xa5\'\x8a\xaf\xa8\x14\xe3P\xe0\x85-J\x0e7\xf2\xd2\xda\x0bj\xda\xd2\xed:\xf4\xb9\x801X\x0cvGr\x11\xc0\x18\xbagS\xad\xcb\xbf&\xf2\x8a\xb8\xf9\xb3\xb8\xe3*7jg\x83\xcc\xa0>>\xe9\xef\x92\x06\x1f\x00?g\xab\xf4{\xb6\x83\x88\t\x0c\x84\x85\xc3][O\xee\xc60:\xbd\xbd\xe3i#\xd8\xb1s\x89I\x98\xc6<\x82\xce\t\xcd=qT[}\xc02d\xba\xb8\xd3W\x89s<\x12p\xda\xfepJo\x85\xcfD\x86v\x13\x0e\x1e\xa85&b\xd0\xde\xe5\xcc_G\xba\xd9\xb0\xf5j\x1a\xa6\x86\xdb\x83\x94\xadxj\xb1\xa7\xdc\x98\xf7\xb8\xd2\xc1I\xfd\x99\xde\xb7\x81r\x12)\xa0\x00d\x83\x0f\x14\x15\x0b?^\xc0\x97\xf9(\x03Sx\tc\xe7R|\xccy\xe8\xba/\x03\xe2\xfe\xf6(\x0by\xc9&\xf7\x9e\x86\xb0`-\x8do\xb9\xd1\x90\xa9E\xb2\n\x1e\x88\xd1D2\x0f\xd2\xec"\x81\xf2]TqW\xfd\x88\x94\x13\x98\xf2\xd1\xdc\xb8,Y\x0c\xea}y\xe4A"\xd0\xda`\x06s\xb6\x80\xd3\xdb\x89D\xa5\x9c\x05\x87\xca\x82L+\xa9\x9bA\xbe\x92\x89\x07\xbdA]\x85\x1b\x99F\x93\xa2\xbdA\xfd\xaf\x10\xc4\xa8\xa7\xa2_\xfc7\xa7\xa5%m\xc2\xb7#u\x8f\xb58s\x81\xaaT\xd0+\xb8[~\xa9\xb9`\xc2\x8f\xfd\xe9\xbbp\x0cs\xe9\xb52\x9a\xf1\xa0\xa7*N\x13\x9d\xbb_\xa30\xad\xd9\n\xaa\xf8{\xbe\x82qzs\x92\x08r\xd2\xdcf\x8f\xab50\xcb\x88l\xc0\xf8u\xcf4\xfe\x91\xb3\xae\n|\xdfm[a\xfb\x8f\\E\x1d\x83\xee\xabS\x10\xf2\x07\xe9\x19\xfa\x10\xf8m\xc7\x8fqx}V\xe6\xd1\x834\xd6\xef\xd8Ih\x7fN\n\xff\x93m\xb3f`Fa\x98hP\xb0T\xfcST\xff\xa5\xcf\xe8g\xc80\x7f\xb6V\x0c\xe2\xd2\x1e\xc54\r.\x9e\xb6\xc80s\xde4d\xca\x91\xb2hP\xed\xf3\x03\xe19\xd2\x1c\xd0\x1d\xd3\x0f\xe2\xc17\x91\xc3V\xfa\xe6\x8e\xb6\xd6\xfda*ZoGv\xf0i\xfa\xc6x\'\x0b\x0c\x0f\'\xc4vo\x9f\xbd\xdf\x12\xd83MbG\xa8\x04\x87h\xef\x0c\xa4\xa4\xb4G\xdcMjV1\x92\xdd\xca\x89\xa8\x16f|\x19\xfc"\xe5\x91MH\x82\xadb~\x17\xcdA\x19\xf9E\xe29\xad\xfa1\xe4\xa4\xec\xb4\xe3\xd1\xca;\xde\x03K\xa5\x90\xa0\xe8\x03{\x9f\xe4C\xeaU\x8c\xbd\x89`\x8a\xb8Q!\xe7/\xc6\x10\x13\xf6J\xa0\xd2;H\n\x8d0\xbeN\xe52\x1e\xcb\xbb\xd8\xb9(\x98x_\x0f\x99\xf6\x8d\xa4Do\x7f\xb6,\x0e\xd9\xc3\xdc\xe1\x8a\xe1\xf0v\x08\xbb\xb3O\xe8\xef\x99\x8euo\xaa\xd3m\xe3\xb4\xb8\x11\x82\xee\xce}\x14#$\xefM\xe4%\x83\\\xd5\xd01\xc1\xce{\xd0\x80Tu`\xdeVL+\x98g5JtP\x80\x19h{\x12\xa3\xc1L\xabDN\xe1\x04\xbd&\x8b\x9d\xcf_\x00w\xcc\xcbu\x18\xf6\xfd\xd9\xff\n\xbcoS\xf05[\xc8\xec\x00\xf4\x7f\xd1\t\xdbv\ra\xf2\x00\x18:Y\x85\x7f\x97\xc6\xce\xd5S\\\xe2j0\xb3t\x12!\xa9n\xc5\xa6_\x057`&\xc8\xf2w\xa3\x99\'\x83\xd1\x86\xc6\x1b\xf1^S\\\xe8\xad\xb5\xb8\xcb\x05\\\xea^\xf6\xce\'q\xa9\xa6\xf8\x0e\xaf\x15\xabu\x0e\x85\xe7\xc9S\xd5\xd5~\x7f\xc8\x1b\xad\xd6\x1d\x9c\xda\xe7 \xdfkzy\x17\xddXV\x8evKd\xee\xa3\x14\rQ:9m\xfa\x99\x05\xfe\xf0\x87\x17\xaf#)\x1e\x1a\x85\xca\x9bTGC\xe5\xb1\xa9q\xe3\xe2I]\xfb\x0cy\xd7\xd8\x94\xdb{\xaf\xb9\xe6\xb6}\xbc7\x98Z\xae\xe5J\xda&_7"\xe65;9\x1c\x1a\x8d\xd3\xe4;\xc8\x8e\xc11P(\xd8HJ\xa0\xe7\xe9\xbc\xb7\x02\x96\xa6\x91\xdb\n\xd3i\xc1\xca\xf3\xc7\xef\x85\xdd\xd5*\x9bw\xfa\xce\xc8F\x7f\xae\xe1]y\xf2\r?=\xc2\x83[\x95\xff\x00n\xa5\xb5^\x0b\x1ai\xf1\x82)\x18\x10\xaf\xe2`(R\xb7\xb0\xefT\xe4]t\x8ea\xc58\xd3\xa3\xb4\xae\xe4\xe4\xe6\xc9@\xf7\xe4\x18u\xe2\x11qq,7\xa4g\xf4\x0c\xb8\xad\x11\xab\x8d\xce\xf1\xa0\x02a+\x9d\xb2\xb61\x80\x81/\x7f<oyw\x12ANE\xec\x11\x1c\x8a\xffn\xca\x1c\xc1J\x17q\x8fb\xc1u\xdc\xc5A[\xe9\xaaU\xaf\xdau\x1d\xb29\x98[\xad\xcb\x0e\x1f\xf3\x9cj\xb2\x01\xe7o\x82\xec\x00t\x9d_\xcas\x85\xb3\tT\xd7\xe0\xa0\x80\x11\x96\x10\x03\x8c!\x0f\x8f\x0f\xf4!@@u\xb9vj~\xc2 \xf6\xd2\xdd~\x14\x0eh\xf5a/\xe5>\x9b7\x8c\x9b!\xef\xa1\x8c\xbbpK"\xe6a\r\xc1C\x13\xa6\xac\x8557xS\x13\xea+\x8f\xb3\xf0\xd3.\xbf2\xc2\xac\xe4\xc0\x9f\x9bA\xd4\x15T{\x03Ei\xea\x91a\xaeRU2\x06\x01\x9fa\xf7a\x05\xf1\xe7\xc6GY3\x8e\xa7\xe5\xc4\x9daD\xa2\x9ac\x12\xf3\x83+\xc2\xe31M\x84\xef\x8fJ\xe0\xd1\xc7\x03:\xaei\x9b\xabI\x16;\xc6\x9b\xe0\xed\x1c\xd3@u\xb7\xd5\xc8:\xda\x1d\xc12\xb6\xabxp\xbf\x91\xfa\x9eX\x98\xaa\xe8\x0b\x85>"\xdc\xc8`\xf8"NW\x84\x07##\xf7\xbf|l\xdd\x9d\xd1F\\\xd18\x92\xa0\xeb\x1fJV\x8aE2\xaf\xbb\xcfz@f\xf6\xe1\xc6H\x8ah\xeb\xb0q\x9c\xa8WR:\xc5\xe3\x12\x1a,k\xbe\xdfqe\xc4\x1ad\x16\xbb\x12G\xe8"0!X\xcfa\xf0\xf9\x06\xb9L\xb2)$\xaf\x02\x06\x83\x81\x1f1\xf88\xd4e\x9c\xe7-CL\x91&\x05\xedM\xc6\xdc\xb4V!\xaf\xd6\xcc\xaf\xbc\xa1f\xa0\xf8\xa4E\xb7\xda\xca\xf3Z@C}\xf7\x9cw\x99\x956y\x8d\xf5\xcex\xc6(\x91T\x1e\x1f\xc1z\xc0\r)[\x9188\x0e\xb1\xf9\xf1I\xdd\xe4\x8bSD\x05\x8cUw\xc7\x1c1V4\xa1@\x05q\xe0\xf8\xd0BkG\x9c\x91\x1a\xb1\x19u\xfd\xb7\x8b\xc9\xe8\x08C\xae~\x8d\x16\x83`\x95]F\x80\x92yo!\x86p\xdf\x1d\x1b?F\xfa\xb0\xec9\xa1#\x83\x8e\xe06lE\x9dR\xee\xbe\xff\x12\x90$A\x1d\xd4\x0cW\xea\x826\xaf\xf4\x1cG\xf7%@\xf7\xe1\x14n@?\x9b)\x19\xbb\xd6\xcf\xbc8"\xfc|\x00\x15\xcdL\xaf\n\x01\x13Zh\xf5\x8c\xb42\x07d\xc8\xd6\x97\xa8v\x91#\tz\xd5\x88\x91g\xa5 N9E\x89\x81\xf7\x9c\xa6`\x08\xc8\x96\xb1\x9cg\x8b;\x193!sj\x88\x82\xfav\xa4\x1bE\x1f\xd7-\xce\xb9\xb4rl\x94\xc5$ILl\xb2\x9b^\x03\xaf\xb2\xe9\xea\xd84\xc5\x86\x900\x1e|(\xd2\xf2\xe4OAp\x12V\x141\\\x04=`N\x99\xe2\x96B\xbc`N\xb1!\xcer\t\xc1\xb5C\x07X\x95\xad@\xc6\x83\x81i\x01\xab\xeb\x18\x96Uwh\xadZ-\x11\x96m\xfb6\xb4\xaa\x13\xe2\x97L\xd4\xb0d\xdb\x1a\xde\x18z\x9bhy&q\x81}hSX\x18,\x8bA\x83\xb5\xe8Q,R\xea\xd6{\x12\xb7\x91\x1f\x8d\xe0\xc1\xb1`\xd3Q\xe1\x05V\xc1\x03\x1e\x95\xccR3\xa4[\xa4\x14\xb0\xda\xa2c\xa9\x9c\xaeE=\x1a\xd7T+\x87\xd4"\x18d\xd5\xfaaw\xe8>\xaaC\xd9\xbc\xf2\x18\xc3\xce\x90S\xcf\'Q\x8cL\x7f\x8b\x8f\xff\xbeN\x94\x80\x19\xc3\xef\xf2t\xf3\xeb(\xb7x\n3#\x11K\xa3\x0e\xc3\xcd\xbdHG \x99c\x8b\x8d\xa8\xb0\xf6\xbc\x92\x9d#\xc94\xc2*\x7f]\xe5\xce\x8d;\xc1\xd5\xf9\x7fN\xfa\x8b\xefV\x82\xbbS<(\x0c@\xbc"\xa5\xa6-\x844<\x9c\xe2x\xbd\x82N\xbey\x95Y\x04\xff\xc70\xd4\'\x93%\xd6\x1b\x08\xde\xa9\xf7\xa0m0\xb6\x84\xd7\xee>\x01B"\x1c\x9dqN\xf2\xebE\xed\xc7\xdc8\xdd\xec\x9a\x97S=\xa8\xd8YB\x91\xec\xd3\x11\xda\xa6p\x86\xee\rq\xa2i\x17!\x8c\xb4\x7f\xe1S\xf7\xc4q_\xf8\x8eQ\x8b\xa8\xb3a\xc2\x9f\x8a\xf3u\xac\x8a]%\xe8\x9e*Y\xd1\xa6\xbe1$\xa9\xc8\x1c\xc4?\xda\xc2\xef\xbb\xc2\x96\xf7Kw,2\x87\xb0o\x98\xb9f\xf0~<\xa3\x8fe\xca\xab\xb7$s\xa7\xdd\x82*G\xbc\x04\xc0\xecg\xa1\xf6\xac\xbe\x81E1\xa9\xb7o\xfa\xdcx\x0c:\x9a"\xb4\xcanc\xb9\x02\xcb\xca\n*\x81A\x13/\xd7EFXv~@\xfb\x94\x89I\x07\xfcx&\x07w\xb0i\xe7o\x858\xc3z\xb3e\xee\xcc\xd7@\x92\x96\x16\x90\xf7\xc7\xf7\xd7\xdbO>\xd5\xd0`\xf4Nk\xc6\xe6\xa4E\xcf\xeaQ\xf4\xd4\x11\xae\x929o7\xf9*u\x8dKU\x0c\x88n\x7f\xe2K\x86\xb6\xb9B\xbex(\x1c\x06*\x08\n\xe5\xb3(\x08\x81&\xa8\x14\xc9Q{V\x0cV\xee\xd8\x07\x90\x91\xb6\xe3\xa5t\xed7\x96\t^\x8c,\x8b\xd0\xc7or\xbc\x0cJ\xd8\xdf\xc5c\xa5\xd0=\x88\xf2J\xf4\xc2\x868\xfe)\xeb`\xc2\xa2\x95\xb5\xd9\xde\xd0\x1f\x13\xccyq\xf5\xaaV={2\xef\x1aj\x90\xa6\x0e\xfc\xe6\xf7\x80F\xbc\xf0H\x12\xed3\xe9\xa3H\x01N\xa8q\xe4S\xb5y\x01\x9b\x8dRK\x13-\xa7\xc3v \xe9\xd5"\xea>\xbe\x1b\xcf\x90\x1e\xff\xac\x05\xa8\x05X$r\xad%:\x00>\xc4\xd0b%\x97g\x9cb=>\x02\xb7t\x01\x8c\xf1\'4\xbf%i\xbc6\x83u\\RNe\xbf\x80[\x97\xad%\xf6\x15\xbf\xf0\x81\xba\xe9\xef\t,\xd4^\xf5P34*\xc9\x17\xbe\x00y\x05\x14\x7f\xab\xf9\xc9\xa3c\xf7\x06\xe6\xe5/\xb5RF\x99\xde\xb8O\xd0B\x07\xd7\xfdet\x80\x99\x93\x1b\xc4\xd0\xd1\xd3\xfe\xaa*~\xa3\x18Z\x1b\x06\x8f\xc3i\xe5\xef\x8b\x00i\xd0\xab\xd6L\xa0N\xbcl\xa7\xbb\x04s(\xa9l\xdc\xdb\xd7\x91y=\x92\x14\xe7.\xe6\x15%\x80~\xb9\xd5&\xf5$8\x17\xaeFX\x80K\xd9\x18&\x05\xe5\xfevD\xe3?,\xcb\tC\x12)L\xa7@\x8b\xa9 *\xfa\xce\xc1*#\xd6-\'iN\x8f,\xb7\xa4\x16\xfa\xeb\'on\x01\xd3f\xd4"\x0f\xdc\x1a\x9e\x80\xc8\x8a~h\xc4N[e\xb7\xb0\xd8\xa5\xe5\xbe\xdd\xa0\xda\xb2#\xfe\x8a\xb1-\x14\xf3\xbc\x13N\xd0K~\xcc\xda\x06\x90G\'\xd2\xb3\xc5.\x14\xda$\x01|~\xce\xdc!\xa7\xb3\x87\xed\xbf\x9c\xcc\x04\xbe\xbf\xb0\xe6gk\xf3F\xe8\x14\x81G\xf1\xdeD\xe6V-\x002\n\x97\xfc\xfb:\x9e:\xcc\xb7\x19\x1bV\x8fp\x9b~\xab\x07\x0b\x04\x03\x18\xc2\xd2$f[\x96 \xaa\xe0T\xe6\x95\xd1\xe6\xdf\xcd\xc8!\xa7\xe3\x12y\xdf\x9c\x8b\xad\x86\x9b\xfd\xe8\x9f\x18\x03Z?\x1eU\xe5\x8a#n:!t\x16Q\xa0B\xc8~\xe6\x91\xa0\xa8\\Jq\x80*B\xed1\xa2\x82j\x90a1\x0fX1O\x87d\xa4\x1e-\x8eX\xd6\xecG\x96\x9e\re\xe6\x9fBZ\x18Qk\xdb%X\xb1`\xf561GDsMG\'O\xca\x19\xde\xdd\tZx\xdb\x07e\xab\xd7\xb1\xde\xe8\xcd\x08\xcd!\x0bS\x1f\xa0\xe2?yg\xdcT\xda\x012\xa2f\xd1\xd8\xe0\xdc\x0f\x9d\xce\xe7\xb4\xe0\x08\xe0\xd8\xe8\xc6\xe7E|\x93k\xbaO\x82X\xa8\x80\xf3\x86\x0e\xfd)\xab\xc5\x10\xa3\xe1\x0b\xe7\xd4\r\xb94\xb7\x01\x16\x95dcS\x81z`e\x89y\xeek\x8e>\xf2I\x9b9\x80\xbf>\xa1\x93\xf1\xdb\x91V-@]\xf8]\x0ck\xe4[\xc1-\x1b\n\xd2|k\x9b\x0e`^\xa5\xa7(\x13*C\xfdq\x89ma/\rB\xb5QA\xfd1\x08\xd7\xfc\x1e\xed\x19\xf4~C\xb1W\x1c\xd1\x16^\xf1\xcc"\xb9\xf3\xf4\xcb\x9b\x93v\xb0\x94\xbd\x7f\x9e\x86\x00\x85\xa1p\xaa\x04f\x06o\xf9\xfa\x1a+J\xaetEp(pY\x17\xd5\x11\xe8\xb1\xae\xccC\x12\xaf\xc3P\xd6\x87\xb4!@/\xc0\x8a\x8aM\xef\x10$\x8d4%\xea&\x80c\xaa\xeb\xc38\xee\xb3\xeaI\x90_\x88t\x0bQ=\x8c[\x94\xae\xd7Q`W\xc6$\xf5@\x9d\xb2K\xfe%Q\x91k!W\xe8\xdd\xc0\xd9f\xef_\x14\xb2\xdd\xc1((]\x87m\xbf\xb7\xd3\xb5y\xf7\xc4gm\xf8\x10.\xe1\t&[\xda\xb07\xbb\xf5p\'\x19^\xb4\x99\xe7T\xb9\xf0>j\x05f\xf2\xa1vg\xfc\x8c\xfaD\xca\xf4\x7fq\'\x86\x8d\x13\xa6\xfa\x13-\x0c@\xc1_\x07\x8c\xb3l\xb6Td\xffU\x80\x9dZd\xc9[\xfa\rl\xd1F\xdb\x87k\xb4\xb1@n\xbe\xe32R\xf8#\x1e\x98\x9d\xa3\x03\xa6\xf2\xa6W\x08aH \'\xb1\xc0\x88\xae-l\xf0\xd5\xf2\xea\xb6\xb7n\xa1*\xb4\xd4\xc5\x8cqS\xbeju[p\x19\xd9q\x9f\xf8|\xbd\xe4\x13f\x8f\xee\xe0^\xaf\x9diR\xf7\t\x1c\xdd\x93\\6\t\x0e\xd5\x90YK\'\xfaY\x1aV\x9d\x14\xaeTJ\x16\xa1]\xba\x89Y\xfd\x8d|tFf\xac\xbf\x9fj\xf6\xc7\xed\x86\xadHGn-1\xe3u\xfc!b\t<8\x9f4\xc4\xc7\xed\xe7\xb1\x81\xee\x11\xb1]\xc0`\xf2\x11\xc3@\xd7%d\xd8=)B\xb6\xedS\x1a!\x05\x97\xefs\xcd`\xddF\xc5\xa9\x8f#\xe8J!A\xb8\xb4,"<(\xec\xd3\x1e<\xbc\xfaQ\x1d\x9a+\x81[R\xee\xd2G\xd5\xcf8\xb2\x8c\x92\x83\x92\x002\xb6\xedh\xfb?\xe9\xace\xd0uA%.\xab+G\xedB\xe3~\xcd\xd4\x8c3\x96\x8f:py\x15\xe3\xe2[\xea\xc9\x18\xbf\x88^\x19]`4\x91o\x02\xef\xda\xa0x\nVy\x95\x0f(\x9d\x1aA\x9eo\x87^3\xb5L\x91b\xc7\xe8K\x95D\x19\xc1\xda\xd8X\x07\x02\x93\'\xf5.\x01?&\xd7R`\xb8\x10\xa2\xbb\x12)\xb3\x18\xa4FI\xf2\xf0\x8b\x07pj\xf4\\<G\x02\xb4\x1c  6l z:\x9a\x94@B\xb5\x06\xa3qY\x96S+S\x93\x92\xfb\xddC[o\x15\x8e\xc2\xbd\xf57ie=L\xf5\x83\xea0\xd1\x9d?\x9e\xcb\xc7\xce3\xe4^\xdd\xe5SlS\xdd]F\x1a4\x83C?\x1b\x1b\xb8\x0eD\xbb \xdess\xb5S\xfeu\xbe\x15\x8dom\x02e\xe6H\xaaG\xe9S\xd9\x9a_\xdd\x97\xca\xa3\xdff\xb8\xb6\xf5\xf4\xc0\xc9d\x9d\xbc\xb2\xdbH\xfb\x81\x82\xa2\x10\xff%\xcb\xffFz\xcf\xdc_\xf0\x89\xcb2K\xa768;\xf7\r\xa8\xfba\xb0#}\xb6r\xe5\x18p\xe5\xae\xd6J\xee\xcd\xf9\x98\x8e\x9a\x07?r\xab\xac\x1cm\x1b\xea\xb7s\xe3v\x8e\x04\x83\xcca8v\xda\xce\xa4\x94t0\\c\x9eo\xb4\xfc \xadr\xfe\x94F\x92[\x8d\xa4\x06\xd5\x9b>\x85Hp\x89U\xec\x80\x1dcsO\x1dX\x81\xf5\xcdQ\xdc\xfd$\xe2\x9b\xdb\xbc\xad\xcb\xac?\x18D\xfaDDf5h\x99\x90.d\x08\x91\x14P7\x00\xbe\xc4\xacu\xde\x06%\x9e9eD\xaf\x1c\x03\xcc\x0c\x8c8Q\xe2^\x9f\xb6\x9c\xed\x90\x8d\xc9v\xa7\x97qz\xdcriw\x01\x01\x83\x14\xe9\x81Z\xa8\xa3\xb1\x02\xabyx\x1d\xf6\x8e\x10\x94\xf4\xe6\x8fbq\xf4v\xbc\xf90n5\xff@\x0f\xa7L&\xbc\x1f\\\xbbC\xd5\x04NY\xbfz\xc6\x12\x08\x04D\xbd\xa9\x83\x8f\xdf\x1d\xae\xa2\xcf{\xfd5.\t1\x94)t\xc5\xad\x9e3+ \x056(\xcd\x81|\xe7i$V\xba7\xfa\x9f\x1f\xf5\x08\xbf\xcb\xf7g \xa1\xf5_S\x1e\x96H\xab\xe8\x9a;C\x8e\x1dy\xe8o\xf1 \xc9\xbd&q\xb82\x1eJ\xef\xf2j Z=\xe0\xdc\xa1*T\xb3\xa2z\xaf\\Y6l\xc2b\x19\xa4\x90^=\x1cO<|-\x08\xc8\x8f\x91%\xedd \xfa\xf9\x95\xe9q\x05\xfd\xd9\x04;\xdd\xf2x\x00\xa0\x99\xd2r\x1f"\xb9\x8e\xe5\x84\xf3\x0f\xe6L!Jl\x1ax\xb3t\r\xba^U\xa5H\xd1\x80:c\xd8{\xb6\xbbN\xd3z\x85\x06\x07\tC\x16cI\xb0\xfbPK\r\xc0F\t:\x89\\\xb6\x07dg\xe9\xff\xb08\xc2A\x86C\xb8B\\\xad$\x13\xd3\x9dM\x95w8\x10i\xb6\x93\x8am\xe2A\xcb\x9a\xa2\x13\x1d\x96\xfa\x86\x9ac\xe1\\\xda-\xdeG\xdf\x80\x0b\x9aP\xc9V\x11K\x97\x06X\xf1\x04\xdb\x98<\xc4X;a\x82\x0e\xa9\x84K\xe2LqP\x86\xf8W\x8d\xa8:\xe8lq\xd9.\xa0\x0e\x9f\xd7\xe0X\x9a\x91+\x97\x9c\x0f\x95g\xcc\x00e[\xbb\xa1\xd2\xae:\xc2\xf6\x846A\xa5G\x99\xd8:\xaet\xf0\xf6\x0168\xe1\rqQ{\x1e\x81J\xda\x1d\x8b\xbb\xe4c_\x00\xf5\xcc\x1c\xad^\x18\xc4\x14E;\x8e\xf4J\x9e\xed\xe6\x8a\x97\x02\xa6\x05mu\xb5\x8d\x08\x10h\x81\xb1\x07\x04\x02>\x14BD\x1c\xe4x\xdb{\x00)\x1b\xee\x0b\xfa\xd5\xe0_\x9e\xb7\xd9\xb3\xfa\xc3\xe0\xc7o6\x11\xac\x89\x95_d\xcb9\xeb%\x87hU\xfb\xa2^U\xa0\x9cv\xd6$\xad\xc20Y0\x18\x1b\xb9\xd6*\x18hh\xe4\x08\xfd\x8c\xf6\x99N\xbb\xe7S\x82\x9c\xf1\xec\xf9\xfb\xec\xe0\x01\xa3\x12f\xd9x\xcci7)\xe3\xe1\xb4\x82:\x08\xb4G\x17\x14\x919\xe3\xf4:i\xff\'RbO\xbf\xa2\x9c\xbb\x98\xdc\x89\x8dh\x0c\x06sC>\xa0\x01\x93\xa0\xcfQ\x0c\xa8m\xc9\xcc\xc0\x03\x0e\xa9\xf5\x02#;\x95\xb1\xf7\xa7Mv\xbe;\xda@\xe6*UhJ\x0cM\x05$\xa6z\xe5\x06\x0f\xfe[\r\xce\x8e\xec\xe3\xb3\xd1\xb1\xf5\x19A\xb4\xda\x9c\x12\xcaWBS\xf7\xddlY\xf9\x16\xeaZj\xe3F\x8e\xf3\x83\x9d9\xd6\xaa\xf7\x88HIn\x88\x0c\xacX\xa7+9q\xc1q!\xa8\xa8\xabj\xa2"\xf2nC\x94\xe4Z2\xd2\xb9t@IBY0\xfa"\x04\xf3\xd4\xe3R\x12:\x9c\xc7\xd1\x14\xb0\xb0\xafK\xd3\x16\xbc<p\xd9\xe4\x9c\xb4tlZ\xda\xed\xaf\xd5\xa2]\x8b\xaf7\x8f\xec\xa7\xb9(\xa2\xda$\tJ\xd9\xb5\x9f\xf8\xd6+\xc6\x8a5\xdf_\x9eYj\x1f\xfard^u\x95\x0b\t2\x88]TQI<x\xb8_<#\nX\n\x02\xe0w\x03PC\x1e\x90_Uy\x81\xb1\xe7+\xec(\xfde\xdc8\xb5\xbe\xf89\xbbU\xe7\xe1\x1e\x93\xef\x9a\xe8\x94\x05\x8a\xd8y\x90\x99\xe8\xa5\xf87\x06\xc0\xd3\\d<_\x7f\xcb\x0b\x14\x17\xa0z\xc6kq\n\xba\x92^\xa9\xba8\x9a\xfdi\x0e\x10\xef\x92\xa6\x18tj\x06K\x1c\xa6f\xea Q\xe7qx\x0cp\xd19:.O\xe0\xcb\xea\xe0\xe0\xce1\xab\xac\x9b\xf5\x9e{m\xcaq\x7f\x06m6\xf1\xf27\x86\x1f=\x15\x83\xba*\xbf\x18)\xf9\xe7\x1e\x1ce\x89M}\x87"\xf0R((\xc0\x12\xb8\x8fO\xa8X\xab\x82\x82\xe1(\xfb\xd2\xe1_\xd98\xb5\x91\xf7\x96\xa6\x10\x87\xb8\x9d\xa6\xadP\x1c\xd9f\n\xc1i\x16\xb1\xd1%\x00T\x92t\xc31\xf3\xednz{\xb9=\x1aR\xde\xe8\xba\x82\xbd\x89\xba\xbd\x98akGPNZs\xa4\x03<\xc2Z>\x8b`\x9f\xd0=\xbe=/=\xc1\xe17\xa6<]\xd4P\xd4I\xda\xe0\x9e`\x1ey\x88\xa3\xb6{\x08\xfb$\x93\x8b\xfe\xbc"\xdbV\xfa\xeds\xe7cc\x93R\xe1p\x0b\x8b\xc4\x8d\x18\xd2e7\xcf\xc0W\x86\xe1\x88\xc0(\xf0\x14\x98?\\\xb31\x86\xebD?\x86!\x06\xaa:\x9b\xe8OI\xa2\xa9\x96\x83V\xaebFw.[\x7f\x82\xb1\xedc5\xbd\xc5\xc9\xb6Ww+\xc0\x94&\x92\xdeA\x03_3>X\x03\xa9D!\x06\xc0\xdct*\x97\x9a\x92C\xa5\xef\x9e\x9b\x97\x0e\x1b!\x84u\xf9]\x1b\xcbG\xa0d\x07\xdfBQ\xb5\xf3\x14Y\x04\x96\xa8\x11\xea\xfe\xd6\x13P8u!6\xe5z\x90\xb9\x90\xb0\xd7;\xc9\xce\x9f\xa1k\xb9\x153\x14)\xe5)\x14}\xb7\x11\xfc \xb0:7\xb8e\xd5\xed\xc7RG\x15*Q\x83\x8e\xa1\x00\xb8\x1e-\x0c\x92"\xa4\xc5\x8d\x12\xed\x13.|\xac\x15\x1e\x1eO\xa4n\xef"BA\x07\xea\x9f\x83\xdc\x03\xc7#\xb1\xaaV\xc3\xae\xe2yj\x91N\xf3\xd4{y:\xc3\x91Z\xb2\x91~\xdf\x01\x0e31\xc3\xb8_\xc9\xe9\x9d\x0f\xb6y\x81\xa2S\xd3|o\x8f\x8ew\x91~$\xe4\xead\x92\x03\x18\xd4?,\xac\x97\xf1 \xc7\x93j\xff\x1b\xea0 i\xdf\xb0\x9b]\x83\xb7\xc2\xd3\x12:\xce\n\xc0\xa1j\xeb:\t\xfd\x19\xed,\xfd:\xa9\x1b\x85$U\xfe+\xb9\xc1\x1d1\xfb\xf9\xa9\x1c\x9a\xf6r\xb8\xadz\xa1\x87\x02E\x90N\x85\xb5\x98jt\xa8\xf1\xb8rh\xf179%\xa2\xa3U\xa7_R\x85\xd9.?\xdb[l\x14^mO\xbc=\x81T\x05-\x16\x08\x8a[\xc2\xa1+_!.\xa5\x02\x02\xeb)bgn\xe4\xb1\x06d\x10P\x9b7\x05+\x0c\x9a\xd1q\x0el-\xbc\xfb_\xe1\x88S\x0fXB?\x16\xc1\xdd\xe9\xe6\xd0\x1b\x9b:\xe9\x19\x07\xfb\x1f\xc2TWs\xe6\xf1SU&\x86\xf0\xb0Q]A\x01T\x08\xae\xb1<\'\xe6\x95\x1c\xae\xb9\xc5\x14\x86f\x82\xd9\xba\xf1b\xb3\x1aj:]dV\xe8\x00\x050\xf9R\x14\x9aO\xa6\xdc\x84\xef\\\xff\xd6i\x91\xf7n\nz,H\x8a;p\x15\xf1\x02VZ\xf9\xa3f\xc4\xa2\x87\x80<wv\xfb\xf5\x98X\x183\xd8Q=\xb7\xdcGe\xe8\xcf\xc7\x03\xb8\x8c*ycnf\x94]\x10\x93.\xd6\x8d\xb1q~\xaf\x8bE<\xc3;\x07\xb7l\xb9)\xab\x99|\xa4\xb0\xf2B\xfb\x8e\x13\x9d\x8b\xcfp\xb2;[[\x0f\xe0\x8d\x99\xdb\x9e\thu=\xa3~\xe6!\xd9"\xc5\xcaW\xb9\xb4@d\xe4q\xea\x96\x07\x13\n\xb3\xb2\xe1\x1b\xb1hw\xdc\x13H\xa5\xf1\xfc}D\x86\xb0\xb8\\gQ\xbf\x85\xb8\xed\x00\x8d\x7f&N\xed\x8d\x9b\x8e\xa4\x80"B\xf1G\xdd\x9a/\xd5;\xd3\x7f\x15\xf9\xb1\x16\xb7\xe6U\x10\xb6\x89\x07\xb2\xd0\x16\xa3\xf3\x9c5tq\xd3\x94\xc5\xac\xb8[D\xea\xe3Ugw\x94\xe5\xddX;B\x1c\x85\xd5\xaf\x03\x1e\xe4\x86\x18\x8e!\xec\xcc\x95\x00aXD\xa2\x0f~\xd7\x9c\x9b\x13+\x90\x83<\x9c\x88\x1dE&\xf4\xe4W\xda8.\x91DB\xb4\xf0e\xe3c\x90{2\xd3\xaf\x9dD\xef\x9bhv4\xa3\x01\xe3\xfa\x9cu\x0eF\x9de\x1e\xaa8\xa6\xbb\xd5\xe7\xb5&\xee\xde6\x1dV]\xe9\x02\x87k\x1a\xdf\xcb"\x02\xb4#4\x96=i\xa7\x17\x05\xf6\x19l\xcbn\xb5\xa8\x7f\xb5e\x96\xda\x08\xfb\xeb\x8f\xe6F,W&V\xaa\xec\xcd\xc1\xe3\xf0\xe4\x90\xff\xc1\\\x9f\xecO\x0bCh\x07\xa1Kpo/J\xbecW\xd9u\x90\x84\xa2)\x82\x8f5_(-\xd6\x92r\x06\xf9?*r\xcdO\xe5\x10*;u\x10\xecv\xe9\x1f_\xb4%\xf9;\xd6a\x80\xe6:\x98\xfc\xad\x0f\x07\xbb=\x81\xd8\xa8\xc7\x84\xb6W!"\xd6\xb6\x9f@"\xcb\x93\xcb\xdc\xc3\x0b\xb3\xbdY\x1eye\xa3\x9e\xdb\x91K_\xcb\xcb\xe1\xd9}\xc26\x93\xe4 \x84\x12\xf6\x1f{W\x83\x19]\x95U\xc9\xdf\xbbk\xdb\x92\xec\xc3\x07\xe7\xc6/\xa7\x11\x8d~\xe3@\xdf\x903|\xe0k:\x06\xf9\xb4\x88\x80!1D\x0c\xc4\xe4~6I\xc5Rj1\xb3\xf3\x98U7\xbe\xd6Qeg\t\xa4K\xc0\x93\xccP\x19E\x93\xea\xa7\xf5\xbe\'\x97\x0b, \xf0,\xc4\\\xf1\x1a\xe6\x8b\xbe\xc3\'hO\xa7qj\xbf\xbb`\x12Ac\xc6^\xf4\xb3\xdf\xca\x96kR\xc7"\xc2\xe3\xe9o\xbe\xd2$\xc2\x1d\x14\x17 \xad#$\xb2\x16_\x1f\xb8\x97\x9a\x8f\x93x\xdb\xf8\xf61Hxq\x10B\xe3\x0b\xc7Wm~\xf7\xf3:\x00&\xc0\x0e\x8c\x98J\x1fbPA\xads\x81VY\rx,\x0b(\xf9\xf1.\xb4b\xdc\x98\x8dr:W\x82\xf5\xf8\xea@\t\x85\x14\x8epN\xe1\xde\x9d\'G\x0c2]\xf0\x9dZ)h\x1d{T\xddR\xf9&\x05d\xc6\xac\xcc\xf0C0\xd0\x90\x14\x97\xa1\x15oF}\xdb\xaa\x9c\xb2\xb9\xe40\xfd}\x94\xb3P\x9f\xd8\xc0\x9cz\x1f\x0b\x9f\xbab`\xe3\x1a\x0b7r\'\xbc\x1f\x1c\x83\xef\xa5\x81\x9c\xe4\x95\xffh#\x13L\xac\x1d\x92\xe3\x17Ml7\xea\x7f\xfb\xaf\x00p\x0c\xa2>\xab1\xa0\x98\x90;\xc8[K\xd5i00P\xb6S\xf8Y\x9c\x01G\xe1\xb5\xdd\xe8b\r\x9d\xf0\x81Co\x8bZ\xdb\x7f\x0c\xa1\xf0\xc0?d\xe9\xef\xb3N\x97\x07\xc2F\'\x85Y\xff\xbd>+>\xc8\xfbr\xc2"\xde\x8d:\x9b\x1e\x99"\x11+\x99\xfc\x92\x97\x05s\x03\x89&\xfd,X\xf4\xba\x8a\xc7B\x10\xbd\xbf{\x96\xb1\xf5\x03\xc7z\x07\xe1>\xc7\xf1V\xcd\xab\xf3\xf9\xfe;1\xec<ux\xda\x97\xd4\x89Q\x12\x12\xf1\xc7$l\xb7c<1\xfe\xcf\x8d\xb8\xb7\x8f\t\xa9!\xd2\xb8_R\t\x17=\x87\x86\x86\xf7\xf4\x05\xe4\xa6V\xbbS\xa5a;\x8b\xee \xb5\xaeq\xff\xf1\xbf\xb7\xd4\xbc\xcf\x10\xc9M<\x8c\xbcw\x96O\x14\xe8!\x0f\xa0\xb9C\x9d\x06;p\xb6\xf9\x13U_\x1flfri;\xc3\xc1\x1e\xda\xafa\x0b\x1ewaDX\xba\xdd\xedHp\n\xd3\xfbv\xd4r\xbc\xca"`q\xca\xe3\xb3\xe3U\x86\x94u\xf8\xfbj\xb2n\x14c\x02d\xa9J&W\xe6z\xf3\xefE^P\xdd\x96\x91\xd0\x8c\xce\xb3@\x81\x17\xe6R\xfd\xbb\xa1\x9a3kA\x1a\xf2\x13\xfb\xc4\xd1\x7f>l\xdd@\xcf\xf7\x17\x1a-\xecH\xd2\x8f\x88\xb0\xcc\x8a7\x90(i\x96B\xed\x14F\xb8\x00\xbf4K\x9b\xd1,W\x83\xfd\xef\xd0\xa2\x92\x06\x8b\xbeq\x7f5\xd0\xfa\xdf,dDOQo\x11\xe7\x7f\x07S\xabr\x92,crOn\x98U\xc4\xa7\xf2FQLp9X\x96\xe8Z\xd7&\x84\x04\x14\xec\x8d\xb6\xc9x\xf4?\x7f*\x8d#\\G\t\xdbS\xf3&\xb2D\t4\xd0\xc4\x96.a\x14\xd8\x99ra\x13"\x021#\xf1\x8e\xaf\xbdF\x83\xed\xe8?w\xc1D\xa9\x9d\x8a\x9d\xd2\x00b\xda\x9e\xf7\xe5\xc0\x89\x1aqb_:\xc8v\xc5r1E\x0b\xb5\x7fI\xcf\x10\xc1>\x93\xb0e\x891WR\xbbJK\xeaI\xb6\xcd)+.\x94\xf5\x18\xa5\xfe\x84#\xd6_:\n\x03\x15\x8a\xc0\x85\t\xa2N\x06bJn\x0c2\xed\x9d\xa3\xca\xa4\x90\xbd\xe0\x14\xf7\xb7\xc4\xeat\xb9\x11z\xa6v5\xaarSS\xfbh\x1e\x11S\xc3\x11\xda&\x0f\xd4\xa4*\x14\xcc\x1f\x01\xe4\xde\xf1K\xc1T\x98l_\x05\xf5\x9c\xd1\xeb\xf1\x9f)\xe8\x88\xbc\xa7\x84Z\xcd\xe2\xf6*\xfa\xa2I\xdd\xac\xa6y\xef\xdb\xbac\xb2\x0c\'[\x0fF\xacd\x9a^\x00\x9b\xfb\x99\xf0\xdc\xea\xe8d13^M\xf28S\x10-np\xad4\xe3\x92&\xee\xba\xad\xe4h\xb4\x12\xa6\xd2\xac~F\xee\xd3YV\x16\x1b\x1a\x9e\x0f\x7f\xde\x87$"\xf1\x8b\xd7\xe5\x12\xaf.t^\xe90\xbe\xb3A\x10\xcd\xf9\x06\x7f\x04\xbf3\x8e\xe4\x12\xad\xcc\xe0\xdf\xc6\xd4"\xf7\x80\xb4\xb0!]i&i\xa7(w\xf1\x97 \xc7\xb9W\xac\xf5fI\'\xb3\x8a\x83lx+\xf1\\\x06\x83\xe7)\xf2\x8d\x1dI"\xd1\x01\xce/\xcc\xf9\x8a|\xbf\x90q\xfa\x1c\xfb\x95\x81_\\\x9f\xb9\xcf=\x18~\xc9;\x15\xfd`\x04BL\x14\x0e\xf1\x8cQ\xf9\x87\xbar\xb1T\xf7\xd0\x1a\x9a\x997\xbf08\xd4\x02\x13\xe1\x03\xfd\x0fe\xbcR\xdc6v\xd4\x08:m\xd5\xa3J}j\xee\xf75\x91\xdf\x15\x02\xfc\xd3\n\xcf\xee\x98\xa9&\t:\x11c\xe8\xad\xfd|\x7fzI\x03\xdc\xfaxH\xe4+N\x12e@:#\x14\xa0\x9d\x0b\x00\xdfq\xfb\xdf\xf9\xcb\xa8\xe7\xb9\xf7w\xd4\x1a~P\x7f;\x05%\xa43\x08\x13(HJ4\xd6U\x13\xb1\x89\xd9\xe7\xde\x07\x02\x9b\xffk\xd1\xc5+\xb2\xd1N/\xe4\xcc6y\x7f\xf2\x1aW\xaf\xdbe\xdek\xf1fybr\x8es\xff\x02\xb0\xe91^\xc7\x08C\x89\\\xe8\x15\x89Us\x89\xbf\r\xee\x9a\xa3S\xc1\x02A\xe8sf\xe5\x01=\xc2\xed\x84b\x90\xbd\x93c\xddm\xec8\xeeZ\xaak\xee\x97r\xddfpds\xbcR\xbbv0\t\xd4\xc2n\x10\x87\xaa\xad\xbf\x1a\x01\x1bc\xa1l\xff-\xa5m\x87P\xad\x84;_5\x84\x14-@\xb0T\xf3\xc9\x1a\xa2\xd6\x19i\xcaC\x03O\xda\x0e\x95<\xe5T\xa7\xd6\x1a-\x80\xb0*\x0f-u\xc00\x8aBt,\xf0\xcb"\x03\xf0Z\xe8_q{\x17\xba\xbe\xa5\xc8\'\\)L|\x8b\x07\xe9|\x8b1\xd7\xd0\xe0\xdaLr\xb9(\x0f\xaa(\x8f\xf6\x8d\xd6O\xf0\x88\x8a\xffyq\nJ\xb7\xb8\x12H#\xfe\x11T\rz\xe9Z43\x8c\n\xcc\x96\x06\xacx\x9d3\xea\xc1U\xf9z7\x06\xd4\x94A\x8f#\x8a\xa2\xbf\x0cl3_\xf5\x07\xf6f#\xcai(a\xf6\x9f\xe3:\xac\xe2\xa7Q\xdf\xb2\x84x=\xb8\xc8\xd3\x99@=9\xa6\x1c\xb1W\x8f\xef\x9c]\xe4\xef\xbb\xfd\xb0\xe6\xcc\x1c\xfb\xa2\x9e\x07\x19\xa0=\xf1.\x9b\x1d\x02\xdb\xb2=\x83\xdb}\xbb\xcc\xdfu^\xe0\xb1R\xf8P\x152\xb6sJ\xfc<F\xe2\xfcL\xb0\xe9\x0b\x8dr\x82#<wn\xee0\xe1\xd5\x7f\xa5\x8c"\x9c\x7f\x9e\x10\xe2\xb1_\xf1\xf4$_\x8b\xba(5 ?\xd69R\x01@Q\xee"\xa7\x1ej\xa3\xdbIc"\xedt\xf0\x88@\x1cHA\x16\xe3j\x81\x91\x1bVU~\xa0\xa8\x81N\xb2\x11\x014^-\xf8\xb9\x81\xa8u!\xe2p-\x15\x97\xb3\xb8\xf0(\x8a\xbc{\x9c!\x07\xb9[Lw\x87H.k\xdb\xad\xa8\xfb=\xd4\x08\xec\xa3S\xfe\xef*\xc1\xb2Q\n\xe8\x1e\x8b\xad\x12\x911\x13U\xf1\x95p\xbbF\x1f\x08\xfa\xc2\x8d\xf9{\x9f\xfe\x1f^\x19]\x89\x0e\x12\xf1\xa5\xef\x88\xdb\xab\x90\xa5\xc4g\x83\xf2\xc0u\xa4\x9aSsZ\x97\x15\x17\x86c\xc5\xd9\x88\x00[\xcb[\xf6\x04\xca\x87C\x93\xaehgP\xfa\x9a\xe2\xcbg\x7f\xdf\x1a\x88\x1a\xc9\x1e\xca\x92\x1e\xf5\xf5\xb6\rv \xc6,/\xb4x\xe6`\r\xce\xbbM\x99\xb0\xe9K6w\xdc\xbc\x12\xf8\xd3\xc5\xc2\xa2P\xe3h{G\x817\x01\x0b\xea0\x14\xca\xd12%\xdb\xd3\xdf\xde\x8b\x128\x1e.\n(cn\xcd\x11\x84a~5\x8f2Z\xfb\x957\x1aM\x06\xfd\xcc\xcc\xcd\xce~\xf1\xfa.Z\x16EH;sLo\xa0\xd4f\xb3\xa9n\xa6U\x1b\'\x04\x8c\xf5\xa1\xea\xceX\xb3d\xba;FaR\xfb$\xaf\xeb\xbd\xdb\x95\x96\xd8V\xd5f;\x9d\x8b\x17\xea\xf9^W\x92\x88\x05\x02\\\xdfkP\x13.\xbb\x02.\xee\xf6p4s4\xb7\xf1\xc9\xcb\x04\xbb\xdd\x12\xfe\x9ai\xab\xa2~\xa7\x9f=\xea\x91\xb1\xf16\x87\x05\x84\x87.5J\x92\xa8\xfa\xbb\xd7\x95`\x0f\x15\xcaOys\xc5\x83\xa9\xca\xa2\xf0G3`\x85z\xc62\xb8.\xe3QLM\xeaL\xb5\xf1\xf0q\x9c\xa1\xed\xbd\xe1\xa3 \xf3\x10\xb1\xae\x96\xb1o\xe2\xef \xf4(\xfb\x01\xfc\x02\xdd\xd6\x1e\x91m\xb1\xba\xe1\x89\xd0\x12\x03\xc7\x0e\x02\xab\xb8^\xf6\tI\xf1\xac\xa5\x1c+\x1d\x80\x90hQ\x92]\xc3\xb55t\xfe\xf2\xbd\xc3y\xe0\xee\xabF[\x82\xa9Lh%\xaer\xc5{\xb5BO\xcc\x12\x19\xaf+\'G\xcam\x1e\xe3\xaa\xc9\xe4^\x87}\x93\xc7\xef\xca\xd1\x9d4\xaecXc 8\xbe\xf8\x9bM\x14\xa1\xb4\n\x90\xc4\xd2\xf4\xe1\x1d<\xd9\xa4\xd3\x97\x97\xaa\x91/:\xc7\x99\x82_\x94\x0c\xea\xa3=\'4"#\n\x03\x9fa\xe3\x9eW\xdb\x01Q\x80G\xa9\xf7`\xf0o\xc4"q\x800Y\xb30v\x9c\x0c\xfd\xe2\x1a\rSz\xa1\x90u\x98\x13\xa2\xe2H\x05\x86q\x93\xd6B\xa3\xaf-\xa7\x94\xdf.I\xa6S\xc2ik\xb8N\xae\x18\x81\xe3\xf2i1\x0bz\xeb\xf0\xcb\x84Wd\xb5Z\xc6\x0c\xf0\x03\xceO\xb4\x85\xd4\xdc\xd9\xebA\x00\x84_6$c}qg\xe2_6x\xb0a\x144V\x16\xc1\x01\xadm/\xec\xdfSho\x82\xa3\xf0\x9b:w3\xebQ\xa4Du?\x07q49\xbf\xbdg;\xcdF\xddo\xff\x8f\xfc\xb3\xd7\xae\x0c\x91T\x12\x97 c\xe7\xef"\xff\x1e\x99\xe8~\x1d\xac@%-_r\xba\xc0\xf0\xd2\xda\xfd\x0e6%`U\x81\xfe\xa2\xbf\xaf\xea\x817Tc\xb8\x1f\x8a\x14d9\xa6\xde0\x7f?p\x95NHQ\xb0\xc0\xc3-\xfb\xc8\x0b\x87+\xd9<\xda-(\x18z\xd8\x1d\xa9wi6\x10^\xad\x15P\x8b\xfb\x93\x13\x87[ P\xbe\x0e\x0b\xca\xd6j\xd7n\xfb\xd0\xcdfe&c\xd8\xbf\xac24@\x9b\xebn+\x83\xaf\xc0\xc8\xbf\xa8\xe3s\x1ck\xf2\xf1\xc9\x068X\xf0\xde\xe5\t\xf9\xdc\xcb\x90u]`\x15\x16\xcd\x99\xf5\xfb\x925\xc6\xfb\xe8\xcfX\xaepB\xb2\xe4l\x94y\x15r)\xda?`\xa3c\x9f\x1b\xfb\xab\xf4a\t\n\xe9x<\xe84p\t\xf4m\x18\xd5\x95\xac\xed\x135\xfc\x8d\x94\xbc\xe4v\xe3$\x9b1\xb1\xf9\x8fo\x14\xfdO\xdb\xca\xa9\x90;\x80F\xde\x1e\x88 \xd0K\xcc\x88(\xce\xa8\x91\x04\xfb\x8a\x07\xc1\x13:`\x1c\x13B\'\xaf6D\x80\xf9\x02\xe0\xc7\x1a\xbf7\xab\xac\x8a/\xa6\xe8\x8a\x896\xa3TIh\xad\xea\x0c0\xf3\xd5-\x1dx\xd8\xa5/7Y\tp\x90\x85\r\x0cl\xf4\xf6\xc5>\x9e\x08\x95\x83P\x8c\x89\x84\x82\xc1&\xde\xe3SaG\xb2\xa4W\xb7{\x85\xfcB\x9e26\xef\xa8\x83\xb3\xe2""Y(\xde\xd2W\x9d^D\xa4d\xa3,\xa4\xd3\t\x8a\xe7,\rQd\x9d\x9f\xf7\x13h\xb0o\xcf+\x08\x02\xc2\xb7\x80\xf2\x7f\n\xd1\x00yKcrEMec\xc4\xe7\x13\xf7\xabC\xf2\xeb\x94\xad6&Sk\x82#\xa0\xda\xa8\xd9\xa0\xc1\xf1\x15}(\xd0\xe7\xe9ZQ\x89$9\xf2$\x19\xe8\xdfYC\x96\xf7(\xa5Sq\xd4v{\x05\xbf!\x9c\xf3v\x17*c\xc9\x99\x0e\xdfp\'\x15\x9af\x83\\\xdc\x0b\xc0\xc5*z=\xb9\xb1IPp\xbc\x0b,dGQ:\xd1\xd1\nK\xaa\x86\xfe\xf4\xdc\xf3\x88~\x82U\xd9Y"\xe6^\xcb\xaa\xdbby\xd4\xcfpy\x13\xd9\xe3\xa3|\xaa\x8a{\x85\xb3,zJl\xcf\xe4\x01\xf7j\x8b)\xb7\x8e$\x93\xf6\x99y\x8eL\xbfc\x86\xd3\x19\x08@\xc4\xfdG\t\x1a\x88\'\x89\x9d\xc5\x10\xd4Y\xad\xdd\xaeRo\xe3\nR^\x9f\n<v*\xca\xca,f\xd8\xe3R9F\xfa\x08p\x8d\x12WJl\x8cf\x1dWe\xaav\x1aj\xdc\xcbi"o\x8bh\xb6\x00\x05G\xcf\x05\xda\x127\xa2R\xadA}\x88\x89\x1e\x8b\xafR\xb7\xd0_\xfdE7\x04\xca\xfcPW\xc6&\xd9!\x8c\x80\x93-k\xd0\xa2@z?\xef0\x81\xc7\xec\xb5y(*N\x97i*\x86I*I\xd3\xe7\x00\x8a\xa2pT\xffD7\x91u{k\x9b\xd4\xf6\x8c\xee\xcbf\xee\xcf\x04|\xbe\xb7gm\xd9\xe9\xa0\xed\xf2\x04\xc8H\x00#\x97\x0b\xaf\x06\x05\x96\xf1\xab\xc2\x00*T\xb4t\x00\xfd\xber|\x185#aG\xef\x9f\xffl\x95Ds\x917m\x8bj<o\x16\n\xc8\x10\x94\xe6\'\x8cL=7\x9e\x980 \x91)\x93\xb0|\xee\x82\x81\x00\xc4\x99-:\x06ly\x12\xa23\x90^x~Pi\xd6\n\xd1\xc7\x95\x17\xc4\x9c\xdd\xb8^\x8f\x14\x01@\x17\xc6\xa7\xdd\xb6\xf1`\x84\xb9\x87+\xbd\xc4\xd9m\xd0\r\xbe\xa5\x91*7\xf3\x85`\xc7\xc1\xaa\xa1 \xa3"
  },
  "encoding": "UTF-8",
  "creation date": 1529449174,
  "announce-list": [
    [
      "wss://video.blender.org:443/tracker/socket"
    ],
    [
      "https://video.blender.org/tracker/announce"
    ]
  ],
  "created by": "PeerTube",
  "announce": "wss://video.blender.org:443/tracker/socket",
  "url-list": [
    "https://video.blender.org/static/webseed/b37a5b9f-e6b5-415c-b700-04a5cd6ec205-720.mp4"
  ]
}

After some research I got a solution: Pieces are simply bytes of mutiple sha1 hashes. sha1 output = 160bits = 20 bytes

So after some changing of the code you can understand now how the torrent file works.

import re
import json
import binascii

def tokenize(text, match=re.compile("([idel])|(\d+):|(-?\d+)").match):
    i = 0
    while i < len(text):
        m = match(text, i)
        s = m.group(m.lastindex)
        i = m.end()
        if m.lastindex == 2:
            yield "s"
            yield text[i:i+int(s)]
            i = i + int(s)
        else:
            yield s

def decode_item(next, token):
    if token == "i":
        # integer: "i" value "e"
        data = int(next())
        if next() != "e":
            raise ValueError
    elif token == "s":
        # string: "s" value (virtual tokens)
        data = next()
    elif token == "l" or token == "d":
        # container: "l" (or "d") values "e"
        data = []
        tok = next()
        while tok != "e":
            data.append(decode_item(next, tok))
            tok = next()
        if token == "d":
            data = dict(zip(data[0::2], data[1::2]))
    else:
        raise ValueError
    return data

def decode(text):
    try:
        src = tokenize(text)
        data = decode_item(src.next, src.next())
        for token in src: # look for more tokens
            raise SyntaxError("trailing junk")
    except (AttributeError, ValueError, StopIteration):
        raise SyntaxError("syntax error")
    return data

data = open("torrent.torrent", "rb").read()

torrent = decode(data)

pieces = torrent['info']['pieces']
piece_len = 20
pieces_count = len(pieces)/piece_len

list_pieces = []
for x in range(pieces_count):
    current = (x*piece_len)
    list_pieces.append(binascii.hexlify(bytearray(pieces[current:current+piece_len])))

torrent['info']['pieces'] = list_pieces
print torrent

And here a readable torrent file.

{
  "info": {
    "length": 7250102,
    "piece length": 16384,
    "name": "The Daily Dweebs 720p.mp4",
    "pieces": [
      "b6ef23d7eef8330f1ab3bd59f5881696c0df63f4",
      "0596f1abc2002a54b47400fdbe727c1835236147",
      "ef9fff6c95447391376d8b6a3c6f160ac81094e6",
      ...
      "278c4c3d379e983020912993b07cee828100c499",
      "2d3a066c7912a233905e787e5069d60ad1c79517",
      "c49cddb85e8f14014017c6a7ddb6f16084b9872b",
      "bdc4d96dd00dbea5912a37f38560c7c1aaa120a3"
    ]
  },
  "encoding": "UTF-8",
  "creation date": 1529449174,
  "announce-list": [
    [
      "wss://video.blender.org:443/tracker/socket"
    ],
    [
      "https://video.blender.org/tracker/announce"
    ]
  ],
  "created by": "PeerTube",
  "announce": "wss://video.blender.org:443/tracker/socket",
  "url-list": [
    "https://video.blender.org/static/webseed/b37a5b9f-e6b5-415c-b700-04a5cd6ec205-720.mp4"
  ]
}

Everything else is perfectly explained here: https://en.wikipedia.org/wiki/Torrent_file

Magnet Links

More About Magnet Links: https://en.wikipedia.org/wiki/Magnet_URI_scheme

Trackers

A tracker reference to nodes that give you a piece of the file. Also keep in mind the checksums are binary and must be encoded as binary not as representative hex string. For URLs simply using the "%nn" encoding.

Definition of General Requests

Announcement Request

This request is announce your current download status to the tracker.

Scrape Request

This request get you a overview of the peers that are currently connected to the tracker.

HTTP/HTTPS

This trackers are the simplest trackers. For HTTP Connections the The Safe Habor Problem is relevant too.

UDP/TCP

UDP is used for the server here and the client actually using TCP to transfer the data to you. For both of these connection The Safe Habor Problem is relevant.

UDP Requests

Connect Request

This request is for obtaining a id that you can use later in other requests.

Announcement Request

Scrape Request

TCP Requests

Webtorrent

WebRTC

WebRTC is not protocol but a Collection of Interfaces that can be used for Peer2Peer or Peer2Gateway2Peer. The Interfaces are based on NAT Protocols and using in end the RTP or RTCP.

NAT

TURN

STUN

P2P

RTP

RTCP

Websocket

Websocket connection are keeped like a big mystery, because the protocol is ugly and no web dev wants to known what their are using, they just want that the thing work. For the websocket protocol their is currently only one webtorrent javascript interface that every web app seems to using. Their is no protocol description at all. For unencrypted websocket only connection The Safe Habor Problem is relevant.

Websocket only works in combination with WebRTC because you can not open a connection directly to another peer in the browser.

DNS Protocols

The Safe Habor Problem

If their is no encryption so you need to have a safe internet connection or a safe habor vpn (My Recommendation is ipredator.se). This Problem strongly depends on you ISP (Internet Provider) and Internet Connection Providers. Their are ISP and Internet Connection Providers that do evil stuff and thats somehow is one hand of the problem. My experiences with this kind providers are that their is no save habor. So choosing a VPN is only solution that moving this problem to another provider and also add another provider to the stack of providers that are used to create the internet routing/connection, the VPN provider. Also the connection to the VPN depends on the same structure, so on quantity aspect of the providers that are used a VPN is not a safe habor.

Your ICP Struct + Your ISP Struct + Your Side Internet Struct
+ VPN Provider Structure + VPN ICP Struct + VPN ISP Struct + VPN Side Internet Struct
= not a safe habor

You are not using a VPN warning (?)

Before we init a connection to these construct, I think it is important that we warn users that not using a vpn. So we need to detect if the user is using vpn to warn the users that not using a vpn.

We can do this by using the ip address check sites, which is a heavy task because their are mutiple out their. If we want to start with this the simplest site is https://check.ipredator.se/ (ip detection working without js).

The other way is simple but not safe enough I think. Android got the possibility of create a VPN Interface. This is used by VPN Providers but also by Adblocking Apps that not really care about your Internet Connection their care mostly about ads (thats somehow their jobs). As long as I know its currently not possible to detect the app that init the network. We can detect with the ConnectivityManager if their is a VPN Connection with the methods getAllNetworks (for lolipop and upper) and getAllNetworkInfo (api v1-lolipop). For lolipop we should use hasTransport with TRANSPORT_VPN and for all other versions we should use getType with equal check for ConnectivityManager.TYPE_VPN (maybe we need a existing check for this constant)

Proxy Solution

A alternative would be to host tracker proxy, this is somehow the same solution and get also the general problems that I mention above. On the user side this is the simplest method. This needs to be setup with a RSA Encryption Protocol. And normaly their is only a little RSA Encryption Protocol for exchanging a shared secret and then using this shared secret for encrypting with AES. So we need for this a proper UDP and TCP Protocol proxy that first handle a handshake over RSA and share a secret for AES encryption later, if their is any protocol that is a solution for this, let me know. Depended on my current knowledge we could also use RSA, but shouldn't cause their can be multiple attacks to a RSA Protocol.

Solution: "Simply" use a libsodium bindings for this. :-) (PGP emails are here a different because its a encryption extension for the content of the mail, MIME Headers and Pseudo HTTP Features should be the Email Content)

Need more Content and Explainations !!!1!

yep, I am working on it

thx to https://wiki.theory.org/index.php/BitTorrentSpecification this helps a lot but is somehow is also confusing

Nutomic commented 6 years ago

I am starting a bounty on this issue. I will pay 250€ to whoever implements Peertube integration in Newpipe. Required features to receive the bounty are:

Display trending videos on the main screen of Newpipe
Search for videos/channels
Display videos/channels
Download videos (Will work automatically)
Make NewPipe automaticity choose an instance from joinpeertube.com

You should also commit to maintaining these features in the future, after the bounty is completed.

@theScrabi will decide when the bounty is completed. If multiple people implement these features, he will decide how the bounty gets split up.

I will pay out the money in the cryptocurrency of your choice (it has to be available on Shapeshift).

Edit: Just in case this wasn't clear, anyone is eligible for the bounty, including Newpipe team members. All that matters is that you (help to) implement the Peertube support.

theScrabi commented 6 years ago

Preference to select a Peertube instance (I suggest you select an instance as default to make it easier for the user, eg my instance https://peertube.social)

Maybe we can add a list of instances if the ping is to high, or (if possible to detect) if the response takes to long. @TheAssassin maybe we will be able to host our own instance one day.

FlorianSteenbuck commented 6 years ago

We can use https://instances.joinpeertube.org and the ajax endpoint they are using: https://instances.joinpeertube.org/api/v1/instances?start=0&count=500

Chocobozzz commented 6 years ago

Hi, I'm the developer of peertube.

@FlorianSteenbuck yes you can. Sorry, it's not documented yet. If you need help or some additional parameters regarding this endpoint or anything else, do not hesitate.

rugk commented 6 years ago

BTW as someone mentioned in this the first comment: I would not consider NewPipe to have 100% PeerTube support unless it actually (also) uses p2p. Unless that is done, it's just straining the server with the API and not making advantage of the unique thing about PeerTube, so that is not completly supporting PeerTube then.

theScrabi commented 6 years ago

BTW as someone mentioned in this the first comment: I would not consider NewPipe to have 100% PeerTube support unless it actually (also) uses p2p. Unless that is done, it's just straining the server with the API and not making advantage of the unique thing about PeerTube, so that is not completly supporting PeerTube then.

Thats true, but I guess on a phone you don't want to do that always, especially if you are in a cellular network (in germany). If we implement p2p we should add some switch that only enables this if the network connection is wifi, and good enough.

rugk commented 6 years ago

Of course you may add a switch, and maybe also a switch, which can enable it for wifi, but disable it for mobile connections, sure. But it should be on by default (at least wifi), as that's the way PeerTube works.

theScrabi commented 6 years ago

So yes on mobile make it opt out and on wifi make it opt in sounds good already :).

rugk commented 6 years ago

@theScrabi I'd suggest the reverse, but yeah… :smile:

rugk commented 6 years ago

What you could also try, of course, is that you only download on mobile connections from p2p connections, but not seed/upload.

theScrabi commented 6 years ago

What you could also try, of course, is that you only download on mobile connections from p2p connections, but not seed/upload.

Yes this whay a specific server does not have to suffer.

However what i ment with switch was not some UI ellement, i would much rather prefere some algorithm that decides that. This way the frontend does not have to be changrd and the user does not have to be bothered.

rugk commented 6 years ago

What kind of algorithm? I think only the user can decide whether they have a big enough mobile data plan, so they can upload/share data.

TheAssassin commented 6 years ago

@rugk that won't work well. Who would take into account all that data just for using PeerTube on mobile data? When in a WiFi, it's perfectly fine to upload data, otherwise it should be disabled. Otherwise, NewPipe will be blamed for its high traffic consumption.

rugk commented 6 years ago

That's exactly what I meant:

disable p2p upload on mobile
enable p2p upload on wifi
but just let the user decide, maybe they also want to use it on mobile, or don't want to use it on wifi

theScrabi commented 6 years ago

Thinking about all once more what @rugk says is not bad. I mean p2p via cellular ... well I would not enable it, and how much sense it would make is something else, but making it opt out sounds ok for me.

On the other hand making p2p via wifi opt in, but letting the user decide weather he wants to disable it (maybe because of a public hotspot, or limited data access) is also a good idea, and in the end we may not have to let NewPipe automatically decide whether we should upload or not.

For now I'd say first we need support for the PeerTube Service, once we have that we can continue discussing about the data plan :)

rugk commented 6 years ago

I mean p2p via cellular ... well I would not enable it […] making it opt out sounds ok for me.

That's opt-in: P2P is disabled by default, but you can enable (opt-in to) it.

making p2p via wifi opt in, […] letting the user decide weather […] want[…] to disable it

And that's opt-out: Enabled by default, but user can disable (opt-out) of it.

You just seem to confuse opt-in/out or have another definition or it (do you consider "not p2p" the "true" state?).

spaetz commented 6 years ago

Let me top up the bounty with another 50€ :-). P.S. As for the uploading, there are Android APIs on whether you are on a metered network or not (see https://stackoverflow.com/questions/23877476/android-why-connectivitymanager-isactivenetworkmetered-always-returning-true)

theScrabi commented 6 years ago

You just seem to confuse opt-in/out or have another definition or it (do you consider "not p2p" the "true" state?).

Aaah logig error :D. I would have said that if something is not enabled by default, but you can enable it its opt out. Am I wrong?

rugk commented 6 years ago

I would have said that if something is not enabled by default, but you can enable it its opt out. Am I wrong?

That's opt-in. :smile:

"opt" is actually a real English word more or less meaning "to decide to do sth.". So opt-in e.g. means "to decide to get in(to) something" e.g. in a newsletter or so. "Opt-out" thus means to "decide to get/get out of sth.".

theScrabi commented 6 years ago

Optional out and optional in then, that makes more sense :D

ghost commented 6 years ago

P2Play v0.1 released (Sep 16, 2018) — first open-source (GPLv3+) PeerTube player for Android

So, is it possible reuse some sources of P2Play for NewPipe or maybe need call Ivan Agosto, P2Play developer, for help?

P.S.: Think, should give bounty to Ivan Agosto ;)

FlorianSteenbuck commented 6 years ago

@Symbian9 This is just a app that using the embed url in a WebView. Maybe in the future the dev get the head more into WebRTC, WebTorrent, Bencode, UDP- and HTTP Torrent-Trackers. Then we can talk about a implementation. Currently we can only use this project to get inspired from the API implementation. And even this strongly depends on Google/Android APIs that we not using for Json Parsing.

theScrabi commented 6 years ago

Well NewPipe can run Javascript code, maybe we can somehow wrap the player JS code with java and than use it from the newpipe site. @FlorianSteenbuck what do you think?

BasixKOR commented 6 years ago

Implementation Note: WebTorrent developers have made some a few changes to the tracker protocol In order to support WebRTC's connection model.¹

1: https://github.com/webtorrent/webtorrent/blob/master/docs/faq.md#how-does-webtorrent-work

FlorianSteenbuck commented 6 years ago

@theScrabi yep this can work but as well as I know their is no DOM, WebRTC, RTP or RTCP interface in rhino. Also I think it is not good idea to depend on too much javascript code from the site we implement. For a reason the usage of internal apis worked in the past. Javascript code can get changed and can try to detect the execution interface, which can end up in block of the execution or injections. Which could be a thing because PeerTube is open source and can be customized by the instance owner. What would help are java libraries for the single protocols or stack of protocols (webrtc). Their is a Stun Library and a ICE Library that could help us with Stun and the SDP Offer System.

@BasixKOR Currently I have not done tests what got changed but I hope they remove or changed the SDP offer system over websocket, which is the most confusing and hard part to implement.

My current investigation are in the field of DNS over a encrypted Protocol (DNScrypt) and namecoin. DNScrypt in the end ends up in the safe habor problem. This investigation is needed for strict secure code that I would recommend, because NewPipe needs to be usable securely by normal users.

Another Problem is my current Internet Connection which drops randomly, that is currently a blocker/down for me. Feel free to implement the stuff yourself.

rugk commented 6 years ago

My current investigation are in the field of DNS over a encrypted Protocol (DNScrypt) and namecoin.

What does this have to do with this issue? If the answer is nothing may there be an appropriate issue, explaining this in more detail (what you want to do etc.), you can link to? If not, could you create one?

Nutomic commented 6 years ago

FYI here is a Java implementation of webtorrent. Might be a good starting point.

https://github.com/svn2github/vuze-plugins/tree/master/azwebtorrent/org/parg/azureus/plugins/webtorrent

FlorianSteenbuck commented 6 years ago

@Nutomic this helps a lot, but currently depending on the usage of self referencing plugin api. It is quit hard to read and may have Javascript Backend according to the name of some Classes maybe someone can figure out where the server or client actually create and parse Data Packs. The Interfaces seem to be a good as their provide a offer parsing, but can not and don't have the time to figure out where actual data transfer happend.

Now we got more Resources for the WebTorrent Protocol, so if someone want to implement PeerTube start with WebTorrent.

According to my knowledge WebTorrent is more secure than BitTorrent. I think even so secure that you can use it on mobil devices. Metered Data Contracts as not detectable, so let the user decide maybe with popup on mobil network connections.

FlorianSteenbuck commented 6 years ago

@rugk this may be off topic, but DNS Security is quit important for a lot of things and here important for as I mention before TURN or STUN Server DNS Discovery. DNScrypt is a Collection of Security Extension Protocols for DNS: DNS over TLS, DNS over DTLS and DNS over HTTPS Namecoin is a Blockchain that contain multiple domain names for the .bit domain.

Optional Namecoin Support is not only important for security it is also Important for include PeerTube Instances that got a .bit domain. Their are DNS Servers that support this domain, but with the disappearing DNS Server List and a new RPC Application Namecoin clearly state that they not want that people use the DNS Protocol for .bit resolving. Also Namecoin can support basically every kind of redirect from their DNS JSON Data that can be passed into a string, which includes DNScrypt Stamps, I2P and Tor.

If we want to split this I would suggest to move the safe harbor problem to a new issue.

rugk commented 6 years ago

as I mention before TURN or STUN Server DNS Discovery.

Okay, never heard of this discovery method. Anyway, I think once you have the domain name of a PeerTube instance (and you need to have this), they very likely have some API/way to query the TURN server details and such stuff or you just use one you trust. Actually, I don't know if PeerTube even uses TURN or what exactly they do there. (as obviously the originating server has to "broadcast" the video/torrent somehow, when no one else is streaming it currently)

DNScrypt is a Collection of Security Extension Protocols for DNS: DNS over TLS, DNS over DTLS and DNS over HTTPS

Nowadays, yes, in the past it DNSCrypt was it's own protocol and the software still includes it but offers support for the other stuff, as you mentioned.

Optional Namecoin Support is not only important for security it is also Important for include PeerTube Instances that got a .bit domain.

Possibly, yes, however do you really think the effort of implementing a blockchain and stuff is worth it? Especially, as there are not even any PeerTube servers I currently know that use it. And if there were, you could argue in the same way you now need to implement .onion support just because a few PeerTube servers use it. (or, as that is relatively easy and may already work, totally different blockchain-based TLD) Or things like OpenNIC etc. You can continue here…

Getting back to "TURN or STUN Server DNS Discovery" I see it is 1. "optional" 2. it is just STUN. Remember what STUN is (note) it is really just for returning your own IP addresses so you can connect. It literally does not matter for security reasons what STUN server you use or whether the response is intercepted or so. The only thing an attacker could do is block the connection and you may then not be able to stream the video.

So, all this DNS stuff is totally complicated and unlikely to be worth implementing – at least considering Peertube.

yausername commented 6 years ago

@FlorianSteenbuck are you working on this? If not I am thinking of writing the extractor for peertube (using APIs) and video playback using https://github.com/butterproject/TorrentStream-Android.

rigelk commented 6 years ago

@yausername beware, TorrentStream-Android relies on jlibtorrent, which AFAIK doesn't support WebTorrent.

yausername commented 6 years ago

Yes I am aware of that but I thought of trying it for a start. I guess we need to use https://webrtc.org/native-code/android/ for webrtc support as suggested https://github.com/Chocobozzz/PeerTube/issues/250

FlorianSteenbuck commented 6 years ago

as I mention before TURN or STUN Server DNS Discovery.

Okay, never heard of this discovery method. Anyway, I think once you have the domain name of a PeerTube instance (and you need to have this), they very likely have some API/way to query the TURN server details and such stuff or you just use one you trust. Actually, I don't know if PeerTube even uses TURN or what exactly they do there. (as obviously the originating server has to "broadcast" the video/torrent somehow, when no one else is streaming it currently)

STUN DNS Discovery is normed by DNS RFC which is used for get the first STUN Server this are fixed domain names, this variables are completely unchangeable in the browser (please proof me wrong, this seem to be a bug in my eyes). Then they connect to the Tracker and get JSON Format that support binary content in json strings and get over this mutiple Offers over the SDP Norm for RTCP, but it seems to be used for the next STUN server and then we got a P2P DTLS Connection, that I am unable to decrypt.

Optional Namecoin Support is not only important for security it is also Important for include PeerTube Instances that got a .bit domain.

Possibly, yes, however do you really think the effort of implementing a blockchain and stuff is worth it? Especially, as there are not even any PeerTube servers I currently know that use it. And if there were, you could argue in the same way you now need to implement .onion support just because a few PeerTube servers use it. (or, as that is relatively easy and may already work, totally different blockchain-based TLD) Or things like OpenNIC etc. You can continue here…

You are right its a huge effort to implementing as I call them In-Network Protocols.

But it turns out to be not that much effort for only namecoin and onion as they are currently two libarys out their orchid namecoinj. Of course I would to suggest to understand the network structures, protocols and the implementation to rewrite a lighter version of these, because their are depending on too much google code.

But anyways the current recommend usage of namecoin is over the namecoin rpc interface which can be simply used with one dependency or even a one class implementation.

Getting back to "TURN or STUN Server DNS Discovery" I see it is 1. "optional" 2. it is just STUN. Remember what STUN is (note) it is really just for returning your own IP addresses so you can connect.

TURN is STUN extend with a Channel Feature

TURN is got a channel feature that not only return you own IP

Both Protocols are describe a way to communicate IP Addresses and not only for return your own IP

STUN servers are used to get a list of IP addresses used by both conversation partners

The only thing an attacker could do is block the connection and you may then not be able to stream the video.

This is the main feature of PeerTube, the thing is you going to be able to stream the video, because PeerTube do not force the user to use WebRTC, they got a fallback to HTTP Range Requests.

PeerTube is crap, it just using a framework that using a new technology that some guys from google created, with old technology that is only used for SIP and NAT, to realize in the end a p2p video stream in a browser and a old technology that is used for p2p file sharing.

A attacker could not only block your connections, they can trace your through your connections, which shouldn't be possible at all.

So, all this DNS stuff is totally complicated and unlikely to be worth implementing – at least considering Peertube.

DNS is one of the biggest mistake that was made in the history of the Internet. I only suggest to implement a encrypted version of this. Namecoin was only a Idea and is Possible in light way as I mentioned above.

Maybe I am little bit radical, but Privacy is equal important as Security.

A issue split is still ok for me @theScrabi

FlorianSteenbuck commented 6 years ago

@yausername

@FlorianSteenbuck are you working on this? If not I am thinking of writing the extractor for peertube (using APIs) and video playback using https://github.com/butterproject/TorrentStream-Android.

Currently I am not working on this caused by to many problems in private life.

Yes I am aware of that but I thought of trying it for a start. I guess we need to use https://webrtc.org/native-code/android/ for webrtc support as suggested Chocobozzz/PeerTube#250

As you already know me I not so cool with using google dependency so maybe you can implement it yourself the source code seems to readable: https://webrtc.googlesource.com/src/+/master/examples/androidapp/src/org/appspot/apprtc

rugk commented 6 years ago

@FlorianSteenbuck

PeerTube is crap

Then why are you still commenting in this issue? If you don't like/want it here, go somewhere else and complain about it. This way, it does not contribute to this issue here. I only miss your comment "NewPipe is crap, because it depends on Google services, i.e. YouTube.".

FlorianSteenbuck commented 6 years ago

@rugk PeerTube is crap, but is still a challenge for a dev to implement. That is the reason why I still comment on this issue. Most of the Mozilla devs can not implement WebRTC. It is extremly hard to implement this thing. And it is important to implement it to understand what are problematic s are and understand how it actually works. Learning by Doing

NewPipe is great, because it providing a structure for State of the Art Media Platforms in a logic way

I complaining more about Android dependencies than Google dependencies If you only look at the Source Code of WebRTC you can see that they clearly using Android Classes for the implementation.

Their are multiple things that I mention without my opinion Sad to see you only want to bash my opinion and take this out of context and even mix it with something that I would never say.

But if you want go to this level I also could: (Don't take this to serious I just try to show you how wrong you can be with a judging someone to quickly) @rugk do you got a life ? It seems to be the case that you a got in the abstract understanding of threema, but it seem to be the case that you now after creating a threema library only talking on issues instead of solving issues. That would be absolutely ok, if you know what you are talking about, but it seem to be the case that you not reading the rfc and not reading the norms. You are just one of these web devs that are fascinated by new fancy stuff that is not available in older versions of the languages. You are just some of these jquery devs that do not known that even their fancy loop like function ($.each) can be implemented simply and can be even made more useful by own implementation than their would thought. You not want to solve problems, you want to use fancy stuff to made a whole shit of money of it.

If you want detailed explanation, why this are my first thought about you and your profile, let me know.

supermamie commented 5 years ago

Just 2 cent about an Android app for PeerTube : https://github.com/sschueller/peertube-android

It seems it is not using WebRTC yet

TheAssassin commented 5 years ago

It seems it is not using WebRTC yet

It was suggested already to add PeerTube support w/o the webtorrent stuff to have a first version in NewPIpe. A "proper" implementation could then be written eventually.

yausername commented 5 years ago

It seems it is not using WebRTC yet

It was suggested already to add PeerTube support w/o the webtorrent stuff to have a first version in NewPIpe. A "proper" implementation could then be written eventually.

I already added peertube service in newpipe extractor https://github.com/yausername/NewPipeExtractor/tree/peertube and it works well with NP app. But I am struggling with WebRTC data channels.

TeamNewPipe / NewPipeExtractor