Bugout doesn't work with HTTP, HTTPS tracker servers

[draeder]

Is this expected if the trackers are not ws or wss?

I have been working on building an npm module that tests and returns working tracker servers listed on https://github.com/ngosang/trackerslist, specifically the lists for ws, wss, http, and https tracker servers. I also built my own tracker server that runs with wss, ws, https, and http.

In my testing with Bugout (in the browser: both using express, and straight html) to make sure my module is working correctly, I found that known good tracker servers that use http and https, including my own tracker server, do not work with Bugout. No error is returned; simply nothing happens.

One other thing I noticed is that the node version of Bugout code doesn't ever emit the "Low level WebTorrent events". Only the browser version of Bugout code does that (with or without ws/wss tracker servers).

The following is the extended Bugout logging for a browser example where I try to pass some known-good http/https tracker servers into Bugout. I wonder if this is some kind of CORS or HTTP header issue?

bugout torrent +34ms my cool bugout 
X {_events: {…}, _eventsCount: 2, _maxListeners: undefined, _debugId: "2134521", client: E, …}
announce: Array(3)
0: "http://explodie.org:6969/announce"
1: "https://tracker.coalition.space:443/announce"
2: "https://tracker.tamersunion.org:443/announce"
length: 3
__proto__: Array(0)
bitfield: t.exports {grow: 0, buffer: Uint8Array(1)}
client: E {_events: {…}, _eventsCount: 0, _maxListeners: undefined, peerId: "2d5757303030372d737958326c49484350496c74", peerIdBuffer: Uint8Array(20), …}
created: Wed Feb 17 2021 01:17:11 GMT-0600 (Central Standard Time) {}
createdBy: "WebTorrent/0007"
destroyed: false
discovery: t.exports {_events: {…}, _eventsCount: 5, _maxListeners: undefined, peerId: "2d5757303030372d737958326c49484350496c74", infoHash: "2134521fb8ca64df004f82e2480c7b7683dec588", …}
done: true
files: [h]
info: {length: 14, name: Uint8Array(14), piece length: 16384, pieces: Uint8Array(20)}
infoBuffer: Uint8Array(90) [100, 54, 58, 108, 101, 110, 103, 116, 104, 105, 49, 52, 101, 52, 58, 110, 97, 109, 101, 49, 52, 58, 109, 121, 32, 99, 111, 111, 108, 32, 98, 117, 103, 111, 117, 116, 49, 50, 58, 112, 105, 101, 99, 101, 32, 108, 101, 110, 103, 116, 104, 105, 49, 54, 51, 56, 52, 101, 54, 58, 112, 105, 101, 99, 101, 115, 50, 48, 58, 107, 126, 52, 101, 105, 171, 28, 240, 19, 137, 213, 141, 196, 109, 95, 226, 212, 103, 142, 48, 101]
infoHash: "2134521fb8ca64df004f82e2480c7b7683dec588"
infoHashBuffer: Uint8Array(20) [33, 52, 82, 31, 184, 202, 100, 223, 0, 79, 130, 226, 72, 12, 123, 118, 131, 222, 197, 136]
lastPieceLength: 14
length: 14
magnetURI: "magnet:?xt=urn:btih:2134521fb8ca64df004f82e2480c7b7683dec588&dn=my+cool+bugout&tr=http%3A%2F%2Fexplodie.org%3A6969%2Fannounce&tr=https%3A%2F%2Ftracker.coalition.space%3A443%2Fannounce&tr=https%3A%2F%2Ftracker.tamersunion.org%3A443%2Fannounce"
maxWebConns: 4
metadata: Uint8Array(369) [100, 56, 58, 97, 110, 110, 111, 117, 110, 99, 101, 51, 51, 58, 104, 116, 116, 112, 58, 47, 47, 101, 120, 112, 108, 111, 100, 105, 101, 46, 111, 114, 103, 58, 54, 57, 54, 57, 47, 97, 110, 110, 111, 117, 110, 99, 101, 49, 51, 58, 97, 110, 110, 111, 117, 110, 99, 101, 45, 108, 105, 115, 116, 108, 108, 51, 51, 58, 104, 116, 116, 112, 58, 47, 47, 101, 120, 112, 108, 111, 100, 105, 101, 46, 111, 114, 103, 58, 54, 57, 54, 57, 47, 97, 110, 110, 111, 117, 110, 99, …]
name: "my cool bugout"
path: "/webtorrent/2134521fb8ca64df004f82e2480c7b7683dec588"
paused: false
pieceLength: 16384
pieces: [null]
ready: true
received: 0
skipVerify: true
store: t.exports {store: n, chunkLength: 16384, mem: Array(1)}
strategy: "sequential"
torrentFile: Uint8Array(369) [100, 56, 58, 97, 110, 110, 111, 117, 110, 99, 101, 51, 51, 58, 104, 116, 116, 112, 58, 47, 47, 101, 120, 112, 108, 111, 100, 105, 101, 46, 111, 114, 103, 58, 54, 57, 54, 57, 47, 97, 110, 110, 111, 117, 110, 99, 101, 49, 51, 58, 97, 110, 110, 111, 117, 110, 99, 101, 45, 108, 105, 115, 116, 108, 108, 51, 51, 58, 104, 116, 116, 112, 58, 47, 47, 101, 120, 112, 108, 111, 100, 105, 101, 46, 111, 114, 103, 58, 54, 57, 54, 57, 47, 97, 110, 110, 111, 117, 110, 99, …]
uploaded: 0
urlList: []
wires: []
_amInterested: false
_critical: []
_debugId: "2134521"
_downloadSpeed: ƒ (e)
_events: {wire: Array(2), close: ƒ}
_eventsCount: 2
_fileModtimes: undefined
_getAnnounceOpts: undefined
_hashes: ["6b7e346569ab1cf01389d58dc46d5fe2d4678e30"]
_maxListeners: undefined
_peers: {}
_peersLength: 0
_queue: []
_rarityMap: t.exports {_torrent: X, _numPieces: 1, _pieces: Array(1), _onWire: ƒ, _onWireHave: ƒ, …}
_rechokeIntervalId: 9
_rechokeNumSlots: 10
_rechokeOptimisticTime: 0
_rechokeOptimisticWire: null
_reservations: [null]
_selections: []
_servers: []
_store: ƒ n(t,r)
_uploadSpeed: ƒ (e)
_xsRequests: []
downloadSpeed: (...)
downloaded: (...)
numPeers: (...)
progress: (...)
ratio: (...)
swarm: (...)
timeRemaining: (...)
torrentFileBlobURL: (...)
uploadSpeed: (...)
_numConns: (...)
_numQueued: (...)
__proto__: r
addPeer: ƒ addPeer(t)
addWebSeed: ƒ addWebSeed(e)
constructor: class X
createServer: ƒ createServer(e)
critical: ƒ critical(e,t)
deselect: ƒ deselect(e,t,r)
destroy: ƒ destroy(e)
downloadSpeed: (...)
downloaded: (...)
getFileModtimes: getFileModtimes(e){const t=[];v(this.files.map((e,r)=> {…}
load: load(e,t){if(this.destroyed)throw new Error("torrent is destroyed");if(!this.ready)return this.once("ready",()=> {…}
numPeers: (...)
pause: ƒ pause()
progress: (...)
ratio: (...)
removePeer: ƒ removePeer(e)
rescanFiles: rescanFiles(e){if(this.destroyed)throw new Error("torrent is destroyed");e||(e=p),this._verifyPieces(t=> {…}
resume: ƒ resume()
select: select(e,t,r,n){if(this.destroyed)throw new Error("torrent is destroyed");if(0>e||t<e||this.pieces.length<=t)throw new Error(`invalid selection ${e} : ${t}`);r=+r||0,this._debug("select %s-%s (priority %s)",e,t,r),this._selections.push({from:e,to:t,offset:0,priority:r,notify:n||p}),this._selections.sort((e,t)=> {…}
swarm: (...)
timeRemaining: (...)
torrentFileBlobURL: (...)
uploadSpeed: (...)
_addIncomingPeer: ƒ _addIncomingPeer(e)
_addPeer: ƒ _addPeer(e)
_checkDone: _checkDone(){if(this.destroyed)return;this.files.forEach(e=> {…}
_debug: ƒ _debug()
_destroy: _destroy(e,t){if(!this.destroyed){for(const e in this.destroyed=!0,this._debug("destroy"),this.client._remove(this),clearInterval(this._rechokeIntervalId),this._xsRequests.forEach(e=> {…}
_drain: _drain(){if(this._debug("_drain numConns %s maxConns %s",this._numConns,this.client.maxConns),"function"!=typeof x.connect||this.destroyed||this.paused||this._numConns>=this.client.maxConns)return;this._debug("drain (%s queued, %s/%s peers)",this._numQueued,this.numPeers,this.client.maxConns);const e=this._queue.shift();if(!e)return;this._debug("tcp connect attempt to %s",e.addr);const t=u(e.addr),r={host:t[0],port:t[1]},n=e.conn=x.connect(r);n.once("connect",()=> {…}
_gcSelections: ƒ _gcSelections()
_getMetadataFromServer: _getMetadataFromServer(){function e(e,r){function n(n,a,s){if(t.destroyed)return r(null);if(t.metadata)return r(null);if(n)return t.emit("warning",new Error(`http error from xs param: ${e}`)),r(null);if(200!==a.statusCode)return t.emit("warning",new Error(`non-200 status code ${a.statusCode} from xs param: ${e}`)),r(null);let o;try{o=C(s)}catch(e){}return o?o.infoHash===t.infoHash?void(t._onMetadata(o),r(null)):(t.emit("warning",new Error(`got torrent file with incorrect info hash from xs param: ${e}`)),r(null)):(t.emit("warning",new Error(`got invalid torrent file from xs param: ${e}`)),r(null))}if(0!==e.indexOf("http://")&&0!==e.indexOf("https://"))return t.emit("warning",new Error(`skipping non-http xs param: ${e}`)),r(null);let a;try{a=y.concat({url:e,method:"GET",headers:{"user-agent":G}},n)}catch(n){return t.emit("warning",new Error(`skipping invalid url xs param: ${e}`)),r(null)}t._xsRequests.push(a)}const t=this,r=Array.isArray(this.xs)?this.xs:[this.xs],n=r.map(t=> {…}
_hotswap: ƒ _hotswap(e,t)
_markAllVerified: ƒ _markAllVerified()
_markVerified: ƒ _markVerified(e)
_numConns: (...)
_numQueued: (...)
_onListening: ƒ _onListening()
_onMetadata: _onMetadata(e){if(this.metadata||this.destroyed)return;this._debug("got metadata"),this._xsRequests.forEach(e=> {…}
_onParsedTorrent: _onParsedTorrent(e){if(!this.destroyed){if(this._processParsedTorrent(e),!this.infoHash)return this._destroy(new Error("Malformed torrent data: No info hash"));(this.path||(this.path=B.join(Y,this.infoHash)),this._rechokeIntervalId=setInterval(()=> {…}
_onStore: ƒ _onStore()
_onTorrentId: _onTorrentId(e){if(this.destroyed)return;let t;try{t=C(e)}catch(e){}t?(this.infoHash=t.infoHash,this._debugId=t.infoHash.toString("hex").substring(0,7),n.nextTick(()=> {…}
_onWire: _onWire(e,t){if(this._debug("got wire %s (%s)",e._debugId,t||"Unknown"),e.on("download",e=> {…}
_onWireWithMetadata: _onWireWithMetadata(e){let t=null;const r=()=> {…}
_processParsedTorrent: ƒ _processParsedTorrent(e)
_rechoke: _rechoke(){if(!this.ready)return;0<this._rechokeOptimisticTime?this._rechokeOptimisticTime-=1:this._rechokeOptimisticWire=null;const e=[];this.wires.forEach(t=> {…}
_request: _request(e,t,a){function o(){n.nextTick(()=> {…}
_startDiscovery: _startDiscovery(){if(this.discovery||this.destroyed)return;let e=this.client.tracker;e&&(e=Object.assign({},this.client.tracker,{getAnnounceOpts:()=> {…}
_update: ƒ _update()
_updateInterest: _updateInterest(){const e=this._amInterested;this._amInterested=!!this._selections.length,this.wires.forEach(e=> {…}
_updateSelections: _updateSelections(){!this.ready||this.destroyed||(n.nextTick(()=> {…}
_updateWire: _updateWire(e){function t(t,r,n,a){return s=> {…}
_updateWireWrapper: ƒ _updateWireWrapper(e)
_validAddr: ƒ _validAddr(e)
_verifyPieces: _verifyPieces(e){v(this.pieces.map((e,t)=> {…}
get downloadSpeed: ƒ downloadSpeed()
get downloaded: ƒ downloaded()
get numPeers: ƒ numPeers()
get progress: ƒ progress()
get ratio: ƒ ratio()
get swarm: ƒ swarm()
get timeRemaining: ƒ timeRemaining()
get torrentFileBlobURL: ƒ torrentFileBlobURL()
get uploadSpeed: ƒ uploadSpeed()
get _numConns: ƒ _numConns()
get _numQueued: ƒ _numQueued()
__proto__:
addListener: ƒ (e,t)
emit: ƒ (e)
eventNames: ƒ ()
getMaxListeners: ƒ ()
listenerCount: ƒ _(e)
listeners: ƒ (e)
on: ƒ (e,t)
once: ƒ (e,t)
prependListener: ƒ (e,t)
prependOnceListener: ƒ (e,t)
rawListeners: ƒ (e)
removeAllListeners: ƒ (e)
removeListener: ƒ (e,t)
setMaxListeners: ƒ (e)
_events: undefined
_maxListeners: undefined
constructor: ƒ r()
__proto__: Object

^^^ Nothing happens after this

[draeder]

I think I've answered my own question: browsers can only communicate with websockets servers for WebRTC signaling. So, the HTTP, HTTPS servers are accepting the torrent announcement, but the browser never hears back because the browser wants a websocket. This is why nothing else happens after announcement. Please correct me if I'm wrong...

[chr15m]

@draeder yes that's correct, sorry I didn't get back to you in time. You're exactly right that webtorrent/bugout require a websocket connection. This is because the WebRTC handshake is two way (offer and answer) unlike traditional bittorrent where you can just find a peer and then connect directly to them using TCP/IP sockets. It's a fundamental limitation of WebRTC and browsers unfortunately.

[draeder]

Thanks for confirming and no worries about any delay in response. This is just a hobby for me after my day job... You might find this module useful for your webrtc-signaling-mesh project: https://www.npmjs.com/package/fake-bittorrent-client. I'm using this in mine, after just a basic host ping, to test the viability of trackers.