Cannot get rtc-app.js to browserify

DamonOehlman commented 10 years ago

It seems that the other files with the browserify section work fine, but for some reason rtc-app.js won't work. I've had a bit of a poke around in docpad-plugin-browserifydocs but I can't see why one would work and not the other :/

balupton commented 10 years ago

I've had this happen a few times. I think it is an issue with the router not waiting for the generate to complete before serving. Will look into it tomorrow.

DamonOehlman commented 10 years ago

OK - thanks mate :)

balupton commented 10 years ago

Yeah, I can't exactly pin it down. Sometimes it works fine, sometimes it doesn't. Can you proceed with this working intermittently?

DamonOehlman commented 10 years ago

I'm working on implementing some additional smarts into rtc-quickconnect at the moment to deal with limitations in the browser knowing about when connections go away, so yeah, I'm all good for the moment.

balupton commented 10 years ago

If you git pull then npm install you'll see the error.

It is actually caused by one of rtc.io's modules, or one of rtc.io's module's dependencies, as the require of quickconnect is what breaks it, this is evidenced by setting rtc-app.js to the following:

---
browserify: true
---

require('rtc-quickconnect')

Will generate the following error:

error: Something went wrong with the action
error: An error occured: 
Error: Line 2: Unexpected token :
  at throwError (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/node_modules/esprima/esprima.js:1156:21)
  at throwUnexpected (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/node_modules/esprima/esprima.js:1213:9)
  at consumeSemicolon (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/node_modules/esprima/esprima.js:1295:13)
  at parseStatement (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/node_modules/esprima/esprima.js:2791:9)
  at parseSourceElement (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/node_modules/esprima/esprima.js:3049:20)
  at parseStatementList (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/node_modules/esprima/esprima.js:2041:25)
  at parseBlock (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/node_modules/esprima/esprima.js:2056:17)
  at parseStatement (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/node_modules/esprima/esprima.js:2727:24)
  at parseSourceElement (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/node_modules/esprima/esprima.js:3049:20)
  at parseSourceElements (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/node_modules/esprima/esprima.js:3082:29)
  at parseProgram (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/node_modules/esprima/esprima.js:3096:19)
  at parse (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/node_modules/esprima/esprima.js:3843:23)
  at module.exports (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/index.js:4:41)
  at module.exports (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/index.js:9:16)
  at Stream.end (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/index.js:63:15)
  at _end (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/through/index.js:65:9)
  at Stream.stream.end (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/through/index.js:74:5)
  at f (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/module-deps/index.js:217:19)
  at makeTransform (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/module-deps/index.js:279:46)
  at ap (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/module-deps/index.js:202:13)
  at applyTransforms (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/module-deps/index.js:219:11)
  at /Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/module-deps/index.js:171:17
  at fs.js:266:14
  at /Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/safefs/node_modules/graceful-fs/graceful-fs.js:103:5
  at /Users/balupton/Projects/interconnect/node_modules/docpad-plugin-text/node_modules/bal-util/node_modules/safefs/node_modules/graceful-fs/graceful-fs.js:103:5
  at /Users/balupton/Projects/interconnect/node_modules/docpad-plugin-sass/node_modules/safeps/node_modules/safefs/node_modules/graceful-fs/graceful-fs.js:103:5
  at /Users/balupton/Projects/interconnect/node_modules/docpad-plugin-ghpages/node_modules/safefs/node_modules/graceful-fs/graceful-fs.js:103:5
  at /Users/balupton/Projects/interconnect/node_modules/docpad/node_modules/istextorbinary/node_modules/safefs/node_modules/graceful-fs/graceful-fs.js:103:5
  at /Users/balupton/Projects/interconnect/node_modules/docpad/node_modules/safefs/node_modules/graceful-fs/graceful-fs.js:103:5
  at Object.oncomplete (fs.js:107:15)

Unfortunately such errors don't tell us what file it was actually failing for!

@DamonOehlman can you look into this?

balupton commented 10 years ago

By updating the line:

at module.exports (/Users/balupton/Projects/interconnect/node_modules/docpad-plugin-browserifydocs/node_modules/browserify/node_modules/insert-module-globals/node_modules/lexical-scope/node_modules/astw/index.js:4:41)

To be:

module.exports = function (src) {
    try {
        var ast = typeof src === 'string' ? parse(src) : src;
    }
    catch (err) {
        console.log(src, err);
        throw err;
    }
    return function (cb) {
        walk(ast, undefined, cb);
    };
};

It reveals the following source which it fails on:

{
  "name": "rtc-signaller",
  "version": "0.16.0",
  "description": "rtc.io transportless signalling for WebRTC",
  "main": "index.js",
  "directories": {
    "test": "test"
  },
  "dependencies": {
    "uuid": "~1.4.1",
    "cog": "~0.4.1",
    "collections": "~0.2.2",
    "rtc-core": "~0.7.1"
  },
  "devDependencies": {
    "tape": "~2.3.0",
    "rtc-switchboard": "~0.5.0",
    "messenger-memory": "~0.3.0",
    "mocha": "~1.17.0",
    "zuul": "~1.3.0",
    "fdom": "~0.6.3",
    "async": "~0.2.9"
  },
  "scripts": {
    "test": "node test/all.js && zuul -- test/all.js",
    "gendocs": "gendocs > README.md"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/rtc-io/rtc-signaller.git"
  },
  "keywords": [
    "rtc.io",
    "webrtc",
    "signalling"
  ],
  "author": {
    "name": "Damon Oehlman",
    "email": "damon.oehlman@nicta.com.au"
  },
  "license": "Apache 2.0",
  "bugs": {
    "url": "https://github.com/rtc-io/rtc-signaller/issues"
  },
  "contributors": [
    {
      "name": "Damon Oehlman",
      "email": "damon.oehlman@nicta.com.au",
      "url": "https://github.com/DamonOehlman"
    },
    {
      "name": "Silvia Pfeiffer",
      "email": "silvia.pfeiffer@nicta.com.au",
      "url": "https://github.com/silviapfeiffer"
    }
  ],
  "readme": "# rtc-signaller\n\nThe `rtc-signaller` module provides a transportless signalling\nmechanism for WebRTC.\n\n\n[![NPM](https://nodei.co/npm/rtc-signaller.png)](https://nodei.co/npm/rtc-signaller/)\n\n[![Build Status](https://travis-ci.org/rtc-io/rtc-signaller.png?branch=master)](https://travis-ci.org/rtc-io/rtc-signaller)\n[![unstable](http://hughsk.github.io/stability-badges/dist/unstable.svg)](http://github.com/hughsk/stability-badges)\n\n## Purpose\n\nThe signaller provides set of client-side tools that assist with the\nsetting up an `PeerConnection` and helping them communicate. All that is\nrequired for the signaller to operate is a suitable messenger.\n\nA messenger is a simple object that implements node\n[EventEmitter](http://nodejs.org/api/events.html) style `on` events for\n`open`, `close`, `message` events, and also a `send` method by which\ndata will be send \"over-the-wire\".\n\nBy using this approach, we can conduct signalling over any number of\nmechanisms:\n\n- local, in memory message passing\n- via WebSockets and higher level abstractions (such as\n  [primus](https://github.com/primus/primus))\n- also over WebRTC data-channels (very meta, and admittedly a little\n  complicated).\n\n## Getting Started\n\nWhile the signaller is capable of communicating by a number of different\nmessengers (i.e. anything that can send and receive messages over a wire)\nit comes with support for understanding how to connect to an\n[rtc-switchboard](https://github.com/rtc-io/rtc-switchboard) out of the box.\n\nThe following code sample demonstrates how:\n\n```js\n// create a new signaller, connecting to the target switchboard\nvar signaller = require('rtc-signaller')('http://rtc.io/switchboard');\n\n// when a new peer is announced, log it\nsignaller.on('peer:announce', function(data) {\n console.log('new peer found in room: ', data);\n});\n\n// when a peer leaves the switchboard, log it\nsignaller.on('peer:leave', function(id) {\n  console.log('peer ' + id + ' has left the room');\n});\n\n// for our sanity, pop a message once we are connected\nsignaller.once('connected', function() {\n  console.log('we have successfully connected');\n});\n\n// send through an announce message\n// this will occur once the primus socket has been opened and active\nsignaller.announce({ room: 'signaller-getting-started' });\n```\n\n## Signal Flow Diagrams\n\nDisplayed below are some diagrams how the signalling flow between peers\nbehaves.  In each of the diagrams we illustrate three peers (A, B and C)\nparticipating discovery and coordinating RTCPeerConnection handshakes.\n\nIn each case, only the interaction between the clients is represented not\nhow a signalling server\n(such as [rtc-switchboard](https://github.com/rtc-io/rtc-switchboard)) would\npass on broadcast messages, etc.  This is done for two reasons:\n\n1. It is out of scope of this documentation.\n2. The `rtc-signaller` has been designed to work without having to rely on\n   any intelligence in the server side signalling component.  In the\n   instance that a signaller broadcasts all messages to all connected peers\n   then `rtc-signaller` should be smart enough to make sure everything works\n   as expected.\n\n### Peer Discovery / Announcement\n\nThis diagram illustrates the process of how peer `A` announces itself to\npeers `B` and `C`, and in turn they announce themselves.\n\n![](https://raw.github.com/rtc-io/rtc-signaller/master/docs/announce.png)\n\n### Editing / Updating the Diagrams\n\nEach of the diagrams has been generated using\n[mscgen](http://www.mcternan.me.uk/mscgen/index.html) and the source for\nthese documents can be found in the `docs/` folder of this repository.\n\n## Reference\n\nThe `rtc-signaller` module is designed to be used primarily in a functional\nway and when called it creates a new signaller that will enable\nyou to communicate with other peers via your messaging network.\n\n```js\n// create a signaller from something that knows how to send messages\nvar signaller = require('rtc-signaller')(messenger);\n```\n\nAs demonstrated in the getting started guide, you can also pass through\na string value instead of a messenger instance if you simply want to\nconnect to an existing `rtc-switchboard` instance.\n\n### signaller#send(message, data*)\n\nUse the send function to send a message to other peers in the current\nsignalling scope (if announced in a room this will be a room, otherwise\nbroadcast to all peers connected to the signalling server).\n\n### announce(data?)\n\nThe `announce` function of the signaller will pass an `/announce` message\nthrough the messenger network.  When no additional data is supplied to\nthis function then only the id of the signaller is sent to all active\nmembers of the messenging network.\n\n#### Joining Rooms\n\nTo join a room using an announce call you simply provide the name of the\nroom you wish to join as part of the data block that you annouce, for\nexample:\n\n```js\nsignaller.announce({ room: 'testroom' });\n```\n\nSignalling servers (such as\n[rtc-switchboard](https://github.com/rtc-io/rtc-switchboard)) will then\nplace your peer connection into a room with other peers that have also\nannounced in this room.\n\nOnce you have joined a room, the server will only deliver messages that\nyou `send` to other peers within that room.\n\n#### Providing Additional Announce Data\n\nThere may be instances where you wish to send additional data as part of\nyour announce message in your application.  For instance, maybe you want\nto send an alias or nick as part of your announce message rather than just\nuse the signaller's generated id.\n\nIf for instance you were writing a simple chat application you could join\nthe `webrtc` room and tell everyone your name with the following announce\ncall:\n\n```js\nsignaller.announce({\n  room: 'webrtc',\n  nick: 'Damon'\n});\n```\n\n#### Announcing Updates\n\nThe signaller is written to distinguish between initial peer announcements\nand peer data updates (see the docs on the announce handler below). As\nsuch it is ok to provide any data updates using the announce method also.\n\nFor instance, I could send a status update as an announce message to flag\nthat I am going offline:\n\n```js\nsignaller.announce({ status: 'offline' });\n```\n\n### isMaster(targetId)\n\nA simple function that indicates whether the local signaller is the master\nfor it's relationship with peer signaller indicated by `targetId`.  Roles\nare determined at the point at which signalling peers discover each other,\nand are simply worked out by whichever peer has the lowest signaller id\nwhen lexigraphically sorted.\n\nFor example, if we have two signaller peers that have discovered each\nothers with the following ids:\n\n- `b11f4fd0-feb5-447c-80c8-c51d8c3cced2`\n- `8a07f82e-49a5-4b9b-a02e-43d911382be6`\n\nThey would be assigned roles:\n\n- `b11f4fd0-feb5-447c-80c8-c51d8c3cced2`\n- `8a07f82e-49a5-4b9b-a02e-43d911382be6` (master)\n\n### leave()\n\nTell the signalling server we are leaving.  Calling this function is\nusually not required though as the signalling server should issue correct\n`/leave` messages when it detects a disconnect event.\n\n### to(targetId)\n\nUse the `to` function to send a message to the specified target peer.\nA large parge of negotiating a WebRTC peer connection involves direct\ncommunication between two parties which must be done by the signalling\nserver.  The `to` function provides a simple way to provide a logical\ncommunication channel between the two parties:\n\n```js\nvar send = signaller.to('e95fa05b-9062-45c6-bfa2-5055bf6625f4').send;\n\n// create an offer on a local peer connection\npc.createOffer(\n  function(desc) {\n    // set the local description using the offer sdp\n    // if this occurs successfully send this to our peer\n    pc.setLocalDescription(\n      desc,\n      function() {\n        send('/sdp', desc);\n      },\n      handleFail\n    );\n  },\n  handleFail\n);\n```\n\n### loadPrimus(signalhost, callback)\n\nThis is a convenience function that is patched into the signaller to assist\nwith loading the `primus.js` client library from an `rtc-switchboard`\nsignaling server.\n\n### signaller process handling\n\nWhen a signaller's underling messenger emits a `data` event this is\ndelegated to a simple message parser, which applies the following simple\nlogic:\n\n- Is the message a `/to` message. If so, see if the message is for this\n  signaller (checking the target id - 2nd arg).  If so pass the\n  remainder of the message onto the standard processing chain.  If not,\n  discard the message.\n\n- Is the message a command message (prefixed with a forward slash). If so,\n  look for an appropriate message handler and pass the message payload on\n  to it.\n\n- Finally, does the message match any patterns that we are listening for?\n  If so, then pass the entire message contents onto the registered handler.\n\n### signaller message handlers\n\n#### announce\n\n```\n/announce|{\"id\": \"...\", ... }\n```\n\nWhen an announce message is received by the signaller, the attached\nobject data is decoded and the signaller emits an `announce` message.\n\n##### Events Triggered in response to `/announce`\n\nThere are two different types of `peer:` events that can be triggered\nin on peer B to calling the `announce` method on peer A.\n\n- `peer:announce`\n\n  The `peer:announce` event is triggered when a new peer has been\n  discovered.  The data for the new peer (as an JS object) is provided\n  as the first argument of the event handler.\n\n- `peer:update`\n\n  If a peer \"reannounces\" then a `peer:update` event will be triggered\n  rather than a `peer:announce` event.\n\n#### leave\n\n```\n/leave|{\"id\":\"...\"}\n```\n\nWhen a leave message is received from a peer, we check to see if that is\na peer that we are managing state information for and if we are then the\npeer state is removed.\n\n##### Events triggered in response to `/leave` messages\n\nThe following event(s) are triggered when a `/leave` action is received\nfrom a peer signaller:\n\n- `peer:leave`\n\n  The `peer:leave` event is emitted once a `/leave` message is captured\n  from a peer.  Prior to the event being dispatched, the internal peers\n  data in the signaller is removed but can be accessed in 2nd argument\n  of the event handler.\n\n## License(s)\n\n### Apache 2.0\n\nCopyright 2014 National ICT Australia Limited (NICTA)\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n     http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\n",
  "readmeFilename": "README.md",
  "homepage": "https://github.com/rtc-io/rtc-signaller",
  "_id": "rtc-signaller@0.16.0",
  "_from": "rtc-signaller@~0.16.0"
}

Which is strange, as that is valid JSON.

DamonOehlman commented 10 years ago

Thanks for tracking it down Ben - I'll have a look at this. I tried doing an isolated test of browserifying the rtc-app.js code without the --- header lines and it worked with my globally installed version of browserify, so I ruled it out (incorrectly).

DamonOehlman commented 10 years ago

I've published rtc-signaller@0.17.2 which should come in automatically if you remove node modules and reinstall. Basically, I've removed the package.json require at the moment and have hard coded the version into the module - I'm not happy with the implementation but it's something that I can fix later on (when it isn't affecting the development of another project).

silviapfeiffer commented 10 years ago

I had the same issue using rtc-signaller with browserfy yesterday. While reading package.json as a json file is fine, it's not valid JavaScript (which happens when using "require" in browserify using esprima for minification). So, read the package.json file in using JSON.parse() instead.

balupton commented 10 years ago

So, read the package.json file in using JSON.parse() instead.

Interesting, would that still get packaged up by browserify for use on the client-side? As I was under the impression browserify only bundles requires, which if so, JSON.parse'd files won't be bundled, and be missing on the client package no?

DamonOehlman commented 10 years ago

Yep, long and the short of it is that using JSON.parse() in isolation wouldn't work. You would have to combine it with a browserify transform (brfs in this case) and import the file as such:

var fs = require('fs');
var package = JSON.parse(fs.readFileSync(__dirname + '/package.json'));

You would then have access to version information and everything else from package.json in your JS files. Unfortunately this has two drawbacks:

You need to include any required browserify transforms as module dependencies (not devDependencies) to ensure the module can be browserified as required.
I have to import the full package.json file to get the version string (which is probably around 5-6) characters. As I mentioned in rtc-io/rtc-signaller#8 I wasn't really happy with this when I believed I was including approximately ~1Kb of package information in the resulting client JS. Once I realized last night (I should have worked this out earlier by the way) that the npm published version of the package.json was being used when libraries like rtc-quickconnect were getting browserified the package.json import actually added close to 13Kb to the end user JS - which is totally not acceptable.

So as a result I've moved to including a small snippet of package metadata into the rtc-signaller source:

https://github.com/rtc-io/rtc-signaller/blob/master/index.js#L13

And then last night, I added some additional functionality to versionit to be do some AST walking and update version strings that it finds (given a strict set of rules, i.e. it's part of a JS object with a key of version containing a string literal). So, now when I do a versionit bump on the signaller module that version in the source is kept in sync with the package.json. This way I don't have to worry about using browserify transforms (which add dependencies) or turning a simple module into a packaged behemoth.

Hopefully that all makes sense :)

silviapfeiffer commented 10 years ago

Yup, makes sense to me. :-)

interconnectapp / discuss

Cannot get rtc-app.js to browserify #24