Lightbeam in VR using A-Frame

[princiya]

Issue #202

Launch Lightbeam web-extension and click the VR button [below gif] to see Lightbeam in VR.

@jonathanKingston @cvan @caseyyee let me know what you think.

[princiya]

This comment is outdated. We don't require the two repos approach anymore.

---

Initially I followed the following approach: Click the VR button and this launches vr-index.html page from the Lightbeam web-extension.

The above method didn't work for the following reasons:

• vr-index.html was launched as moz-extension://xxxyyyzzz/vr-index.html
• But <a-camera> component didn't work as it requires a server to serve assets etc. from A-Frame.

The existing solution that I have in this PR requires two repos, please let me know of any other inputs to make it possible from within the web-extension without using an additional repo. Thanks.

[jonathanKingston]

But component didn't work as it requires a server to serve assets etc. from A-Frame.

We should just fix that I think. I don't know enough why that restriction is in place at the moment. However we can't merge this like it is.

This is looking really great as a demo though.

[princiya]

@jonathanKingston i think the restriction is due to serving content across cross origin issues and that's why there is a hard requirement to serve A-Frame via a server. @cvan @caseyyee shed some light?

[jonathanKingston]

@princiya why does it need to be from a server at all though? Can't the files be hosted via the add-on? We can't ship anything with lightbeam that requires an install really.

[cvan]

@princiya why does it need to be from a server at all though? Can't the files be hosted via the add-on?

yeah, that's approach I'd recommend (if it works). there's a section in the A-Frame FAQ docs that briefly addresses this.

the CORS requirement isn't the case for every type of media. like images should load fine, for example, but if you're trying to load a local file (that's not over a http:/https: protocol), fetching a JSON file using XHR (instead of fetch) will fail.

I can take a look at it - should be pretty quick to fix (but could be a non-obvious change).

[cvan]

But component didn't work as it requires a server to serve assets etc. from A-Frame.

this seems like an A-Frame bug. which part didn't work exactly? were there any console errors?

I assume if I just pull the repo, I can reproduce that?

[jonathanKingston]

I would work on fixing the issue with the server being required when hosting from the web extension and remove the need for a JSON file, instead updating the nodes as we do for the main diagram.

Perhaps we should just call this "3d view" and leave the goggles icon for people that have a headset.

If the extension can host the code itself then we can just make it a different page that the browser action filters for, so the code to check for "localhost:3000" in the isOpen filter can be removed too.

[princiya]

@cvan thanks for the nits 👍 will update the code based on your feedback.

this seems like an A-Frame bug. which part didn't work exactly? were there any console errors?

a-camera it was!

[princiya]

@jonathanKingston agree on your feedback about fixing this PR to solve the issue of having a server from web-extension.

Any inputs on how to go about this?

[jonathanKingston]

@princiya I think you should be able to just host the files yourself in the addon like we do for dexie, we might need to compile them (I don't know enough about a-frame to know if that is the case) either way just ignore their example and load the scripts into an extension page like we do for d3 for the main lightbeam page.

[princiya]

@cvan cc @jonathanKingston

this file looks like it’s autogenerated from Webpack? did you write this or is it a vendor, third-party lib? in other words, how could one modify the source and regenerate it?

@cvan Here is the main source.

• I was trying to load forcegraph components by passing nodes and links and it kept complaining about wrong JSON syntax.

  <a-entity forcegraph="nodes:[{id: 1, name: 'first'}, {id: 2, name: 'second'}]; links: [{source: 1, target: 2}]"</a-entity>

• The jsonUrl property for forcegraph component worked while running on a local server, but as you pointed out earlier, when I was trying to load indexVR.html via moz-extension:// it failed.

  <a-entity forcegraph="jsonUrl: data/data.json"></a-entity>

• The forcegraph component uses qwest here which uses XMLHttpRequest and doesn't seem to work without a local server.
• Keeping MozFest in priority, the next task was to quickly look into forcegraph's source code and I decided to use this hack.
• This is off-course temporary, and I will consider submitting a patch to this forcegraph component itself to make it work without requiring a local server for qwest (to fetch the JSON file).
• I attempted my hack directly on the Webpack generated file because I am running short on time (MozFest is this week) and I didn't have to worry about the forcegraph component failing to build etc.
• Considering this is a 1-2 week hack solution, can we leave it this way or are we violating any license rules?

[jonathanKingston]

The forcegraph component uses qwest here which uses XMLHttpRequest and doesn't seem to work without a local server.

Can you pass in a fully qualified URL by using: browser.extesion.getUrl("data.json") or similar? https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/extension/getURL

I was trying to load forcegraph components by passing nodes and links and it kept complaining about wrong JSON syntax.

I assume you used JSON.stringify here? I could believe it would complain about single quotes for example.

[princiya]

@jonathanKingston

Can you pass in a fully qualified URL by using: browser.extesion.getUrl("data.json") or similar?

This is the patch (and a couple of other things) I was talking about submitting to the forcegraph component. https://github.com/vasturiano/aframe-forcegraph-component

Because of time constraints, I decided to put the tiny hack solution into the webpack generated file from the above component.

At this point, the forcegraph component is limited to run only through a local server.

[cvan]

friendly ping: @princiya would you or someone on your team happen to have some spare cycles to resurrect some of the work on this? let me know if you need a hand. I or someone on my team might be able to. thanks - and, again, great work here!

[princiya]

@cvan certainly I would love to see this PR merged. I shall try to clean-up (and recall :)) things this weekend and will ping you. Thank you for writing :)

[jonathanKingston]

@cvan to be clear @princiya is a contributor at the moment and I'm not actively working on Lightbeam at the moment. I can review PRs and help out where possible though. Maybe @princiya should clean the code up and we could pass some of the visualisation work off to your team. I know the 2D could really merit from having someone with real visualisation perf knowledge look at it.

[cvan]

@jonathanKingston: @princiya: completely understand! you have already gone way above and beyond. I do not want to set any expectations (you have already exceeded them :smile:), and I want to make sure it is okay with you both that we build on top of and extend the great work you have done already.

we have some spare cycles and can work on the art assets. @caseyyee in particular can help with any perf analysis.

@princiya: no pressure, really. do not feel obligated to work on this over the weekend. I will check back next week, and let me just know the roadmap, processes, and potential other folks I ought to rope in in conversations if I we want to add the VR visualisations mode properly into the extension. thanks!

[princiya]

@cvan It would be great if you could extend Lightbeam for A-Frame. I am happy to talk about the work done so far, roadmap etc. I have NDA volunteer access to Mozilla's slack and I am in the #lightbeam channel. Also, princiya dot marina at gmail dot com is my email address. Thank you.