GSoC Infragram.org full-screen UI and video upload Discussion and Planning

[Forchapeatl]

Hi, everyone! This is the Planning Issue for the Infragram.org: full-screen UI and video upload for infragram.org. The project is in collaboration with the Outreachy proposal: Design a new full-screen user interface for Infragram.org . All are open for discussion and improvement.

Process Video Locally

• [x] Allow conversion of entire video.
• [x] Implement video controls on canvas.
• [x] Include another Canvas Element ( second Canvas ).
• [x] Record video on process at second canvas.
• [x] Save processed video file.
• [x] Integrate the above functionalities to with UI of outreachy collaborator. @stephaniequintana

Allow selecting multiple output resolutions as well

• [x] Dynamically change the size of second canvas to fit required resolution.
• [x] Integrate the above functionalities to with UI of outreachy collaborator.

Integrate Teamsys latest camera selection API Issue #39

• [ ] Download Temasys/AdapterJS library.
• [ ] Change all instances of the word webRtcOptions.context to webRtcDetectedType
• [ ] Remove code if (webRtcOptions.context === "flash") Reference 1 Reference 2

Accept multiple webcam resolutions

• [x] Maintain css canvas size.
• [x] Change HTML canvas size to fit requested resolution.
• [x] Integrate the above functionalities to with UI of outreachy collaborator.

Write UI tests for new Infragram UI Jest

• [ ] write test for infragram UI functionalities.

Allow zooming/panning of video

• [x] Zooming : Add uScale of uniform vect2 to shader.frag and shader.vert.
• [ ] Panning : Add uPan of uniform vect2 to shader.frag and shader.vert .
• [ ] Implement above functionalities to infragram drawscreen function.
• [ ] Integrate the above functionalities to with UI of outreachy collaborator.

Miscellaneous

• [ ] Add option to close webcam input
• [ ] Enable image and video stream mirroring
• [x] Add file filters on input to optimize infrared analysis

[TildaDares]

This looks great @Forchapeatl🎉. You're off to a great start!

[TildaDares]

I've created labels for planning and gsoc, so you can add them if you want. I love seeing colors on Github so I like to add labels wherever I can😅.

[jywarren]

Hi @Forchapeatl - this is looking great. I wanted to ask for some clarification on the video portion - are we equally supporting uploaded video (through drag and drop) and live recorded video? Do they use almost all the same functions? If you can just give me a quick summary of this I'd love that!

Also I wanted to say, i know the panning has been challenging. If you want to take a break from it that's fine! The Temasys adapter is a high priority for us, so if you wanted to try taking that one on i think it'd be wonderful -- it'll unlock Safari/iOS functionality, for one -- and should allow for other improvements. What do you think? And/or, am I behind on things at all regarding this? Thank you!

[Forchapeatl]

Hello @jywarren . Thank you . Yes it works on recorded videos and live videos. Please have a look at it here https://forchapeatl.github.io/infragram/index3.html#

Okay . I will commence with Temasys immediately . Thank you.

[RaviAnand111]

Hey @Forchapeatl @jywarren , If there is anything can I help you too ?

[jywarren]

Sounds great thank you Forcha!

@RaviAnand111 so helpful, thanks! Would you like to test out the version Forcha linked w both uploaded and live videos and let us know how it works?

[Forchapeatl]

Friday 29th July 2022. Hi @jywarren. Please here are a few of my worries

1. I have made analysis on this issue https://github.com/publiclab/spectral-workbench.js/pull/172 I tested the Publiclab Spectral Workbench on an Apple phone (IOS 11). (chrome and safari). but all I got was a blank webcam stream. Please am I missing something ?

On Chrome (IOS)

[RaviAnand111]

hey @Forchapeatl @jywarren , I tried to open the link @Forchapeatl gave https://forchapeatl.github.io/infragram/index3.html# , but I am unable to get any video, on uploading I am neither able to see an image nor video. In live video function as well, there is no video on screen just black screen. I am using firefrox and Manjaro Arch Linux OS. Thanks

[Forchapeatl]

hello @RaviAnand111 you are off to a great start. It should work fine on Chrome.

[jywarren]

Oh hmm, @Forchapeatl is this possibly the error you're seeing on Safari? (BTW we may want to move to a different more specifically named issue for this?) https://github.com/publiclab/spectral-workbench.js/issues/266

I believe it had been working in Safari as of https://github.com/publiclab/spectral-workbench.js/pull/172#pullrequestreview-490769230, so I hope we can figure out what happened if it is no longer!

[jywarren]

@RaviAnand111 can you confirm that basic webcam access is working on your system with https://webrtc.github.io/samples/src/content/getusermedia/gum/ ? Thank you!

[Forchapeatl]

@jywarren . Please have a look at this comment https://github.com/publiclab/infragram/issues/39#issuecomment-1200529321 .

[RaviAnand111]

hey @jywarren, I tried to open this link https://webrtc.github.io/samples/src/content/getusermedia/gum/ and it seems to work neither on chrome nor on firefox. I tried to experiment with dev tools but got confused myself.

Hey @Forchapeatl, I tried this link https://forchapeatl.github.io/infragram/index3.html# to run on chrome and it is working fine n chrome, the webcam is visible there, but I guess we would need to work on it for firefox. Thanks

[jywarren]

Hmm, its strange that the webrtc demo/example isn't running -- but if https://forchapeatl.github.io/infragram/index3.html is working then your webcam drivers are probably OK. But the webrtc.github.io implementation should be the canonical/official correct usage, so that's concerning. I wonder if we should hold off for now on testing on Arch Linux, as it's not as common a platform? The original question here was Safari support, as well.

Alternatively @RaviAnand111 if you can find an implementation that is working on chrome and firefox from an official source or a commonly used library, we can try using their syntax. For example, https://p5js.org/examples/dom-video-capture.html is what I had suggested Forcha look at as an example. Does that work for you in multiple browsers @RaviAnand111?

[RaviAnand111]

Hey @jywarren, I tried https://p5js.org/examples/dom-video-capture.html and it is working fine on firefox but not on chrome and chromium browser. I don't know what to conclude from this.

I guess I should try to find some official library which can be used for firefox and chrome on Arch Linux. I think you are right, Arch Linux (Manjaro) is not a common OS and is used by very few so we can hold off its testing and work on mainstream OS, and by the sidelines I can try to find some resources for Arch Linux and inform you guys if I find any. how does that sound ? Thanks

[Forchapeatl]

Hello @RaviAnand111 . No worries. This https://p5js.org/examples/dom-video-capture.html example uses invert filters. It appears the filter intensity styles differently on various browsers. I send gratitude.