[Feature]-[508]:Camera widget bugs

osloma commented 2 years ago

Is there an existing issue for this?

[X] I have searched the existing issues

Summary

We can see that when Appsmith is run on an Android, pictures are not having all the quality than the camera on the phone can capture. Also, using iPhone, pictures taken are only a completely black image where nothing can be seen

Why should this be worked on?

The use case for this is to capture pictures of documentation (like bills, receipts, etc.) that should be in a resolution that can be read easily. In our case, we are storing the pictures in Cloudinary but pictures have reduced size (around 350x350 pixels) that make them very hard to read their content (letters cannot be read easily). We have tried to change the resolution of the camera on the phones with no result

So far we are finding:

On Android, it works but we get low resolution pictures
On iPhone, pictures are completely black

The goal will be to be able to select the resolution required on the picture before it's taken. It could be done either from the widget (more preferred way) or from the running camera on the application.

Thank you so much!

Nikhil-Nandagopal commented 2 years ago

@osloma thank you for the information. We'll debug this and let you know what is going wrong

dilippitchika commented 2 years ago

@vivekverma2312 can you ask someone to test on an android phone?

It is broken in iphone for sure.

dilippitchika commented 2 years ago

Stats

Stat	Values
Reach	127
Effort (months)	0.5

dilippitchika commented 2 years ago

Found a set of issues Apple devices -

[Blocker] Iphone safari and chrome unable to capture image in camera in iOS 14.4
[Blocker] In Ipad the camera goes black after capturing the image

Android devices -

[Blocker] The back camera feed seems to be mirrored.
[Blocker] When I tap the full screen, the tooltip stays open and I am unable to see any other button
[Blocker] When I turn on/off the video, the icon jumps a few pixel to the top right and back.
[Blocker] In full screen mode I’m not able to switch the camera (the dropdown doesn’t open)
[Blocker] Weird noise after taking a picture
[Blocker] Low resolution of the image leading to hazy images for app developers

Enhancements

[Enhancement] Switching between back and front camera is a bit inconvenient. My general reflex is flip with one press but here I have to click a dropdown and then select the camera
[Enhancement] When I take a photo, a check and a dismiss icon appears, which was fine but the reset of the actions were still visible like the turn on/off video, switch camera which IMO is not relevant when I try to save/discard the photo.
[Enhancement] I feel like the controls should not be a part of the camera view port. As the controls are white in color, if I point at a light background; it’s really hard to locate the controls. This seems to be fine in the full screen mode.
[Enhancement] All the buttons are very closed together, it always felt like I’m gonna press something else
[Enhancement] The button to take a picture should be a bit bigger so that the press area is more prominent

Slack discussion thread - https://theappsmith.slack.com/archives/C02JV8G1MP0/p1653567793725609 Library issue - https://github.com/mozmorris/react-webcam/issues/57

dilippitchika commented 2 years ago

@techbhavin please start triaging all the blockers mentioned first and leave a comment on if all these blockers can be fixed or not. If you need any help with devices, do reach out to me or @vivekverma2312

Here's the sample app - https://app.appsmith.com/app/untitled-application-5/page1-628f70677901344ba8d2e7cb

techbhavin commented 2 years ago

@dilippitchika updated noted of blocker issue

Apple Devices:

Not Reproduce
Not Fixed - camera library issue

Android Device :

Working as expected
Fixed
Fixed
Not Possible - In a full-screen mode popover is not working open issue in Popover Elements (Tooltip, Popover, Overlay) do not open with Portals #3248
Not able to reproduce. (Look like it is the device-specific issue)
Fixed

dilippitchika commented 2 years ago

Regarding

Android Device : can we change the tooltip position to the right? or do we have any approach? Why is the tooltip staying open, is there a reason for it? Tooltip should only open when we have clicked on it right?

Low resolution of the image leading to hazy images for app developers -- Please talk to ashit once, he can show this to you

dilippitchika commented 2 years ago

Scope:

Fix all the items which are marked as "we can fix" by Bhavin
Check the rest of the blockers with QA team's help if they are reoccurring

@techbhavin @vivekverma2312

techbhavin commented 2 years ago

Why is the tooltip staying open, is there a reason for it? The tooltip should only open when we have clicked on it right?

here is the library issue tooltip working fine with a click but there is an issue while touching. @dilippitchika

chandannkumar commented 2 years ago

@techbhavin Couple of observations here while Testing camera widget on Android Device:

When we switch the camera and click to Take Photo/Video, camera widgets breaks until browser refresh. https://images.zenhubusercontent.com/275993885/3d34f410-2975-4f23-8cb7-4b584a1e5443/screenrecorder_2022_06_23_13_37_45_389_0_.mp4
Tooltip blocks the camera/Recorder click button when we exit fullscreen mode
Camera names in the dropdown are weird, names must be simple

dilippitchika commented 2 years ago

@techbhavin what is the suggestion regarding tooltip and fullscreen library? I thought there was a discussion around it.

techbhavin commented 2 years ago

@dilippitchika

about the tooltip and fullscreen

For the tooltip,it is standard behavior with a Mobile Device. here tooltip is visible with long touch and when we touch again on anywhere on the screen it will hide again. At the moment we don’t have any alternative solutions for this.
In full-screen mode, we can’t switch the camera because the popover is not visible. so instead of adding a popover to change the camera can we add a standard round icon for the swap camera? like below-attached image

dilippitchika commented 2 years ago

I'll answer 2 first, if we can detect which is front camera and which is back then yes we can add this icon. The problem I currently think is we don't have a way to do that, please correct me if I am wrong @techbhavin.

You may think it may not matter but the problem is if you have 3 cameras on your laptop. Maybe we can still use an icon here but it should be a different camera icon which changes the camera. This way we can get rid of tooltip completely.

Regarding fullscreen, can we just remove that functionality for now?

techbhavin commented 2 years ago

Agree with your point @dilippitchika,

Yes, we can remove fullscreen but can we do a similar like webcamera to show list of available camera

and Currently, we have a tooltip on capture and a fullscreen icon so can we remove this tooltip?

dilippitchika commented 2 years ago

but can we do a similar like webcamera to show list of available camera

Are you referring to how it shows the settings? If so, then we can proceed from a product standpoint, but i thought there is a tech limitation.

and Currently, we have a tooltip on capture and a fullscreen icon so can we remove this tooltip

Yes we can remove this for now

dilippitchika commented 2 years ago

@vivekverma2312 @chandannkumar let's discuss this issue once on a call, i believe this needs a small discussion to align and decide if we need more issues created from this issue.

vivekverma2312 commented 2 years ago

@techbhavin On ios it's almost impossible to use camera widget

Controls are not visible on the widget (In background actually it exists)
Image click makes the widget black
Changing from camera to video also makes the widget black
Tapping on Fullscreen does nothing

ios:15.5

https://user-images.githubusercontent.com/83569920/176106817-9b9453d5-7fdb-4635-b62f-b7a716d785bb.MP4

chandannkumar commented 2 years ago

@techbhavin A Observation related to picture quality cc: @dilippitchika

Adding two photo which shows the Pixel and details of image taken from Phone's camera and Appsmith camera widget
1. DP Image (camera widget) : *Picture downloaded via image widget

Phone camera(original):

chandannkumar commented 2 years ago

Moving this issue to in progress due to blockers and callouts. There is also a merge conflict in this PR

techbhavin commented 2 years ago

As per our discussion,

I have to put the camera rotate icon instead of the camera menu (list of the camera). but there is one more functionality that is camera turns off. so when we will put the camera rotate icon then how to manage camera turn-off functionality.
Now one more dought is what should we do for the audio menu that is shown in video mode of the camera widget and also the same issue will happen with audio mute functionality like as camera turn-off.

@dilippitchika

techbhavin commented 2 years ago

Fixed the camera unable to capture images on ios devices.
Fixed the camera goes black after capturing the image in ios devices.
Fixed the changing from camera to video and also makes the widget black in ios devices.
Removed the tooltip so it can not cause tooltip-related issues.
Give the mirror functionality in the camera widget.
Fixed the getting weird noise after taking the picture.
Improved the picture quality of the camera.
Fixed the camera widget break when we switch the camera or take pictures after switching the camera.

@chandannkumar @vivekverma2312

dilippitchika commented 2 years ago

@techbhavin we can add another icon if needed, which turns on/off camera and one which changes cameras.

techbhavin commented 2 years ago

Fixed the show video/audio input menus in full-screen mode.
Fixed to show the control panel in the full-screen mode of the safari browser.

@chandannkumar @vivekverma2312

chandannkumar commented 2 years ago

Observation: While recording a video from from front camera as soon as i switch camera to rear, the video stops recording and saves automatically but the recording timer is still continues on camera widget.

https://images.zenhubusercontent.com/275993885/276d845f-802b-4325-b048-1cd48faa2122/screenrecorder_2022_07_26_16_33_41_679_0_.mp4

@techbhavin Not sure about the video recording stops automatically on switching camera is a bug or an expected behavior

techbhavin commented 2 years ago

Fix the camera switching issue when recording video

@chandannkumar

vivekverma2312 commented 2 years ago

@techbhavin On ios when my mic is disabled and I record a video the sound is getting recorded.

techbhavin commented 2 years ago

For now, we are removing the video/audio mute buttons for ios devices because of an issue with the Safari browser. whenever we get the solution for that at that time we fix that.

@chandannkumar @vivekverma2312

cc: @dilippitchika @somangshu

appsmithorg / appsmith