[$16000] Feature Request: App should play the uploaded videos inline - reported by @murataka

[mvtglobally]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Action Performed:

1. Open any chat
2. Send a video file
3. Try to play video in chat

Expected Result:

App should play the uploaded videos inline

Actual Result:

User has to download video to play

Workaround:

unknown

Platform:

Where is this issue occurring?

• Web
• iOS
• Android
• Desktop App
• Mobile Web

Version Number: 1.1.39-1 Reproducible in staging?: Y Reproducible in production?: Y Logs: https://stackoverflow.com/c/expensify/questions/4856 Notes/Photos/Videos: Any additional supporting documentation

Expensify/Expensify Issue URL: Issue reported by: @murataka Slack conversation: https://expensify.slack.com/archives/C01GTK53T8Q/p1644125263019529

View all open jobs on GitHub

[MelvinBot]

Triggered auto assignment to @mallenexpensify (AutoAssignerTriage), see https://stackoverflow.com/c/expensify/questions/4749 for more details.

[MelvinBot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @parasharrajat (Exported)

[MelvinBot]

Triggered auto assignment to @marcochavezf (Exported), see https://stackoverflow.com/c/expensify/questions/7972 for more details.

[MelvinBot]

Triggered auto assignment to @Gonals (Engineering), see https://stackoverflow.com/c/expensify/questions/4319 for more details.

[mallenexpensify]

Sorry @parasharrajat and @marcochavezf , I got ahead of myself. I thought I was assigned as Triage and not CM. Moving along to Engineering / @Gonals now.

[Gonals]

This would be pretty cool! I'm setting it as monthly, since it isn't a priority

[MelvinBot]

Current assignee @mallenexpensify is eligible for the External assigner, not assigning anyone new.

[MelvinBot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @rushatgabhane (Exported)

[MelvinBot]

Triggered auto assignment to @stitesExpensify (Exported), see https://stackoverflow.com/c/expensify/questions/7972 for more details.

[mallenexpensify]

Job posted https://www.upwork.com/jobs/~0103b194cebb2bdef4

[mim1192]

you mean sequence of video is not correct .

[rushatgabhane]

@mim1192 umm no, we just need video playback.

[koushal141]

Proposal from Upwork

Hi there! I can implement an inline video player in chats, but since this is react native, am not sure if I can implement it in web or desktop.

But I can implement it for Android and iOS devices. On other devices it'll fallback to the default behaviour of download option.

[rushatgabhane]

@koushal141 we need it to work on all platforms. But sure, post a detailed proposal. Maybe we can build up on it.

[koushal141]

@koushal141 we need it to work on all platforms. But sure, post a detailed proposal. Maybe we can build up on it.

Ofcourse! I understand, am not saying it can't be done. Am just saying since it's cross platform, there might be a possibility it might not work on desktop. But I'll try my 100% to get it working on all platforms.

Should I send the proposal here or on Upwork?

[rushatgabhane]

Should I send the proposal here or on Upwork?

Here is fine. Also, I'd suggest you to go through Contributing.md before proceeding, thanks!

[rushatgabhane]

@mallenexpensify this is a monthly issue yeah? but Melvin replaced it with weekly xD

[mallenexpensify]

Yeah, the default is, and should, be weekly. tl:dr, if it's worth fixing and a contributor can work on it, let's get it done sooner than later. We need jobs for contributors to work on too :)

[rushatgabhane]

@mallenexpensify we don't have any proposals since 13 days, please consider doubling this issue.

[mallenexpensify]

Price doubled to $500 https://www.upwork.com/jobs/~0103b194cebb2bdef4

[mallenexpensify]

Doubled price to $1000 https://www.upwork.com/jobs/~0103b194cebb2bdef4

[mallenexpensify]

Doubled price to $2000 https://www.upwork.com/jobs/~0103b194cebb2bdef4

[rushatgabhane]

@mallenexpensify should we double this ticket?

[mallenexpensify]

Doubled price to $4000 Thanks @rushatgabhane

[rushatgabhane]

To anyone interested in posting proposals, feel free to use external libs to solve this issue. A few approaches

• Stream the video.
• Download the video first, and then play it back.
• A combination of both??

[bhavyagaur99]

Proposal:

Native

I can use this library for native platforms react-native-video library. This library includes support for iOS, Android, tvOS, Windows, etc.

This library supports more platforms than we require so we might have to remove them.

Web

I can use this library for web browsers video-react library. This library supports Chrome and Firefox.

Is this approach of using two libraries acceptable?

[eVoloshchak]

Proposal:

• For web, we could use react-player. It's lighter, more popular and has native controls
• For native we can use react-native-video, but... Out of the box it has some weird behaviour with it's controls

https://user-images.githubusercontent.com/9059945/163047421-b263337c-cc07-441b-9b6c-f37ce411b6ad.mp4

So the solution is to use react-native-video-controls, which was developed to be used with react-native-video. We would still probably need to fork it and make some changes to the controls, for instance remove the back button from the top left corner and a volume slider.

Implementation:

1. Add isVideo method to Str in expensify-common that looks like this:

   isVideo(url) {
     return _.contains(['mov', 'mp4', '...other video extensions'], this.getExtension(url));
   }

2. In AttachmentView's render method add the following:

   if (Str.isVideo(props.sourceURL) || (props.file && Str.isVideo(props.file.name))) {
     return (
         <VideoView 
             url={props.sourceURL} 
             file={props.file}
         />
     );
   }

3. Inside the Components directory, create a VideoView folder with two files:
4. index.js containing inline video player for the web

   const VideoView = (props) => {
   return (
       <View style={[styles.defaultAttachmentView, { flexDirection: 'column', position: 'relative' }]} >
           <ReactPlayer 
               url={props.url} 
               controls 
               width='100%'
               height='100%'
           />
           <Text style={[styles.textStrong, styles.flexShrink1, styles.breakAll, styles.flexWrap, styles.mw100]}>{props.file && props.file.name}</Text>
       </View>
   )
   }

5. And index.native.js containing inline video player for the native platforms:

   const VideoView = (props) => {
   return (
       <View style={[styles.defaultAttachmentView, { flexDirection: 'column',  }]} >
           <View style={{ width: 200, height: 200 }}>
           <VideoPlayer
               source={{uri: props.url}} 
               paused
               resizeMode={'contain'}
               style={{
                   position: 'absolute',
                   top: 0,
                   left: 0,
                   bottom: 0,
                   right: 0,
               }}
           />
           </View>
   
           <Text style={[styles.textStrong, styles.flexShrink1, styles.breakAll, styles.flexWrap, styles.mw100]}>{props.file && props.file.name}</Text>
       </View>
   )
   }

The result looks like this: Web

https://user-images.githubusercontent.com/9059945/163050002-82b0bada-ce7a-4dae-882c-d7cf9f803a13.mp4

mWeb

https://user-images.githubusercontent.com/9059945/163052752-cc9992a2-fbe0-4dd6-a92e-cbd34e2374bd.mp4

Native

https://user-images.githubusercontent.com/9059945/163052582-c0741981-21bd-4c19-9a9e-8ed3ca95863a.mp4

Some considerations:

1. Since this is more of a prototype, i hardcoded video dimensions for native and for web it's scaling to take up all of the parent's width. What should video dimensions be? Same as for images?
2. I've added file name below the player, but I'm guessing it should include a download icon somewhere too?
3. Do we need the ability to play videos fullscreen? If so, web supports it out of the box, and for native a separate component would need to be implemented. react-native-video-controls has a toggleResizeModeOnFullscreen prop and a onEnterFullscreen event exactly for this.
4. Should the inline player be visible when user is uploading the video (before pressing "Send")?

[gladiator-1]

Proposal

web : you can use createElement from react-native-web to create HTML video element

result

https://user-images.githubusercontent.com/102553449/163058277-6ffcd4b0-03d1-4256-ab89-0cf8c9180332.mp4

native : you can render video element inside a webview using react-native-webview

result

https://user-images.githubusercontent.com/102553449/163058582-12fda4bb-cf49-4606-b926-9348835fbdc2.mp4

or using react-native-video with react-native-video-player

result

https://user-images.githubusercontent.com/102553449/163058886-6361bae4-0950-4609-829f-95ab32ba6a9f.mp4

[mallenexpensify]

@rushatgabhane can you please review the proposals above, thx

[rushatgabhane]

Oh sweet! Three proposals at the same time - all of which are interesting. I'm leaning towards the third proposal because it doesn't use any extra libs.

Note: This issue will require a lot of discussion and planning. Let's just stick to the high level stuff for now, and discuss all the options we have.

• Can we get this done without using any extra library?
• If no, which library to use? and why?
• Weigh the pros and cons of all approaches.

[rushatgabhane]

RNW: react native web, RNWV: react native webview.

@gladiator-1

1. RNW's createElement is an unstable API.

   • What will be the cons of using an unstable API?
   • We have a forked version of RNW. So can't we always keep supporting RNW.createElement() in our forked version?
   • Can we find an alternative to it? How about React.createElement()? see reference

2. There's an open issue related to video for RNWV - rnwv#1632. Please make sure that we aren't facing it.

[gladiator-1]

@rushatgabhane Thank you for showing your interest in my proposal.

Can we get this done without using any extra library?

Yes.

RNW's createElement is an unstable API. Can we find an alternative to it? How about React.createElement()?

React.createElement do the same as RNW createElement, so we can use it.

There's an open issue related to video for RNWV - https://github.com/react-native-webview/react-native-webview/issues/1632. Please make sure that we aren't facing it.

We use inline HTML and it works fine.

Weigh the pros and cons of all approaches.

web: HTML <video> element is the simplest way to play a video file, it can play the uploaded videos inline. if you need more options or play other types of videos like youtube or vimeo you should use extra lib.

native: I don't see huge differences between using react-native-video or RNWV, if you use RNWV you have a good chance to use third party scripts inside the web view which give you more options.

Finally, whatever you use, this video won't work on safari browser or IOS I don't know why? maybe this, anyway you need to fix it first.

[rushatgabhane]

Finally, whatever you use, this video won't work on safari browser or IOS I don't know why? maybe this, anyway you need to fix it first.

Hmm, what URI are you using? Maybe try using the actual URI we'll have to use.

Just wanna confirm, both react-native-video and RNWV, don't work on iOS and Safari??

[murataka]

@rushatgabhane , the fix for ios and safari is possibly not related to this issue directly. Check below code :

<video autoplay loop muted playsinline poster="video_thumbnail/thumbanil.jpg" src="video/video.mp4">
        <source src="video/video.mp4" type="video/mp4"></source>
        <source src="video/video.webm" type="video/webm"></source>
        <source src="video/video.mov" type="video/mov"></source>
</video>

we may need to re-encode the video source to a different video format at server side , or just a small fix like adding

playsinline

might solve that bug . For anyone testing the ios and safari can test the above if it helps.

Apple's video player components are generally non standard, for example sometimes the video will play on a floating window even if you don't want to. This is related to very old apple mobile devices being slow ,where they applied a workaround by forcing all videos to play full screen even if the app should play it inline, or not supporting some video encoding formats for interesting reasons .

[rushatgabhane]

@murataka thanks so much for you help!! https://developer.apple.com/documentation/webkit/delivering_video_content_for_safari

[gladiator-1]

I use URI look like this https://www.expensify.com/chat-attachments/32434134208/w_236f234565322f30d479895a9e62c90b1.mp4?encryptedAuthToken=%2B%2FIIeB7vpitzZYdKopAuan1lzP3cbGIRrlNnsfzhnf8nK68vT30pPOtm5....

both react-native-video and RNWV, don't work on iOS and Safari??

Yes, both not working using this URI.

[rushatgabhane]

@gladiator-1 Cool cool, let me know if playsinline does the trick. Or anything that gets iOS and Safari working. We can then dive further into the details.

[rushatgabhane]

@gladiator-1 how's it looking?

[rushatgabhane]

This issue is open for additional proposals btw.

[gladiator-1]

@rushatgabhane the problem still exists.

for testing:

1. Send any mp4 file.
2. right click on it and Copy to clipboard.
3. Paste it somewhere and get the URI from it.
4. Open this URI using Safari the video will not play.
5. Open it using Firefox or Chrome the video will play fine.

[rushatgabhane]

Hmm, @stitesExpensify what do we do here? Wait for proposals that solve the issue for iOS and Safari right?

[rushatgabhane]

Unassigning because I'll be OOO until 14 May.

There are no untriaged proposals. cc: @mallenexpensify

[stitesExpensify]

Unless @gladiator-1 has any other ideas, I think we just want for more proposals at this point

[mallenexpensify]

Doubled price to $8000 https://www.upwork.com/jobs/~0103b194cebb2bdef4

@mananjadhav would you like to C+ this issue with Rushat out?

[mananjadhav]

@mananjadhav would you like to C+ this issue with Rushat out?

Yeah sure.

[melvin-bot[bot]]

📣 @mananjadhav You have been assigned to this job by @mallenexpensify! Please apply to this job in Upwork and leave a comment on the Github issue letting us know when we can expect a PR to be ready for review 🧑‍💻 Keep in mind: Code of Conduct | Contributing 📖

[melvin-bot[bot]]

A Contributor Manager will be assigned to issue payment via Upwork if we deploy an associated Pull Request to production. Per Contributing.md.

[mallenexpensify]

Assigned @mananjadhav as C+ then re-added the Help Wanted label

[mananjadhav]

Went through the history, waiting for proposals.

[murataka]

For anyone facing the safari issue, you can check using the video source uri as :

this.props.href.replace('https://www.expensify.com/', '/')

( for web only) works .

I tested safari with the above uri, and it seems to wrok on all browsers.