feat: app sounds

[lauridskern]

Details

This PR implements sounds playing when message gets received (only when app is in foreground, i. e. not sounds for push notifications).

To play a sound I used react-native-sound ( and react-native-web-sound for web, underhood it uses howler.js). Initially I wanted to use expo-sound but was getting ridiculous errors and couldn't resolve them (on iOS and web).

Fixed Issues

$ https://github.com/Expensify/App/issues/29835 PROPOSAL: N/A

Tests

1. Open app on two devices
2. Send message from one device to another
   • Done sound -> when user sends a message/emoji/attachment/assigning task/sends money/request money (i. e. if user sends any content to another user we play done sound);
   • Success sound -> when someone completes task/someone completes money request/someone sends money to you (regardless of who marks the task as done, i. e. sender or receiver);
   • Attention sound -> someone mentioned you/someone request money from you/someone assigns a task to you (@here also should produce the sound);
   • Received sound -> when message (excluding success/attention cases from above) received
3. go to setting and mute all sounds
4. verify step 2 but be sure there is no audio actually played
5. unmute sounds in app setting
6. go to a particular channel -> open settings of this channel -> open "Notify me about new messages" -> set it to "Mute"
7. verify step 2 but be sure there is no audio actually played (excluding "done" case - "done" sound should be played in any case).
8. set "Daily" mode and repeat step 2 - be sure that all sounds are playable
9. set "Immediately" mode and repeat step 2 - be sure that all sounds are playable
10. mute sound in app settings and for channel - be sure that step 2 will not play any sound at all

• [x] Verify that no errors appear in the JS console

Offline tests

QA Steps

1. Open app on two devices
2. Send message from one device to another
   • Done sound -> when user sends a message/emoji/attachment/assigning task/sends money/request money (i. e. if user sends any content to another user we play done sound);
   • Success sound -> when someone completes task/someone completes money request/someone sends money to you (regardless of who marks the task as done, i. e. sender or receiver);
   • Attention sound -> someone mentioned you/someone request money from you/someone assigns a task to you (@here also should produce the sound);
   • Received sound -> when message (excluding success/attention cases from above) received
3. go to setting and mute all sounds
4. verify step 2 but be sure there is no audio actually played

• [x] Verify that no errors appear in the JS console

PR Author Checklist

• [x] I linked the correct issue in the ### Fixed Issues section above
• [x] I wrote clear testing steps that cover the changes made in this PR
  • [x] I added steps for local testing in the Tests section
  • [x] I added steps for the expected offline behavior in the Offline steps section
  • [x] I added steps for Staging and/or Production testing in the QA steps section
  • [x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
  • [x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
  • [x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).
• [x] I included screenshots or videos for tests on all platforms
• [x] I ran the tests on all platforms & verified they passed on:
  • [x] Android: Native
  • [x] Android: mWeb Chrome
  • [x] iOS: Native
  • [x] iOS: mWeb Safari
  • [x] MacOS: Chrome / Safari
  • [x] MacOS: Desktop
• [x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
• [x] I followed proper code patterns (see Reviewing the code)
  • [x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
  • [x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
  • [x] I verified that comments were added to code that is not self explanatory
  • [x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
  • [x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
    • [x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:
  • [x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
  • [x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
  • [x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
  • [x] I verified the JSDocs style guidelines (in STYLE.md) were followed
• [x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
• [x] I followed the guidelines as stated in the Review Guidelines
• [x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
• [x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
• [x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
• [x] I verified that if a function's arguments changed that all usages have also been updated correctly
• [x] If any new file was added I verified that:
  • [x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory
• [x] If a new CSS style is added I verified that:
  • [x] A similar style doesn't already exist
  • [x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))
• [x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
• [x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
• [x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
• [x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
• [x] If the PR modifies the form input styles:
  • [x] I verified that all the inputs inside a form are aligned with each other.
  • [x] I added Design label so the design team can review the changes.
• [x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
• [x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

https://github.com/Expensify/App/assets/22820318/ee35fde4-056c-458b-8d75-32b30580dd44

Android: mWeb Chrome

iOS: Native

https://github.com/Expensify/App/assets/22820318/e328f648-697c-4c29-a552-eb462d9d988f

iOS: mWeb Safari

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/22820318/a8ebbe88-deab-4134-a107-573b269faf3c

MacOS: Desktop

https://github.com/Expensify/App/assets/22820318/3c3072d0-98dc-4afd-8401-40fb7fd86c97

[github-actions[bot]]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[mountiny]

@lauridskern can you please comment signing the CLA as instructed? i will try to rerun the job

[lauridskern]

I have read the CLA Document and I hereby sign the CLA

[lauridskern]

recheck

[mountiny]

@lauridskern CLA passing now, thanks!

[lauridskern]

Awesome! Thanks for your assistance

[melvin-bot[bot]]

@shawnborton @aimane-chnaif One of you needs to copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

[aimane-chnaif]

Hmm, QR build seems to be failed

@lauridskern please fix conflict

[melvin-bot[bot]]

@shawnborton @thesahindia @ One of you needs to copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

[shawnborton]

@aimane-chnaif can you please review this?

I added puller bear and it also assigned @thesahindia - whoever can prioritize this, please help us get this merged. Thanks!

[aimane-chnaif]

yes, was waiting for conflict to be fixed. 4.7k commits are behind

[shawnborton]

Hi there, can we please try to move this one forward?

[aimane-chnaif]

@lauridskern bump for fixing conflict

[aimane-chnaif]

@shawnborton can you please generate adhoc build?

[shawnborton]

Can do!

[github-actions[bot]]

:test_tube::test_tube: Use the links below to test this adhoc build on Android, iOS, Desktop, and Web. Happy testing! :test_tube::test_tube:	Android :robot:	iOS :apple:
https://ad-hoc-expensify-cash.s3.amazonaws.com/android/31055/index.html	❌ FAILED ❌
	The QR code can't be generated, because the iOS build failed
Desktop :computer:	Web :spider_web:
https://ad-hoc-expensify-cash.s3.amazonaws.com/desktop/31055/NewExpensify.dmg	https://31055.pr-testing.expensify.com

---

:eyes: View the workflow run that generated this build :eyes:

[github-actions[bot]]

:test_tube::test_tube: Use the links below to test this adhoc build on Android, iOS, Desktop, and Web. Happy testing! :test_tube::test_tube:	Android :robot:	iOS :apple:
https://ad-hoc-expensify-cash.s3.amazonaws.com/android/31055/index.html	https://ad-hoc-expensify-cash.s3.amazonaws.com/ios/31055/index.html
Desktop :computer:	Web :spider_web:
https://ad-hoc-expensify-cash.s3.amazonaws.com/desktop/31055/NewExpensify.dmg	https://31055.pr-testing.expensify.com

---

:eyes: View the workflow run that generated this build :eyes:

[shawnborton]

iOS feels super choppy, any idea what's happening there?

https://github.com/Expensify/App/assets/2319350/c947dac8-ec16-4ec5-ad7f-885c50be627e

[shawnborton]

Hmm and then once I am logged in, I get a flash of white as it loads too:

https://github.com/Expensify/App/assets/2319350/4483a3bb-8199-4eab-8c24-dda872215b80

Also @ryanschaffer @quinthar notice how the app loads way faster than the actual length of the mnemonic? So I'm fairly certain this won't be viable as a start up sound unless we purposely force the user to wait until the entire thing plays before loading the app, which I don't think is a great idea.

[hannojg]

Oh yeah, i see whats going wrong there, give me a moment to fix! (the video should have a background on ios, while it shouldn't on android)

[hannojg]

Okay, can we rebuild & retest pls @shawnborton

[shawnborton]

On it

[github-actions[bot]]

:test_tube::test_tube: Use the links below to test this adhoc build on Android, iOS, Desktop, and Web. Happy testing! :test_tube::test_tube:	Android :robot:	iOS :apple:
https://ad-hoc-expensify-cash.s3.amazonaws.com/android/31055/index.html	https://ad-hoc-expensify-cash.s3.amazonaws.com/ios/31055/index.html
Desktop :computer:	Web :spider_web:
https://ad-hoc-expensify-cash.s3.amazonaws.com/desktop/31055/NewExpensify.dmg	https://31055.pr-testing.expensify.com

---

:eyes: View the workflow run that generated this build :eyes:

[shawnborton]

Nice, iOS seems to be working way better. Still noting though that the app starts up so fast that you can't even hear the whole sound. Which again, was part of the testing process, but I think that's a good reason not to use this as a start up sound.

[shawnborton]

Anything we can do to get around the whole "Hmm it's not there" page? Otherwise we can't really test this on web, either.

[shawnborton]

Interesting, for the desktop app, the sound doesn't work. Is that expected?

[shawnborton]

@quinthar @ryanschaffer would love your thoughts on my comment above

[shawnborton]

@Julesssss since you have an Android, mind testing the adhoc build above (most recent one) really quickly and letting me know how it goes? No rush on this though!

[aimane-chnaif]

I tested on android. Animation doesn't load at all for me

https://github.com/Expensify/App/assets/96077027/e39f30a4-2463-470b-abc1-baf03f43d103

[Julesssss]

So I'm fairly certain this won't be viable as a start up sound unless we purposely force the user to wait until the entire thing plays before loading the app, which I don't think is a great idea.

I agree with this. The loud sound is already quite annoying and to to artificially delay the app open would make this way worse I'm afraid 😞

Here's Android, same janky issues:

https://github.com/Expensify/App/assets/10736861/3113afa2-bf8a-4fd7-9a26-f297687a700f

[hannojg]

Here's Android, same janky issues:

Can you explain what exactly feels janky to you? Is it the frame rate of the animation, does the animation content feel too jumpy? I see two issues in your video:

• The animation changes its position. Thats due to the fact that the whole app changes its available size. When you look at the status bar you can see how the app on startup isn't behind the status bar, but then it gets rendered underneath the status bar. Thats a bug that existed before this implementation - do we have an issue ticket for that already?
• On the second app open the video has for a short moment a darker background color breaking the feeling of a unified splash screen. @shawnborton thats why we need the animation probably in other formats such as webp with alpha channel or as lottie animation. (The darker background. is because there is probably a little bit of loss from video encoding for a short time)

I tested on android. Animation doesn't load at all for me

Thats seems like a bug we need to fix 😄 which device is that on?

Anything we can do to get around the whole "Hmm it's not there" page? Otherwise we can't really test this on web, either.

Lets ask on slack if that has been seen before! // Edit: Its a known regression: https://github.com/Expensify/App/pull/32158#issuecomment-1845939477

Interesting, for the desktop app, the sound doesn't work. Is that expected?

We haven't checked if we can force to play the sound on desktop, so right now it behaves the same as a regular browser, so its kinda expected. Will check if we can force the audio playback with electron.

[shawnborton]

thats why we need the animation probably in other formats such as webp with alpha channel or as lottie animation.

Cool, our animators are still working on this one but I'll report back when I hear something.

Thanks for clarification on your other points, too!

[hannojg]

@shawnborton i could push a temporary commit to workaround the web issue? (we need to remove that one once we have a working build)

[shawnborton]

Sure, let's try it!

[Julesssss]

Can you explain what exactly feels janky to you?

Yeah, so it's not noticably unperformant or anything like that, I'm more talking about the general idea of showing the anim on cold app launch. Artifically delaying app open is an annoying delay IMO. Even more so with the audio. I was exicted to see how this would look, but I'm 👎 now.

Thats a bug that existed before this implementation - do we have an issue ticket for that already?

I agree, and we did have an issue but that is less problematic as Android statusbar/config changes occur in many apps on startup. Once we introduce the anim it is more noticeable though.

[hannojg]

@shawnborton Alright, can you kickoff another build? Includes a workaround for web now (Couldn't verify if its working or not, might need to try error with the builds 😅 )

[aimane-chnaif]

I tested on android. Animation doesn't load at all for me

Thats seems like a bug we need to fix 😄 which device is that on?

Samsung Galaxy S10

[shawnborton]

Will do!

[aimane-chnaif]

Along with this weird bug:

[shawnborton]

Whoa yeah that is strange.

[hannojg]

Yeah, probably an issue with encoding the video [in that picture the video just isn't visible] 🤔 It may just take some extra time (for wtvr reason) on that device to load the video.

I could add a workaround for that as well where we wait (with hiding the native splash screen) until the video is loaded (which kinda sucks as this is adding more delay)?

A better workaround would be to add a background color to the video while its loading i think 🤔

[hannojg]

Pushed a fix for the android issue. If we could spin off another build @aimane-chnaif could test and see if it helps 🙌

[Julesssss]

Pushed a fix for the android issue. If we could spin off another build @aimane-chnaif could test and see if it helps 🙌

In progress

[Julesssss]

One other concern I have is that we only show the animation/audio on cold app launch. So sometimes the audio plays, sometimes it doesn't. Are we intentionally doing this? Because while the reason is obvious to us, users are not going to understand why this is happening.

[hannojg]

So sometimes the audio plays, sometimes it doesn't

Not sure if we talk about the same. Right now on each app start a random video from four is picked. Two of them are with sounds, the other two are not. I think the reason for that was that we wanted to experiment with how sound feels (so sound playing for you on cold starts was coincidence 😄 )

https://31055.pr-testing.expensify.com/

Workaround didn't work, will check later if i can find another solution!

Sorry, it does, had to clear cache!

However, i think right now we didn't implement the splash for the browser @lauridskern ?

[hannojg]

Okay, for web its its actually implemented (in the index.html), however my browser loads it so fast that i never get to see it 😄

[Julesssss]

Not sure if we talk about the same. Right now on each app start a random video from four is picked. Two of them are with sounds, the other two are not.

Nah I mean that the anim/sound is only shown on a cold boot (app has been killed by OS or user). If the app is not killed we'll launch straight to the LHN or last used page. But once the OS or user kills the app, we'll show the anim/audio again. So to users this is inconsistent.

[shawnborton]

Two of them are with sounds, the other two are not.

Hmm this isn't correct. All four should have sounds. It's just that two of them have a "vocal" layer to the sounds that the others don't.

[shawnborton]

So to users this is inconsistent.

@Julesssss this feels like expected behavior to me. If I already had an app running in the background and I switch to it, I don't expect to see any kind of boot/loading animation.

[aimane-chnaif]

can we get new build generated? (after adding android workaround)