[BUG?] Some audios don't work on Android but work on Web

[Booitt]

When I try to send an mp3 audio file, it gets muted on Android but works on the Web. The problem also occurs on other phones, but I'm not sure if it happens on iOS.

One example of what I'm attempting to do is download a WhatsApp video, convert it to mp3, and then respond with the audio:

export const videoToAudio = async (data: Buffer, start: number, end: number) => {
    const filePath = path.resolve(BASE_DIR, uuidv4())

    const input = `${filePath}.mp4`
    const output = `${filePath}.mp3`

    fs.writeFileSync(input, data)
    return await exec(input, output, start, end)
}

const exec = async (input: string, output: string, start: number, end: number): Promise<string> => {
    try {
        return new Promise((resolve, reject) => {
            ffmpeg.ffprobe(input, (_, metaData) => {
              const { duration } = metaData.format

              let clipDuration = duration
              if (end) clipDuration = end - start

              ffmpeg()
              .input(input)
              .inputOptions([`-ss ${start || 0}`])
              // .toFormat("mp3")
              .outputOptions([`-t ${clipDuration}`])
              .output(output)
              .on("end", () => {
                resolve(output)
                unlink([input])
              })
              .on("error", (err) => {
                unlink([input, output])
                reject(err)
              }).run()
            })
        })
    } catch (err) {
        console.log(err)
        throw new CommandError()
    }
}

On the other side:

const audio = await videoToAudio(data, start, end)

// the jid is being passed on the background, but the object is the same as the second argument from the native function
await group.sendMessage({ audio: { url: audio } }, true)

I'm not sure about my memory, because I just migrated the code from another lib. But I think that in my previous tests it was working fine.

Does anyone know what's causing the issue or have some ideas for me to try?

[Booitt]

After some tests, I made it work by converting the audio to .opus instead of .mp3. Does it mean that MP3 files are no longer supported for WhatsApp mobile?

[eduardogoncalvesfilho]

I had this problem, I solved it by sending the mimetype. If you are sending the empty mimetype, send it and see if it resolves.

[Booitt]

I had this problem, I solved it by sending the mimetype. If you are sending the empty mimetype, send it and see if it resolves.

I've tried several audio mimetypes (audio/mp3, audio/mpeg, audio/mp4, audio/ogg; codecs=opus, etc.), but the result is the same: the audio is muted on Android and works on the Web. I've tested this with my own audio files and also with some mp3 files downloaded from the internet, but the problem persists. So far, my only solution has been to convert the files.

[eduardogoncalvesfilho]

You are passing the wrong mimetype for audio mp3, for audio mp3 it is audio/mpeg.

You can consult the available mimetypes here https://suporte.inter.net.br/knowledgebase/42/MIME-Type.html?action=displayarticle&id=42&language=english

[Booitt]

You are passing the wrong mimetype for audio mp3, for audio mp3 it is audio/mpeg.

You can consult the available mimetypes here https://suporte.inter.net.br/knowledgebase/42/MIME-Type.html?action=displayarticle&id=42&language=english

Yeah, I did try that one too. It still doesn't work.

[eduardogoncalvesfilho]

My problem resolved by passing the mimetype of the audio correctly. Perhaps you are doing something wrong. But if you managed to solve it in another way, everything is fine.

[Booitt]

My problem resolved by passing the mimetype of the audio correctly. Perhaps you are doing something wrong. But if you managed to solve it in another way, everything is fine.

Yes, I'm using .opus as my default audio format right now and it's working fine.

I'll leave this issue open for discussion, as it may be happening to others as well. Perhaps just passing the mimetype, as you suggested, works, or maybe it doesn't. This way, we might be able to figure out what is causing the issue.

[lukasportal]

im having different issues when sending an audio (tried mp3 and ogg) without mimetype, the audio on the web works fine, on android it has no sound, and on an iphone i cant even hit play but if i send a mimetype, it doesnt even reach the iphone, only the web

[lukasportal]

Update: weird thing happening.. when I try to play the ogg audio from the number that sent it on an iphone, it shows an error 'This audio is no longer available'

This only happens to audio files that were not recorded within whatsapp

[lukasportal]

@Booitt were you able to work around it? How exactly did you do it? I'm trying to convert an audio to .opus but its still not working.

[delgiudices]

@lukasportal @Booitt experiencing the same issue, converting with

ffmpeg(inStream)
      .format("ogg")
      .audioCodec("libopus")
      .output(outStream)
      .run()

And experiencing same error. I noticed it plays on the WhatsApp business app but no on the regular WhatsApp app

[joseorlando-nsolucoes]

I struggled a lot to solve this problem and it turned out to be something within the file metadata and its channels. I don't understand much about audio and whatnot so anything I got wrong, please let me know.

During my tests I found out that some audios came up with metadata "time_start" negative, such as -0.67856 and that's what caused the audio not to be playable in some devices.

I'm not really sure about the next one but I noticed that when an audio is recorded from an iOS it has only one channel. So to make sure it works accross every single device, I also set it to one channel only.

I now convert every audio that gets sent through my api to ogg using the following params:

 ffmpeg()
        .input(input)
        .audioChannels(1) //make it monochannel like WhatsApp does in iOS
        .audioCodec('opus')
        .toFormat('ogg')
        .addOutputOptions('-avoid_negative_ts make_zero') //this flag prevents the output file to be generated with negative 
        time_start metadata

I hope it helps! Everything is work fine round here after this workaround.

[rodrigopa]

ffmpeg -i input.webm -vn -ab 128k -ar 44100 -f ipod -y output.mp3

send audio with mimetype: audio/mp4

[lukasportal]

@joseorlando-nsolucoes that worked perfectly, thank you! youre a life saver! well done!

[Booitt]

im having different issues when sending an audio (tried mp3 and ogg) without mimetype, the audio on the web works fine, on android it has no sound, and on an iphone i cant even hit play but if i send a mimetype, it doesnt even reach the iphone, only the web

I have found out that the ogg/opus format is not supported on iOS (audios are not even playable). I had to go back to mp3, but some audios work, some don't.

I'll try the approach suggested by @joseorlando-nsolucoes.

[Booitt]

I'll try the approach suggested by @joseorlando-nsolucoes.

Looks like it worked. I converted audios with these configs but in mp3, and it worked on Android and iOS.

I'll keep testing.

[jeankassio]

I'll try the approach suggested by @joseorlando-nsolucoes.

Looks like it worked. I converted audios with these configs but in mp3, and it worked on Android and iOS.

I'll keep testing.

What mimetype did you send with? I send it with mimetype mp3, it only receives the message on Whatsapp Web, sending the message as audio/mp4 doesn't work on IOS. I'm using PHP and sending a request to an api that has WhiskeySockets, so the way I'm doing it is a little different:

$audio = $ffmpeg->openAdvanced([$audioPath]);

$format = new Mp3();

$format->setAudioChannels(1);

$audio->setAdditionalParameters(
    array(
    '-avoid_negative_ts', 'make_zero'
    )
);

$audio
->map(array('0:a'), $format, $ogg)
->save();

[jeankassio]

I'll try the approach suggested by @joseorlando-nsolucoes.

Looks like it worked. I converted audios with these configs but in mp3, and it worked on Android and iOS. I'll keep testing.

What mimetype did you send with? I send it with mimetype mp3, it only receives the message on Whatsapp Web, sending the message as audio/mp4 doesn't work on IOS. I'm using PHP and sending a request to an api that has WhiskeySockets, so the way I'm doing it is a little different:

$audio = $ffmpeg->openAdvanced([$audioPath]);

$format = new Mp3();

$format->setAudioChannels(1);

$audio->setAdditionalParameters(
  array(
  '-avoid_negative_ts', 'make_zero'
  )
);

$audio
->map(array('0:a'), $format, $ogg)
->save();

Well, I ended up doing it on my own (finally). I ended up converting to m4a and uploading it with the mimetype audio/mp4.

A tip for anyone who is also working with PHP-FFMPEG and ends up falling into this issue

class CustomOGG extends DefaultAudio{
    public function __construct(){

        $this->audioCodec = 'aac';

    }

    public function getAvailableAudioCodecs(){
        return ['aac'];
    }
}
...

$ogg = tempnam(sys_get_temp_dir(), 'audio_n') . ".m4a";

$audio = $ffmpeg->openAdvanced([$audioPath]);

$format = new CustomOGG();

$format->setAudioKiloBitrate(128);
$format->setAudioChannels(1);

$audio->setAdditionalParameters(
    array(
    '-vn',
    '-ar', '44100',
    '-f', 'ipod',
    '-avoid_negative_ts', 'make_zero'
    )
);

$audio
->map(array('0:a'), $format, $ogg)
->save();

[jeankassio]

I figured out a better setup to send the audios. Sending with mimetype audio/aac and converting to aac, simple.

Going on https://github.com/WhatsApp/WhatsApp-Nodejs-SDK/blob/58ca3d5fceea604e18393734578d9a7944a37b15/src/types/enums.ts#L123 It is possible to check which audio formats are valid for Whatsapp. Of all that I tested, the best was AAC.

So... in PHP I have:

$ffmpeg = FFMpeg::create(['timeout' => 3600]);

$ogg = tempnam(sys_get_temp_dir(), 'audio_n') . ".aac";

$audio = $ffmpeg->openAdvanced([$audioPath]);

$format = new CustomOGG();

$format->setAudioChannels(1);

$audio->setAdditionalParameters(
    array(
    '-vn',
    '-vbr', '5',
    '-ar', '44100',
    '-f', 'ipod',
    '-avoid_negative_ts', 'make_zero',              
    )
);

$audio
->map(array('0:a'), $format, $ogg)
->save();

...

class CustomOGG extends DefaultAudio{
    public function __construct(){      
       $this->audioCodec = 'aac';       
    }
    public function getAvailableAudioCodecs(){
        return ['aac'];
    }
}

And in Js I Have:

ffmpeg()
.input(input)
.audioChannels(1)
.audioCodec('aac')
.toFormat('aac')
.addOutputOptions('-vn -vbr 5 -ar 44100 -f ipod -avoid_negative_ts make_zero')

[PurpShell]

There is no universal format other than audio/ogg codecs="opus", which is the one whatsapp uses. Using any other format will 100% cause issues

[daveckw]

I struggled a lot to solve this problem and it turned out to be something within the file metadata and its channels. I don't understand much about audio and whatnot so anything I got wrong, please let me know.

During my tests I found out that some audios came up with metadata "time_start" negative, such as -0.67856 and that's what caused the audio not to be playable in some devices.

I'm not really sure about the next one but I noticed that when an audio is recorded from an iOS it has only one channel. So to make sure it works accross every single device, I also set it to one channel only.

I now convert every audio that gets sent through my api to ogg using the following params:

 ffmpeg()
        .input(input)
        .audioChannels(1) //make it monochannel like WhatsApp does in iOS
        .audioCodec('opus')
        .toFormat('ogg')
        .addOutputOptions('-avoid_negative_ts make_zero') //this flag prevents the output file to be generated with negative 
        time_start metadata

I hope it helps! Everything is work fine round here after this workaround.

This works ok for me. Both android and iphone can play the audio now. Thank you