Receiving and processing audio type messages

[atmtrujillo]

Hi,

Thank you so much for all your work in facilitating the use of the WhatsApp Business Cloud APIs.

I've been trying to create a WhatsApp bot to transcribe voice messages, and I'm struggling to download the audio.

I've downloaded and been able to reproduce voice messages through Postman, but I can't make it work in C#. I looked at your project and couldn't find the implementation to receive and process an audio message.

Is this something you are planning to implement, or did I miss it?

Thanks again

Regards, Alex

[gabrieldwight]

Hi Alex, I have implemented sending audio through using media id and a URL link. For the webhook part to receive audio messages I have not yet implemented a concrete class for audio messages.

The JSON below shows how the image message is received through the webhook, I guess for audio type will have its corresponding properties. To process the audio message if the audio properties contain a value of either the media id or link then you can process the value to get the download link from WhatsApp which you can use to download the audio message received.

Downloading media documentation

{
  "object": "whatsapp_business_account",
  "entry": [{
      "id": "WHATSAPP_BUSINESS_ACCOUNT_ID",
      "changes": [{
          "value": {
              "messaging_product": "whatsapp",
              "metadata": {
                  "display_phone_number": PHONE_NUMBER,
                  "phone_number_id": PHONE_NUMBER_ID
              },
              "contacts": [{
                  "profile": {
                    "name": "NAME"
                  },
                  "wa_id": "WHATSAPP_ID"
                }],
              "messages": [{
                  "from": PHONE_NUMBER,
                  "id": "wamid.ID",
                  "timestamp": TIMESTAMP,
                  "type": "image", // I guess this key changes the message type property to audio
                  "image": {
                    "caption": "CAPTION",
                    "mime_type": "image/jpeg",
                    "sha256": "IMAGE_HASH",
                    "id": "ID"
                  }
                }]
          },
          "field": "messages"
        }]
    }]
}

[atmtrujillo]

Correct! It's very similar to the image use case.

1. I get the first message, as mentioned by you above.
2. I retrieve the media URL.
3. I try to download the actual audio with the same auth token.

But, I can't reproduce the audio locally. I think it's probably something like the audio encoding or even something more fundamental that I might be missing.

It's an ogg audio file (codec opus)

I'll let you know if I manage to solve it.

Regards, Alex

[gabrieldwight]

You can try with a different audio codec to see if it works.

(only opus codecs, base audio/ogg is not supported) I think the base ogg audio file is not supported by cloud api.

Supported audio codecs

[atmtrujillo]

The odd bit is when I do it through Postman. It works well, and it comes back with some headers. One of the headers is Content-Type which says audio/ogg.

I would have thought the binary would have everything it needs, including the encoding and any other properties.

[gabrieldwight]

I will test with a sample audio ogg file to see if I'm able to download it from the webhook then I will let you know.

[atmtrujillo]

Thank you! I'll keep going as well.

[atmtrujillo]

Annoyingly, I've just got it working in PowerShell

PowerShell. This works well

$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", "Bearer <YOUR BEARER TOKEN>")

$responseURL = Invoke-RestMethod 'https://graph.facebook.com/v17.0/<YOUR WHATSAPP MEDIA ID>' -Method 'GET' -Headers $headers 
Write-Output $responseURL.url

Invoke-RestMethod $responseURL.url -Method 'GET' -Headers $headers -OutFile test.ogg

This is what I'm trying in my PoC C# quick dirty code. Not working

var audioId = "<YOUR WHATSAPP MEDIA ID>";
var apiVersion = "v17.0";
var token = @"<YOUR BEARER TOKEN>";
var fileBasePath = @"<YOUR LOCAL PATH>";

var filePath = Path.Join(fileBasePath, $"{audioId}.ogg");
File.Delete(filePath);

string? url;

using (var client1 = new HttpClient())
{
    client1.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);

    var audioRequest = await client1.GetStringAsync($"https://graph.facebook.com/{apiVersion}/{audioId}");
    var jsonDocument = JsonDocument.Parse(audioRequest);
    url = jsonDocument.RootElement.GetProperty("url").GetString();
}

using (var client2 = new HttpClient())
{
    client2.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
    var response = await client2.GetByteArrayAsync(url);
    File.WriteAllBytes(filePath, response);
}

[atmtrujillo]

I found the problem. I was missing the "User Agent" from the headers.

You just need to added this to make it work

var productValue = new ProductInfoHeaderValue("<YOUR APP NAME>", "1.0");
client2.DefaultRequestHeaders.UserAgent.Add(productValue);

[gabrieldwight]

I have also added the implementation for downloading media using the media url and testing it out to see if it works.

[gabrieldwight]

It works perfectly with the user agent. I have added the implementation to download media from the media url generated by whatsapp

if (messageType.Equals("audio"))
                    {
                        var audioMessageReceived = JsonConvert.DeserializeObject<AudioMessageReceived>(Convert.ToString(messageReceived)) as AudioMessageReceived;
                        audioMessage = new List<AudioMessage>(audioMessageReceived.Entry.SelectMany(x => x.Changes).SelectMany(x => x.Value.Messages));
                        _logger.LogInformation(JsonConvert.SerializeObject(audioMessage, Formatting.Indented));

                        var mediaUrlResponse = await _whatsAppBusinessClient.GetMediaUrlAsync(audioMessage.SingleOrDefault().Audio.Id);

                        _logger.LogInformation(mediaUrlResponse.Url);

                        // To download media received sent by user
                        var mediaFileDownloaded = await _whatsAppBusinessClient.DownloadMediaAsync(mediaUrlResponse.Url);

                        var rootPath = Path.Combine(_webHostEnvironment.WebRootPath, "Application_Files\\MediaDownloads\\");

                        if (!Directory.Exists(rootPath))
                        {
                            Directory.CreateDirectory(rootPath);
                        }

                        // Get the path of filename
                        string filename = string.Empty;

                        if (mediaUrlResponse.MimeType.Contains("audio/mpeg"))
                        {
                            filename = $"{mediaUrlResponse.Id}.mp3";
                        }

                        if (mediaUrlResponse.MimeType.Contains("audio/ogg"))
                        {
                            filename = $"{mediaUrlResponse.Id}.ogg";
                        }

                        var filePath = Path.Combine(_webHostEnvironment.WebRootPath, "Application_Files\\MediaDownloads\\", filename);

                        await System.IO.File.WriteAllBytesAsync(filePath, mediaFileDownloaded);

                        return Ok(new
                        {
                            Message = "Audio Message received"
                        });
                    }