Flutter: text-to-audio api is not working: Error: The status code of 500 has the following meaning: "Server error - the server failed to fulfil an apparently valid request"

abdullah432 commented 3 months ago

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

Note

I file this bug yesterday and it was marked completed but it's fixed for me and please ask me if it's fixed or not before marking this completed. Previous issue I raised: https://github.com/langgenius/dify/issues/6339

Our app is completed but can't move a head due to this bug.

Dify version

0.6.14

Cloud or Self Hosted

Cloud

Steps to reproduce

I implemented dify API for chat, vision, audio-to-text. It's all work good.

When I try to do text-to-audio then it's throwing following Exception.

This exception was thrown because the response has a status code of 500 and RequestOptions.validateStatus was configured to throw for this status code.
The status code of 500 has the following meaning: "Server error - the server failed to fulfil an apparently valid request"
Read more about status codes at https://developer.mozilla.org/en-US/docs/Web/HTTP/Status
In order to resolve this exception you typically have either to verify and fix your request code or you have to fix the server code.
[ERROR:flutter/shell/platform/darwin/ios/framework/Source/profiler_metrics_ios.mm(203)] Error retrieving thread information: (os/kern) invalid argument

I'm working on this from second day, so tried 100's of different ways, will share two below, which I think should work good.

First: Pass data with FormData as documentation mentioned it.

  Future<void> textToAudio({
    required String text,
    required String user,
    required bool streaming,
  }) async {
    const String url = 'https://api.dify.ai/v1/text-to-audio';
    var secretKey = "app-RaxAShID6WHEcy8PUo9PRuKY";

    try {
      var _dio = Dio();
      final response = await _dio.post(
        url,
        options: Options(
          headers: {
            'Authorization': 'Bearer $secretKey',
            'Content-Type': 'multipart/form-data',
          },
        ),
        data: FormData.fromMap({
          'text': text,
          'user': user,
          'streaming': streaming,
        }),
      );
      print(response.data);
    } on DioException catch (e) {
      print('Request failed with status: ${e.response?.statusCode}');
      print('Error: ${e.message}');
    }
  }

Second: with simple map form, in case server expect it that way. Also test with "Content-Type": "audio/wav" but it didn't help.

  Future<void> textToAudio({
    required String text,
    required String user,
    required bool streaming,
    required String secretKey,
  }) async {
    const String url = 'https://api.dify.ai/v1/text-to-audio';

    try {
      final response = await _dio.post(
        url,
        options: Options(
          headers: {
            'Authorization': 'Bearer $secretKey',
            'Content-Type': 'application/json',
          },
        ),
        data: {
          'text': text,
          'user': user,
          'streaming': streaming,
        },
      );
      print(response.data);
    } on DioException catch (e) {
      print('Request failed with status: ${e.response?.statusCode}');
      print('Error: ${e.message}');
      if (e.response != null) {
        print('Response data: ${e.response?.data}');
      }
    }
  }

A look into my dify console.

Can you help me what could be the problem. At dify console text to voice is working good.

✔️ Expected Behavior

Text should get converted to audio.

❌ Actual Behavior

I'm getting Exception while calling the API.

crazywoola commented 3 months ago

We fixed this in main branch, and it has not been released to cloud yet. We mark it closed because we have a closed PR. Sorry for that. I've checked that our cloud version is v0.6.14, so it will take time to deploy a new release. https://github.com/langgenius/dify/pull/6349

abdullah432 commented 3 months ago

Perfect, What do you think how much time this will take to go live? Also would like this Github issue to be active until it's the solution is released to live version.

crazywoola commented 3 months ago

Perfect, What do you think how much time this will take to go live? Also would like this Github issue to be active until it's the solution is released to live version.

@abdullah432 You could try it now to see if this persists.

abdullah432 commented 3 months ago

Now it's start retuning bytes but it's crashing the application each time and I think the response is corrupted.

Error message:

flutter: \M-o\M-?\M-=\M-o\M-?\M-=\134^X {D_6\M-o\M-?\M-=;\M-o\M-?\M-=.\M-o\M-?\M-=l\M-o\M-?\M-=\M-o\M-?\M-=vDA\M-o\M-?\M-=&\M-o\M-?\M-=\^L\134^U0*\M-o\M-?\M-=\134^AP\134^UL\134^Z \M-o\M-?\M-=LI\M-o\M-?\M-=\M-o\M-?\M-=LC\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=\M-k\M-)\M^L\134^O@f\^L\134^Z\M-o\M-?\M-=1L\134^Z\M-o\M-?\M-=gL5\M-o\M-?\M-=(\M-o\M-?\M-=\M-o\M-?\M-= \M-U\M^NT\M-o\M-?\M-=\M-o\M-?\M-=\134^NP[\M-o\M-?\M-=\134^Y0D\M-L\M^L"\134^C\^LT
flutter: f\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=c\M-o\M-?\M-=9\134^Fo$:(\134^G
flutter: \M-o\M-?\M-=)\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=0
\M-o\M-?\M-=N\134^H\M-o\M-?\M-=\M-o\M-?\M-=(\134^[\M-o\M-?\M-=\134^T\134^E\M-o\M-?\M-=\M-o\M-?\M-=\134^T+\M-o\M-?\M-=h~\M-o\M-?\M-=R>\134^U
flutter: *
flutter: L!\^P
flutter: \134^U5V\M-o\M-?\M-=uE\M-o\M-?\M-=\M-o\M-?\M-=\134^X\M-o\M-?\M-=\M-[\M^B\M-\\M^L(|E\M-o\M-?\M-=_\M-o\M-?\M-=}^(\134^D5\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=\M-K\M^Y\M-o\M-?\M-=x\134^S\M-o\M-?\M-=\134^]w\134^]\M-o\M-?\M-=\134^T\M-o\M-?\M-=\134^[Y\M-o\M-?\M-=\M-o\M-?\M-=L8\M-o\M-?\M-=*\134^?\M-o\M-?\M-=Ru1EU\M-o\M-?\M-=1\M-o\M-?\M-=p\M-o\M-?\M-=5H\134^]\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=\M-W\M^E\M-o\M-?\M-=\M-o\M-?\M-=\134^[ok\134^[Z\M-o\M-?\M-=vD\M-d\M-'\M-4\M-o\M-?\M-=@\134^T\M-o\M-?\M-=\134^D\M-o\M-?\M-=+j\134^D\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=x
flutter: 3\134^A\M-o\M-?\M-=\134^^\134^R\134^^\134^D\134^T\134^B" \134^\134\134^H1
flutter: {\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=?T\M-o\M-?\M-=(\M-o\M-?\M-=js\M-o\M-?\M-=,\M-o\M-?\M-=\M-o\M-?\M-=(\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=yO\M-o\M-?\M-=\M-o\M-?\M-=\M-Z\M-'\M-o\M-?\M-=[>\M-o\M-?\M-=\134\M-f\M-5\M^Gy\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=\134^]\M-o\M-?\M-=M\M-[\M-$\M-o\M-?\M-=\^Ke\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=q\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=WZ\M-o\M-?\M-=,\M-o\M-?\M-=\M-o\M-?\M-=*P\134^F\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=\134^]1\M-o\M-?\M-=
Bad UTF-8 encoding (U+FFFD; REPLACEMENT CHARACTER) found while decoding string: flutter: bytes:����_�9�Z�5O�3�
yْdr+��j�Рu%�'F�>dB�@�q�Tٝ6eƙI�t��b��i�(�����d�YCc1CrC�#�#�
                                                         !�`��9�9�ɚ. The Flutter team would greatly appreciate if you
could file a bug explaining exactly what you were doing when this happened:
https://github.com/flutter/flutter/issues/new/choose
The source bytes were:
[102, 108, 117, 116, 116, 101, 114, 58, 32, 98, 121, 116, 101, 115, 58, 239, 191, 189, 239, 191, 189, 239, 191, 189, 239, 191, 189, 0, 95,
239, 191, 189, 57, 239, 191, 189, 5, 90, 239, 191, 189, 0, 53, 79, 239, 191, 189, 51, 239, 191, 189, 12, 121, 217, 146, 100, 114, 43, 29,
239, 191, 189, 239, 191, 189, 106, 239, 191, 189, 208, 160, 117, 37, 30, 239, 191, 189, 39, 70, 25, 239, 191, 189, 62, 100, 66, 239, 191,
189, 64, 239, 191, 189, 113, 239, 191, 189, 84, 217, 157, 54, 101, 198, 153, 1, 6, 16, 2, 26, 73, 239, 191, 189, 116, 239, 191, 189, 239,
191, 189, 4, 4, 98, 3, 26, 239, 191, 189, 239, 191, 189, 105, 239, 191, 189, 40, 239, 191, 189, 239, 191, 189, 239, 191, 189, 239, 191, 189,
239, 191, 189, 100, 239, 191, 189, 239, 191, 189, 3, 8, 89, 67, 0, 99, 49, 67, 114, 67, 239, 191, 189, 35, 239, 191, 189, 35, 239, 191, 189,
194, 129, 12, 33, 239, 191, 189, 96, 2, 239, 191, 189, 239, 191, 189, 57, 239, 191, 189, 57, 239, 191, 189, 201, 154]

Bad UTF-8 encoding (U+FFFD; REPLACEMENT CHARACTER) found while decoding string: flutter: bytes:����_�9�Z�5O�3�
yْdr+��j�Рu%�'F�>dB�@�q�Tٝ6eƙI�t��b��i�(�����d�YCc1CrC�#�#�
                                                         !�`��9�9�ɚ. The Flutter team would greatly appreciate if you
could file a bug explaining exactly what you were doing when this happened:
https://github.com/flutter/flutter/issues/new/choose
The source bytes were:
[102, 108, 117, 116, 116, 101, 114, 58, 32, 98, 121, 116, 101, 115, 58, 239, 191, 189, 239, 191, 189, 239, 191, 189, 239, 191, 189, 0, 95,
239, 191, 189, 57, 239, 191, 189, 5, 90, 239, 191, 189, 0, 53, 79, 239, 191, 189, 51, 239, 191, 189, 12, 121, 217, 146, 100, 114, 43, 29,
239, 191, 189, 239, 191, 189, 106, 239, 191, 189, 208, 160, 117, 37, 30, 239, 191, 189, 39, 70, 25, 239, 191, 189, 62, 100, 66, 239, 191,
189, 64, 239, 191, 189, 113, 239, 191, 189, 84, 217, 157, 54, 101, 198, 153, 1, 6, 16, 2, 26, 73, 239, 191, 189, 116, 239, 191, 189, 239,
191, 189, 4, 4, 98, 3, 26, 239, 191, 189, 239, 191, 189, 105, 239, 191, 189, 40, 239, 191, 189, 239, 191, 189, 239, 191, 189, 239, 191, 189,
239, 191, 189, 100, 239, 191, 189, 239, 191, 189, 3, 8, 89, 67, 0, 99, 49, 67, 114, 67, 239, 191, 189, 35, 239, 191, 189, 35, 239, 191, 189,
194, 129, 12, 33, 239, 191, 189, 96, 2, 239, 191, 189, 239, 191, 189, 57, 239, 191, 189, 57, 239, 191, 189, 201, 154]

Check out this screenshot.

crazywoola commented 3 months ago

Not sure if this error is raised by us.

Can you try this endpoint instead. We have update that in order to support real time tts stream.

abdullah432 commented 3 months ago

Thanks for trying to help, we spend few hours but it's quite clear that the base64 response is malformed.

Error Message:

Bad UTF-8 encoding (U+FFFD; REPLACEMENT CHARACTER) found while decoding string: flutter:
bytes:����c�9��v|���)Y�7��Ś���.-3�VS��D����\@E�dmx�!�NZr�y@�&i*}��"N�.'��
                                                                         �(�3�湦z"ǖ�#L��� �`�   C��ttkC�4$cf ����0�0B
��p���P$��ʊ�~;��QL�. The Flutter team would greatly appreciate if you could file a bug explaining exactly what you were doing when this happened:
https://github.com/flutter/flutter/issues/new/choose
The source bytes were:
[102, 108, 117, 116, 116, 101, 114, 58, 32, 98, 121, 116, 101, 115, 58, 239, 191, 189, 239, 191, 189, 239, 191, 189, 239, 191, 189, 0, 99, 239, 191, 189, 57,
239, 191, 189, 2, 239, 191, 189, 118, 124, 17, 239, 191, 189, 239, 191, 189, 239, 191, 189, 41, 89, 239, 191, 189, 55, 25, 239, 191, 189, 239, 191, 189, 197,
154, 239, 191, 189, 239, 191, 189, 239, 191, 189, 0, 3, 15, 0, 46, 19, 17, 45, 51, 17, 239, 191, 189, 4, 19, 86, 83, 239, 191, 189, 239, 191, 189, 68, 239, 191,
189, 239, 191, 189, 239, 191, 189, 239, 191, 189, 101, 8, 92, 239, 191, 189, 8, 64, 16, 6, 0, 69, 239, 191, 189, 100, 109, 120, 239, 191, 189, 0, 33, 239, 191,
189, 78, 90, 114, 239, 191, 189, 121, 64, 239, 191, 189, 38, 105, 42, 125, 239, 191, 189, 239, 191, 189, 34, 78, 239, 191, 189, 46, 39, 239, 191, 189, 239, 191,
189, 12, 239, 191, 189, 40, 239, 191, 189, 51, 239, 191, 189, 230, 185, 166, 122, 34, 199, 150, 239, 191, 189, 35, 76, 239, 191, 189, 239, 191, 189, 113, 8, 239,
191, 189, 1, 16, 24, 32, 239, 191, 189, 96, 239, 191, 189, 9, 13, 67, 239, 191, 189, 239, 191, 189, 116, 2, 116, 107, 67, 239, 191, 189, 52, 36, 99, 102, 32,
239, 191, 189, 239, 191, 189, 239, 191, 189, 239, 191, 189, 25, 17, 48, 16, 239, 191, 189, 25, 5, 48, 1, 66, 32, 239, 191, 189, 239, 191, 189, 112, 17, 239, 191,
189, 239, 191, 189, 29, 25, 239, 191, 189, 16, 80, 36, 239, 191, 189, 239, 191, 189, 202, 138, 239, 191, 189, 126, 20, 59, 239, 191, 189, 239, 191, 189, 81, 76,
239, 191, 189, 24]

Debug Mode View:

The problem with solution you suggested: We're using response_mode:blocking instead of streaming and the tts response only come with streaming.

Note: Before migrating to dify we were using openai text to speech api and it's working without any bug.

Starlento commented 3 months ago

I have the same issue. Only streaming has tts_message, and the audio seems cannot be decoded.

Here is an example output:

data: {"event": "tts_message", "conversation_id": "b4e06894-f6a8-4089-8e79-540256a21351", "message_id": "d6ce5dea-7bb6-4033-b205-61c35d2b9cc9", "created_at": 1721301974, "task_id": "20776968-40a7-4ae3-92b3-2e456ed657fd", "audio": "YkFDHSoAwtMkjAcHJkcXmNxEZFOBn4YGOyoYICZqdnmdTwHBY1CejEKJKwKZAJ5g4YBjsMUgow6RCQDmMxECA2YkBg4BTAAGQfGilOF6goGiILFAJTiHRAj2guLEAGgFLRHAODqwKCVTNONqK/3mgqEwRy9KXZisffCDOwIgEegSACFhEATCoJLSGAgwXkBQQLLLDrzV+2KGGkvBJYXIIayhi3BcueSYQRl64dMDApcJeRChNNY6CctezxdbnQ5FLcAOQ5EUtWLFarbp961L8tXcKemsVL+OG8bd29Xl1TGtSPDP0LqOm38ANISIfFt4fkEN08vv1JXBjoshf5+W3Zq56KDP0pkvmSN2SAeaHZq7T3rVaKVpy7WpcrmrVBWqN3sUtC/8VYk+tK9DS2wtDgWLQJLKKpEVCYMAFMDsCUwyATjpzZjMdQCswghEjDcb9MfslwwkgXDD1BPM5Ya8x6w7zAIAXMM0+w0qBlzWUNTGNQD8nljTshjBUJDLtyjskyjIwKjAwNDFAfTSEBTIADzE8QDBRmDFgOTCIFTEAjj/8+TEaHD8OiQE93Sc3WHAxdBMwOBgwUiYxAOoyyHszUUo+HMIz4L836uM9idU1RIczWDYhKcyWYYzbSUyOI8xyP01sQgxjCszNJo2PFI1JEUwcKIzvKgywP0RCWYgAkYXjwYUDuYXhmZLksZPg4YcCEYHiqYnCOZngYcMa9sdMKa02CXBnUhrD5rU41FMgiOGREBsAhDLkjHIzeLyIiIUBozQYuSYY4IhJkgcmQyR+aOgFWqzmzTfHom/skeFTJ813N0AqQ24YzS0t8bSGbWcbUoZMgoUyF+X5wYcyllrgtdf1yX9neTLsrtXTEHSSGQlKDQAsMoEkMnVBq5S/peGFOy5MOuy7t6VS6zvHHdNapvxuzNqmkUrsy3LHf8r4c5veX8lUu1lcjUajUaf6XXYzGo1Lu95h3O/3DGvLbEzhGbvHio6W5vGtLrX42ccsu/l37deq/tW5E4cZW/MIcmXRGUV5iA+YVJcMFtCATAkAzow+8ntNgm8fTlaQMAwLkXnMmsO7zxolZQw6gFeMKtFmjJezCI0CcyXMKnB1TDjgRgxXsjuM7rqMk2dOCIzMUi9NNBmNAyiNAnJONFfM1hAMnQTNLozNuBRHSSNf7APX48M+EqM4AGNVYuOX1ZMDkH/8+TEd3TEOfgA/3ScTTGujhKMzdpgjWF8jJK0TzbjzOIKDQdNjB5czh5qzWIWzFpajd93jLUlBb8TEtMDRsZjShQzoR2zPkQgE9BhEVhlSWhjKIJkgNoEFQx5EAxIKMxLDEyMHswgDIxdEACBycAObfMd8Mca+ZVKeCqaB0fGwQ=="}

abdullah432 commented 3 months ago

@crazywoola Could you let us know if you've identified the bug and are currently working on it? If so, how long do you estimate it will take to resolve?

crazywoola commented 3 months ago

I have the same issue. Only streaming has tts_message, and the audio seems cannot be decoded.

Here is an example output:

data: {"event": "tts_message", "conversation_id": "b4e06894-f6a8-4089-8e79-540256a21351", "message_id": "d6ce5dea-7bb6-4033-b205-61c35d2b9cc9", "created_at": 1721301974, "task_id": "20776968-40a7-4ae3-92b3-2e456ed657fd", "audio": "YkFDHSoAwtMkjAcHJkcXmNxEZFOBn4YGOyoYICZqdnmdTwHBY1CejEKJKwKZAJ5g4YBjsMUgow6RCQDmMxECA2YkBg4BTAAGQfGilOF6goGiILFAJTiHRAj2guLEAGgFLRHAODqwKCVTNONqK/3mgqEwRy9KXZisffCDOwIgEegSACFhEATCoJLSGAgwXkBQQLLLDrzV+2KGGkvBJYXIIayhi3BcueSYQRl64dMDApcJeRChNNY6CctezxdbnQ5FLcAOQ5EUtWLFarbp961L8tXcKemsVL+OG8bd29Xl1TGtSPDP0LqOm38ANISIfFt4fkEN08vv1JXBjoshf5+W3Zq56KDP0pkvmSN2SAeaHZq7T3rVaKVpy7WpcrmrVBWqN3sUtC/8VYk+tK9DS2wtDgWLQJLKKpEVCYMAFMDsCUwyATjpzZjMdQCswghEjDcb9MfslwwkgXDD1BPM5Ya8x6w7zAIAXMM0+w0qBlzWUNTGNQD8nljTshjBUJDLtyjskyjIwKjAwNDFAfTSEBTIADzE8QDBRmDFgOTCIFTEAjj/8+TEaHD8OiQE93Sc3WHAxdBMwOBgwUiYxAOoyyHszUUo+HMIz4L836uM9idU1RIczWDYhKcyWYYzbSUyOI8xyP01sQgxjCszNJo2PFI1JEUwcKIzvKgywP0RCWYgAkYXjwYUDuYXhmZLksZPg4YcCEYHiqYnCOZngYcMa9sdMKa02CXBnUhrD5rU41FMgiOGREBsAhDLkjHIzeLyIiIUBozQYuSYY4IhJkgcmQyR+aOgFWqzmzTfHom/skeFTJ813N0AqQ24YzS0t8bSGbWcbUoZMgoUyF+X5wYcyllrgtdf1yX9neTLsrtXTEHSSGQlKDQAsMoEkMnVBq5S/peGFOy5MOuy7t6VS6zvHHdNapvxuzNqmkUrsy3LHf8r4c5veX8lUu1lcjUajUaf6XXYzGo1Lu95h3O/3DGvLbEzhGbvHio6W5vGtLrX42ccsu/l37deq/tW5E4cZW/MIcmXRGUV5iA+YVJcMFtCATAkAzow+8ntNgm8fTlaQMAwLkXnMmsO7zxolZQw6gFeMKtFmjJezCI0CcyXMKnB1TDjgRgxXsjuM7rqMk2dOCIzMUi9NNBmNAyiNAnJONFfM1hAMnQTNLozNuBRHSSNf7APX48M+EqM4AGNVYuOX1ZMDkH/8+TEd3TEOfgA/3ScTTGujhKMzdpgjWF8jJK0TzbjzOIKDQdNjB5czh5qzWIWzFpajd93jLUlBb8TEtMDRsZjShQzoR2zPkQgE9BhEVhlSWhjKIJkgNoEFQx5EAxIKMxLDEyMHswgDIxdEACBycAObfMd8Mca+ZVKeCqaB0fGwQ=="}

Yes only streaming will have these kind of messages

crazywoola commented 3 months ago

@charli117 @ic-xu Could you take a look at this one?

ic-xu commented 3 months ago

@charli117 @ic-xu Could you take a look at this one? Okay, I'll take a look now

Thanks for trying to help, we spend few hours but it's quite clear that the base64 response is malformed.

Error Message:

Bad UTF-8 encoding (U+FFFD; REPLACEMENT CHARACTER) found while decoding string: flutter:
bytes:����c�9��v|���)Y�7��Ś���.-3�VS��D����\@E�dmx�!�NZr�y@�&i*}��"N�.'��
                                                                         �(�3�湦z"ǖ�#L��� �`�   C��ttkC�4$cf ����0�0B
��p���P$��ʊ�~;��QL�. The Flutter team would greatly appreciate if you could file a bug explaining exactly what you were doing when this happened:
https://github.com/flutter/flutter/issues/new/choose
The source bytes were:
[102, 108, 117, 116, 116, 101, 114, 58, 32, 98, 121, 116, 101, 115, 58, 239, 191, 189, 239, 191, 189, 239, 191, 189, 239, 191, 189, 0, 99, 239, 191, 189, 57,
239, 191, 189, 2, 239, 191, 189, 118, 124, 17, 239, 191, 189, 239, 191, 189, 239, 191, 189, 41, 89, 239, 191, 189, 55, 25, 239, 191, 189, 239, 191, 189, 197,
154, 239, 191, 189, 239, 191, 189, 239, 191, 189, 0, 3, 15, 0, 46, 19, 17, 45, 51, 17, 239, 191, 189, 4, 19, 86, 83, 239, 191, 189, 239, 191, 189, 68, 239, 191,
189, 239, 191, 189, 239, 191, 189, 239, 191, 189, 101, 8, 92, 239, 191, 189, 8, 64, 16, 6, 0, 69, 239, 191, 189, 100, 109, 120, 239, 191, 189, 0, 33, 239, 191,
189, 78, 90, 114, 239, 191, 189, 121, 64, 239, 191, 189, 38, 105, 42, 125, 239, 191, 189, 239, 191, 189, 34, 78, 239, 191, 189, 46, 39, 239, 191, 189, 239, 191,
189, 12, 239, 191, 189, 40, 239, 191, 189, 51, 239, 191, 189, 230, 185, 166, 122, 34, 199, 150, 239, 191, 189, 35, 76, 239, 191, 189, 239, 191, 189, 113, 8, 239,
191, 189, 1, 16, 24, 32, 239, 191, 189, 96, 239, 191, 189, 9, 13, 67, 239, 191, 189, 239, 191, 189, 116, 2, 116, 107, 67, 239, 191, 189, 52, 36, 99, 102, 32,
239, 191, 189, 239, 191, 189, 239, 191, 189, 239, 191, 189, 25, 17, 48, 16, 239, 191, 189, 25, 5, 48, 1, 66, 32, 239, 191, 189, 239, 191, 189, 112, 17, 239, 191,
189, 239, 191, 189, 29, 25, 239, 191, 189, 16, 80, 36, 239, 191, 189, 239, 191, 189, 202, 138, 239, 191, 189, 126, 20, 59, 239, 191, 189, 239, 191, 189, 81, 76,
239, 191, 189, 24]

Debug Mode View: Screenshot 2024-07-18 at 11 54 47 AM

The problem with solution you suggested: We're using response_mode:blocking instead of streaming and the tts response only come with streaming.

Note: Before migrating to dify we were using openai text to speech api and it's working without any bug.

I'm very sorry for any inconvenience caused to you, https://api.dify.ai/v1/text-to-audio This is called through an API and does not require any decoding. The output is the audio content of MP3, which can be played directly. Its return content is theoretically the same as that of blocking; You can try saving it as an MP3 file and playing it again.

ic-xu commented 3 months ago

Thank you very much for discovering this issue. I have previously used Python to save MP3 files that can be played directly, but I have not tried Flutter; If it's convenient for you, you can directly save the returned content as an MP3 file and try to see if it can be played. I will also conduct synchronous testing and provide you with the results before next Sunday. Do you think this arrangement is acceptable?

abdullah432 commented 3 months ago

@ic-xu I tried what you said, tried to create mp3 file with the response. But it throws this error.

Exception: (-11849) Operation Stopped

Here is my code.

  Future<File?> convertTextToSpeech(String text, AudioPlayer? player) async {
    try {
      debugPrint("convertTextToSpeech: text:: $text");
      var user = read(appUserProvider);

      var data = {
        'text': text,
        'user': user.uid,
        'streaming': false,
      };

      var apiClient = read(apiClientProvider);
      final response =
          await apiClient.post(ApiPaths.convertTextToAudio, data: data);

      if (response != null) {
        Directory dir = await getApplicationDocumentsDirectory();
        File file = File('${dir.path}/converted_audio.mp3');
        await file.writeAsString(response);
        await player?.setFilePath(file.path);
        player?.play();

        return file;
      } else {
        // throw Exception('Failed to load audio');
        return null;
      }
    } catch (e) {
      debugPrint('Exception: $e');
      return null;
    }
  }

Let me know If I'm doing something wrong.

ic-xu commented 3 months ago

I think it should be saved as byte content instead of string content

屏幕截图 2024-07-19 224323

abdullah432 commented 3 months ago

@ic-xu When I asked ChatGPT it's saying following.

This is writeAsByte method

Future<File> writeAsBytes(
  List<int> bytes, {
  FileMode mode = FileMode.write,
  bool flush = false,
})

Which means I need to convert text response to List for which I need to decode the response.

  if (response is String) {
          List<int> bytes = base64Decode(response);
          await file.writeAsBytes(bytes); // Write the decoded bytes
        }

And when I do this. It crash the app with this error.

Bad UTF-8 encoding (U+FFFD; REPLACEMENT CHARACTER) found while decoding string: flutter: ����d41�. The Flutter team would greatly appreciate if you could
file a bug explaining exactly what you were doing when this happened:
https://github.com/flutter/flutter/issues/new/choose
The source bytes were:
[102, 108, 117, 116, 116, 101, 114, 58, 32, 239, 191, 189, 239, 191, 189, 239, 191, 189, 239, 191, 189, 0, 100, 52, 49, 239, 191, 189]

flutter: Exception: FormatException: Invalid character (at character 1)
flutter: \M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=\M-o\M-?\M-=

Note: I was always using file.writeAsBytes which always crash the app, with similar error.

ic-xu commented 3 months ago

It's already late at night in Asia now. Can you wait for me for a moment? I'll try using Flutter and give you a reply tomorrow

abdullah432 commented 3 months ago

Your Image is not visible, please edit it and yes you can go to sleep, I may put another comment but you don't need to reply now. Thank you for your help.

abdullah432 commented 3 months ago

Thank you so much for helping, I figure it out. The solution was to pass responseType: ResponseType.bytes,

    final options = Options(
      headers: {
        'Content-Type': 'application/json',
        'Authorization': 'Bearer $difyApiKey',
      },
      responseType: ResponseType.bytes,
    );

langgenius / dify