search

misskey-dev / misskey

🌎 A completely free and open interplanetary microblogging platform 🚀
https://misskey-hub.net/
GNU Affero General Public License v3.0
9.95k stars 1.35k forks source link

feat: Initial support for TTS functionality #14716

Open MomentQYC opened 6 days ago

MomentQYC commented 6 days ago

What

This is an initial implementation of the TTS functionality, which can subsequently be built upon to improve ease of use, add providers (the existing ones can actually be deployed on our own), and even be set up by the user.

Why

Hopefully it can be an emergency measure for some of the population at some point.

Additional info (optional)

It is currently only a preliminary realization, so only a rough framework has been formed.

Checklist

Thanks to @MarryDream for the guidance in implementing the feature.

codecov[bot] commented 6 days ago

Codecov Report

Attention: Patch coverage is 30.89655% with 501 lines in your changes missing coverage. Please review.

Project coverage is 38.53%. Comparing base (c4c69cd) to head (7b88e74).

Files with missing lines Patch % Lines
...ges/frontend/src/pages/admin/external-services.vue 0.00% 150 Missing :warning:
...ages/backend/src/server/api/endpoints/notes/tts.ts 30.96% 136 Missing :warning:
...kend/src/server/api/endpoints/admin/update-meta.ts 14.60% 76 Missing :warning:
packages/frontend/src/scripts/get-note-menu.ts 0.00% 42 Missing :warning:
...ackages/frontend/src/components/MkNoteDetailed.vue 0.00% 33 Missing :warning:
packages/frontend/src/scripts/misskey-api.ts 0.00% 26 Missing :warning:
packages/frontend/src/pages/admin/roles.editor.vue 0.00% 20 Missing :warning:
packages/frontend/src/pages/admin/roles.vue 0.00% 8 Missing :warning:
packages/frontend/src/pages/settings/profile.vue 0.00% 3 Missing :warning:
packages/backend/src/core/RoleService.ts 71.42% 2 Missing :warning:
... and 5 more
Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #14716 +/- ## =========================================== - Coverage 39.67% 38.53% -1.14% =========================================== Files 1554 1555 +1 Lines 195334 196006 +672 Branches 3565 3335 -230 =========================================== - Hits 77501 75539 -1962 - Misses 117232 119860 +2628 - Partials 601 607 +6 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 6 days ago

このPRによるapi.jsonの差分

差分はこちら ```diff --- base +++ head @@ -157,6 +157,9 @@ "translatorAvailable": { "type": "boolean" }, + "ttsAvailable": { + "type": "boolean" + }, "silencedHosts": { "type": "array", "items": { @@ -598,6 +601,7 @@ "enableEmail", "enableServiceWorker", "translatorAvailable", + "ttsAvailable", "mediaSilencedHosts", "pinnedUsers", "hiddenTags", @@ -14027,6 +14031,71 @@ "deeplIsPro": { "type": "boolean" }, + "hfAuthKey": { + "type": [ + "string", + "null" + ] + }, + "hfSpace": { + "type": "boolean", + "default": false + }, + "hfSpaceName": { + "type": [ + "string", + "null" + ] + }, + "hfexampleAudioURL": { + "type": [ + "string", + "null" + ] + }, + "hfexampleText": { + "type": [ + "string", + "null" + ] + }, + "hfexampleLang": { + "type": [ + "string", + "null" + ] + }, + "hfslice": { + "type": [ + "string", + "null" + ], + "default": "Slice once every 4 sentences" + }, + "hftopK": { + "type": "integer", + "default": 15 + }, + "hftopP": { + "type": "integer", + "default": 100 + }, + "hfTemperature": { + "type": "integer", + "default": 100 + }, + "hfnrm": { + "type": "boolean", + "default": false + }, + "hfSpeedRate": { + "type": "integer", + "default": 125 + }, + "hfdas": { + "type": "boolean", + "default": false + }, "enableEmail": { "type": "boolean" }, @@ -50250,6 +50319,9 @@ "isCat": { "type": "boolean" }, + "isVI": { + "type": "boolean" + }, "injectFeaturedNote": { "type": "boolean" }, @@ -61181,6 +61253,201 @@ } } }, + "/notes/tts": { + "post": { + "operationId": "notes___tts", + "summary": "notes/tts", + "description": "No description provided.\n\n**Credential required**: *Yes* / **Permission**: *read:account*", + "externalDocs": { + "description": "Source code", + "url": "https://github.com/misskey-dev/misskey/blob/develop/packages/backend/src/server/api/endpoints/notes/tts.ts" + }, + "tags": [ + "notes" + ], + "security": [ + { + "bearerAuth": [] + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "noteId": { + "type": "string", + "format": "misskey:id" + } + }, + "required": [ + "noteId" + ] + } + } + } + }, + "responses": { + "200": { + "description": "OK (with results)", + "content": { + "application/json": { + "schema": { + "type": "string", + "contentMediaType": "audio/flac" + } + } + } + }, + "204": { + "description": "OK (without any results)" + }, + "400": { + "description": "Client error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Error" + }, + "examples": { + "INCORRECT_CONFIG": { + "value": { + "error": { + "message": "Incorrect configuration.", + "code": "INCORRECT_CONFIG", + "id": "8d171e60-83b8-11ef-b98c-a7506d6c1de4" + } + } + }, + "UNAVAILABLE": { + "value": { + "error": { + "message": "Convert of notes unavailable.", + "code": "UNAVAILABLE", + "id": "97a0826c-6393-11ef-a650-67972d710975" + } + } + }, + "NO_SUCH_NOTE": { + "value": { + "error": { + "message": "No such note.", + "code": "NO_SUCH_NOTE", + "id": "bea9b03f-36e0-49c5-a4db-627a029f8971" + } + } + }, + "CANNOT_CONVERT_INVISIBLE_NOTE": { + "value": { + "error": { + "message": "Cannot convert invisible note.", + "code": "CANNOT_CONVERT_INVISIBLE_NOTE", + "id": "f57caae0-6394-11ef-8e2a-d706932c1030" + } + } + }, + "INVALID_PARAM": { + "value": { + "error": { + "message": "Invalid param.", + "code": "INVALID_PARAM", + "id": "3d81ceae-475f-4600-b2a8-2bc116157532" + } + } + } + } + } + } + }, + "401": { + "description": "Authentication error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Error" + }, + "examples": { + "CREDENTIAL_REQUIRED": { + "value": { + "error": { + "message": "Credential required.", + "code": "CREDENTIAL_REQUIRED", + "id": "1384574d-a912-4b81-8601-c7b1c4085df1" + } + } + } + } + } + } + }, + "403": { + "description": "Forbidden error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Error" + }, + "examples": { + "AUTHENTICATION_FAILED": { + "value": { + "error": { + "message": "Authentication failed. Please ensure your token is correct.", + "code": "AUTHENTICATION_FAILED", + "id": "b0a7f5f8-dc2f-4171-b91f-de88ad238e14" + } + } + } + } + } + } + }, + "418": { + "description": "I'm Ai", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Error" + }, + "examples": { + "I_AM_AI": { + "value": { + "error": { + "message": "You sent a request to Ai-chan, Misskey's showgirl, instead of the server.", + "code": "I_AM_AI", + "id": "60c46cd1-f23a-46b1-bebe-5d2b73951a84" + } + } + } + } + } + } + }, + "500": { + "description": "Internal server error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Error" + }, + "examples": { + "INTERNAL_ERROR": { + "value": { + "error": { + "message": "Internal error occurred. Please contact us if the error persists.", + "code": "INTERNAL_ERROR", + "id": "5d37dbcb-891e-41ca-a3d6-e690c97775ac" + } + } + } + } + } + } + } + } + } + }, "/notes/unrenote": { "post": { "operationId": "notes___unrenote", @@ -77091,6 +77358,9 @@ "isCat": { "type": "boolean" }, + "isVI": { + "type": "boolean" + }, "instance": { "type": "object", "properties": { @@ -81150,6 +81420,7 @@ "isLocked", "isBot", "isCat", + "isVI", "isExplorable" ] } @@ -81442,6 +81713,9 @@ "canUseTranslator": { "type": "boolean" }, + "canUseTTS": { + "type": "boolean" + }, "canHideAds": { "type": "boolean" }, @@ -81513,6 +81787,7 @@ "canManageAvatarDecorations", "canSearchNotes", "canUseTranslator", + "canUseTTS", "canHideAds", "driveCapacityMb", "alwaysMarkNsfw", @@ -82056,6 +82331,9 @@ "translatorAvailable": { "type": "boolean" }, + "ttsAvailable": { + "type": "boolean" + }, "mediaProxy": { "type": "string" }, @@ -82160,6 +82438,7 @@ "enableEmail", "enableServiceWorker", "translatorAvailable", + "ttsAvailable", "mediaProxy", "enableUrlPreview", "backgroundImageUrl", ``` [Get diff files from Workflow Page](https://github.com/misskey-dev/misskey/actions/runs/11310069374)
kakkokari-gtyih commented 6 days ago

I think we should discuss the feature first (especially large scale one like this) in an issue before actually implement it

MomentQYC commented 6 days ago

I think we should discuss the feature first (especially large scale one like this) in an issue before actually implement it

14721