Closed MachengShen closed 3 years ago
Yes, video
messagge is not supported now, and pr is welcome. I believe this feature is simple for you. You can have a try so you can become a wechaty contributor 👍
Please refer to :
We now support very limit message types:
In order to support receiving more message types, like audio, you need to look at:
I hope the above information can help you and please feel free to let me know if you have more questions. @MachengShen
Thanks! I added 'voice' to the 'knownTypeList', and the voice message triggers the onMessage() method in ding-dong.ts, but the message type is 'text' instead of 'audio'. The log shows that the received message was 'voice' before 'const msgPayload = await puppet.messagePayload(payload.messageId)' was called, so my understanding is that something in the wechaty-puppet repo is messing up with the payload?
I attached the code piece to reproduce this issue and the log (level 'silly') from the console:
Steps for reproduce:
ding-dong.ts:
async function onMessage (payload: EventMessagePayload) {
const msgPayload = await puppet.messagePayload(payload.messageId)
log.verbose('ding-dong', `onMessage: ${JSON.stringify(msgPayload)}`)
}
webhook.ts:
const knownTypeList = [
'text',
'image',
'voice',
]
Then send a voice message to the official account
Logs:
12:33:26 VERB Webhook appPost({url: /?signature=4729e1e02e26de6bf16eefa036a7a66ce1fca448×tamp=1612326804&nonce=1097261724&openid=ozm_X6eCQ_4Z-r1xcEKmLLR_P5cA} with payload: {"ToUserName":"gh_89b011c540f8","FromUserName":"ozm_X6eCQ_4Z-r1xcEKmLLR_P5cA","CreateTime":"1612326804","MsgType":"voice","MediaId":"_0iIuUnF6_UbjOGWYQeScoxKskhaeFdtIz2C2WcT86wTp-ij7vdukdrUyhu-CZ9i","Format":"amr","MsgId":"23082968392782987","Recognition":""}
12:33:26 VERB PayloadStore setMessagePayload(23082968392782987, {"ToUserName":"gh_89b011c540f8","FromUserName":"ozm_X6eCQ_4Z-r1xcEKmLLR_P5cA","CreateTime":"1612326804","MsgType":"voice","MediaId":"_0iIuUnF6_UbjOGWYQeScoxKskhaeFdtIz2C2WcT86wTp-ij7vdukdrUyhu-CZ9i","Format":"amr","MsgId":"23082968392782987","Recognition":""})
12:33:26 VERB Puppet messagePayload(23082968392782987)
12:33:26 SILL Puppet messagePayloadCache(23082968392782987) cache MISS
12:33:26 VERB PuppetOA messageRawPayload(23082968392782987)
12:33:26 VERB PayloadStore getMessagePayload(23082968392782987)
12:33:26 SILL Puppet messagePayload(23082968392782987) cache SET
12:33:26 VERB ding-dong onMessage: {"fromId":"ozm_X6eCQ_4Z-r1xcEKmLLR_P5cA","id":"23082968392782987","timestamp":1612326804,"toId":"gh_89b011c540f8","type":7}
Congratulations! It seems that you can receive the right webhook payload from WeChat Official Server when you sent an audio message to your Official Account!
However, it seems that the Wechaty Message Payload has not been set correctly. In order to make it correct, you need to understand the message processing flow in Wechaty Puppet.
The Webhook get called by the Tencent Server (what you have done already)
The message event will be propagated from the Webhook
class to the OfficialAccount
class:
The message event will be propagated from the OfficialAccount
class to the PuppetOA
class:
After message event be propagated from the PuppetOA to Wechaty, then the puppet.messagePayload() will be called to get the Wechaty Message Payload. Here is the most important part: we need to construct a Wechaty Message Payload from the Raw Message Payload (source code at here):
const rawPayload = await this.messageRawPayload(messageId)
const payload = await this.messageRawPayloadParser(rawPayload)
So it will be very clear that, what we need to do to support the new Message Type (audio in this case), is to implement the messageRawPayload
methods, which you can find here:
I hope the above explanation could help you to move forward, please feel free to let me know if you have more questions.
Review for #21
The Wechaty will call puppet.messageFile(id)
to get the audio file from a message, so you need to implemented the messageFile()
method in puppet. See:
The Wechaty will call message.say(fileBox)
to send audio message, which will call puppet.messageSendFile(fileBox)
under the hood. So you need to implemented the messageSendFile()
method in puppetOA
. See:
Thanks for your prompt reply.
One issue I ran into when trying to update messageSendFile()
is mime type. The function uses mime type to determine OAMediaType
, and bails out if the mime type is not supported.
FileBox.mimeType
is inferred from the name of the file. However, the depending mime
package fails to infer the expected type audio/AMR
from extension name .amr
https://github.com/huan/file-box/blob/master/src/file-box.ts#L270
What would you suggest to work around this?
In my understanding, our problem is that the mime
package can not get the right type for the .amr
file?
My suggestion would like:
mime
package to track this problem, andmime
package for fixing this problem from the root.Please let me what you think about this solution, thanks.
It seems that the mime
package has a mime.define()
method to add a new mime type to the module.
So we can add the following line to FileBox which I believe could fix the mime issue for amr
.
// Require Mime class
const Mime = require('mime/Mime');
// Define mime type -> extensions map
const typeMap = {
'audio/amr': ['amr'],
};
// Create and use Mime instance
const myMime = new Mime(typeMap);
myMime.getType('amr'); // ⇨ 'audio/amr'
@huan Thanks for adding support for AMR to FileBox. The solution makes sense.
I have opened an PR in mime-db (the data source backing mime
) to add audio/amr support. I will update the thread once that is landed.
I saw it has been merged already, cheers!
Please link the issues together in the future, for example, ref to this issue to https://github.com/jshttp/mime-db/issues/228 so that we can always sync the latest updates.
I believe this feature has no blocker anymore, so please go ahead and feel free to let me know if you have any new questions.
Happy Chinese New Year!
🎉👍
Sent from my iPhone
On Feb 13, 2021, at 1:08 PM, Huan (李卓桓) notifications@github.com wrote:
I saw it has been merged already, cheers!
Please link the issues together in the future, for example, ref to this issue to jshttp/mime-db#228https://github.com/jshttp/mime-db/issues/228 so that we can always sync the latest updates.
I believe this feature has no blocker anymore, so please go ahead and feel free to let me know if you have any new questions.
Happy Chinese New Year!
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/wechaty/wechaty-puppet-official-account/issues/19#issuecomment-778564786, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALM6ERQGH5UVPKCEZ7IUUXLS6YCMJANCNFSM4W4A243Q.
Please link the issues together in the future, for example, ref to this issue to jshttp/mime-db#228 so that we can always sync the latest updates.
Good call! I wasn't aware this feature exists. Will definitely do that more in the future. I don't foresee any blocker as well. Will address the comments and update the PR.
Happy Lunar New Year to you, too!
Hi @huan , I have updated #21 to address comments in this thread. I manually tested the feature by running the added echo.ts
with a test OA. Not sure what's the suggested approach to automate testing?
Hi, Thanks for the nice OA puppet. I'm interested in receiving and sending audio messages with wechat users through OA. Currently the puppet.messageSendFile() method throws an 'unsupported file exception' when trying to send SILK encoded audio files, or .mp3, and the 'EventMessagePayload' also does not receive audio input from the user. Would appreciate it if we can add this new feature, please instruct how to do it. Thanks!