Automatically decode html entities

[arcanis]

Shouldn't messages received from Slack be automatically html-decoded ?

[jlsjonas]

I agree, I now have to replace & to & (and in principle the same with < & > ) for url to properly parse the url; alternatively provide a clean() method (in order to decode only when required)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jlsjonas]

2 years later 😅 unless they are meanwhile? (mainly using Mattermost & Telegram currently)

[benbrown]

@jlsjonas thanks for the input -- did you see this when using the Event API or the RTM connection? Or both?

[jlsjonas]

If I remember correctly the project never used the RTM connection; so event API. Both might be affected though (untested)

[benbrown]

OK yes, I can confirm that the entities come through encoded with the events api.

Now the question is, should this happen by default, or via an optional middleware?

[benbrown]

This little snippet of code will do the job, but since it requires a new dependency that would require extra work to keep up to date, I'm not sure if we should include this in core. THOUGHTS?

const Entities = require('html-entities').AllHtmlEntities; 
const entities = new Entities();
// Decode HTML-encoded entities in Slack messages
controller.middleware.receive.use((bot, message, next) => {
  if (bot.type === 'slack' && message.text) {
     message.text = entities.decode(message.text);
  }
  next();  
});

[jlsjonas]

I would vouch to add it to core to remain consistent.

Or alternatively put it in the documentation as a note to using the Slack events API? do you have any stats regarding usage (which could help decide between the 2 options)?

Otoh, if someone creates something based on botkit and doesn't know about the difference for Slack events API this might cause issues that could only (relatively) be solved upstream to them.

Eitherway I would use https://www.npmjs.com/package/entities instead though; as html-entities seems to be no longer maintained (looking at the open issues/pr's)

[benbrown]

Makes sense. Thanks for the recommendation on a better entity package.