search

larryhudson / astro-sqlite-tts-feed

astro-sqlite-tts-feed
5 stars 0 forks source link

Bug: character encoding issue with text in SQLite database #6

Closed larryhudson closed 11 months ago

larryhudson commented 11 months ago

I'm getting some funny character encoding issues when saving text into the SQLite database, for example: The disability royal commission recommendations could fix some of the worst living conditions � but that�s just the start Chlo� Caillet - BBC Radio 1 - Essential Mix

I'm not sure if I need to escape text before I save it with better-sqlite3, or whether there's an option to set unicode encoding on the database (or something like that).

larryhudson commented 11 months ago

I realised that I haven't been using a proper HTML template in the Astro web app - so <meta charset="UTF-8"> was not set in the <head> of the HTML webpages. It looks like that has partially solved the issue, I can now update an article title by copying and pasting in a string with an en dash in it. But I'm still having a bit of trouble when I'm creating a new article. Will keep experimenting.

larryhudson commented 11 months ago

The funny � characters have now been replaced by squares with question marks in them: image

Progress?

larryhudson commented 11 months ago

I'm troubleshooting this issue over here: https://github.com/larryhudson/astro-sqlite-utf8-troubleshooting

larryhudson commented 11 months ago

After a bit more experimenting, I wiped the database and started again, and now the characters seem to be working OK: image

So the lesson of the story is to always include the regular HTML boilerplate. I've been lazy with that because I've been using Neovim and I don't have the Astro + HTML + Emmet stuff set up.

larryhudson commented 11 months ago

Side note: I just got Emmet working in Neovim (I use LazyVim) by installing emmet-language-server in Mason (leader+cp). That was easy!