geigerzaehler / beets-alternatives

Beets plugin to manage external files
MIT License
93 stars 21 forks source link

No support for non-utf8 paths #21

Open wisp3rwind opened 5 years ago

wisp3rwind commented 5 years ago

beets-alternatives currently does not support non-utf8 paths, while beets does (I didn't really test any of this, that's only guessed from reading the code).

When the Python 3 PR is applied, the only remaining problem should be path storage in the database. The issue is that beets' flexible attributes are treated as sqlite TEXT. While sqlite appears to support arbitrary binary data in its TEXT columns as long as no SQL operations tries to interpret it as actual encoded text, Python's sqlite3.Connection.text_factory interprets the data as unicode strings (grep suggests that bees does not change text_factory from the default). Whether the default text_factory supports surrogate escapes is not documented (and from a quick look at the cpython code, I couldn't tell either). Thus, supporting such paths can probably not be solved by using a custom beets.dbcore.types.Type subclass, but would require changes to beets itself. Maybe using the backslashreplace error handler with the utf8 codec could also work (which might bring up issues with backwards-compatibility, though).

Personally, I don't really care about non-ut8 paths, but wanted to track this here either way.

geigerzaehler commented 5 years ago

I don’t think there is a need for supporting non-UTF8 paths and it looks like implementing it would be hard. I’ll mark this as wontfix