UnicodeEncodeError: 'utf-8' codec can't encode character

[volcs0]

I imported my collection, and I see the folder structure, but I don't see an album art or any other art. None. Any ideas how to get this working? unRAID with Docker container using latest pull. I don't see any errors in the log. Sorry if this is the wrong place to log this issue. Happy to move it. Thanks.

[cwilvx]

Hello @volcs0

Thanks for opening the issue.

The music thumbnails do not show immediately, they are processed after all files are indexed. Depending on the size of your library and hardware the duration can vary.

[cwilvx]

Check whether you can see a progress with the following text in your terminal: Extracting track images

If it's completed, try reloading your page (to clear cached images). Please let me know whether it works.

[volcs0]

I'm not sure how to check if image extraction is running.

It finished importing three days ago, and I've seen nothing else. There are no files being stored in appdata/swingmusic.

Thanks for the advice.

[cwilvx]

Huh! 🤔 ...

To be honest, I'm not familiar with unRAID at all (I just googled it - actually I'm unfamiliar with RAID and NAS too) and Docker is not my specialty (The Docker configurations in the repo are contributed).

I'm currently, finalizing the next release v1.3.0. ie. docs, distributables, etc. Can you give me about a week to figure out this Docker stuff? I will get back to this issue once I merge the new version into master and https://github.com/swing-opensource/swingmusic/pull/141 too which related to Docker.

If you like, you can try building the Docker image from the v1.3.0 branch. If you run into Non-Docker issues I can help.

Thanks.

[volcs0]

Terrific. Thanks for responding. I will look forward to the new release. In the meanwhile, I'll try to figure out how to build it myself. Thnaks.

[cwilvx]

😅 if you manage to build the container on the v1.3.0 branch, you'll be the first person on the new release. 🔥🔥

If you manage, please feel free to open a PR.

[volcs0]

So, I rebuilt it and made a docker and I imported one artist - and it worked! At least the albums had artwork, though the artist itself did not. So, then I did a full import of my collection - 190,000 tracks. It did the import but there is no art. In the log, I see this error. Any thoughts? I don't want to import each artist one at a time, but something errors out when I try the entire library. Thanks.

Exception in thread Thread-5 (run_periodic_checks): Traceback (most recent call last): File "/usr/local/lib/python3.11/threading.py", line 1038, in _bootstrap_inner self.run() File "/usr/local/lib/python3.11/threading.py", line 975, in run self._target(*self._args, **self._kwargs) File "/app/swingmusic/app/functions.py", line 28, in run_periodic_checks Populate(key=get_random_str()) File "/app/swingmusic/app/lib/populate.py", line 72, in __init__ self.tag_untagged(untagged, key) File "/app/swingmusic/app/lib/populate.py", line 121, in tag_untagged insert_many_tracks(tagged_tracks) File "/app/swingmusic/app/db/sqlite/tracks.py", line 54, in insert_many_tracks cls.insert_one_track(track, cur) File "/app/swingmusic/app/db/sqlite/tracks.py", line 45, in insert_one_track cur.execute(sql, track) UnicodeEncodeError: 'utf-8' codec can't encode character '\udce0' in position 99: surrogates not allowed

[cwilvx]

@volcs0

The error above is caused by invalid characters (like this one: �) in music tags. I have fixed the issue on v1.3.0 branch with this commit.

---

To try and make things easier for you, I have build a docker image for you using a distributable on Ubuntu from the latest version (with a fix for the problem above). You can try it out:

docker pull geoffrey45/swingmusic.v1.3.0.beta:volcs0

docker run --name swingmusicv130beta -p 1970:1970 \
  -v /path/to/music:/music \
  -v /path/to/config:/config \
  --restart unless-stopped \
  geoffrey45/swingmusic.v1.3.0.beta:volcs0

Replace /path/to/music with your music path, and /path/to/config with the path you want to create the config directory.

I hope the new version fixes some of the issues you encountered. Let me know how it goes.

---

The Dockerfile looks like this:

FROM ubuntu:latest
WORKDIR /
COPY ./dist/swingmusic /swingmusic
EXPOSE 1970/tcp
VOLUME /music
VOLUME /config
ENTRYPOINT ["/swingmusic", "--host", "0.0.0.0", "--config", "/config"]

Docker Hub link: here

P.S Wow! Your library is enormous. 190k files 😅 I'm assuming that's like 10+ years of collecting. I wish I had that many files (for testing purposes). My library is nothing compared to yours (12k).

Let me know whether it works.

Edit: Given your collection size, artist images and similar artists data will definitely take time to download ... unless they are all from the same artist 😂

[volcs0]

Thanks. I did this - was easy.

When I ran the full import, this is the error I got. Also, no additional album art is being added.

Adding 188633 tracks to database Exception in thread Thread-5 (run_periodic_scans): Traceback (most recent call last): File "threading.py", line 1016, in _bootstrap_inner File "threading.py", line 953, in run File "app/periodic_scan.py", line 26, in run_periodic_scans File "app/lib/populate.py", line 81, in __init__ File "app/lib/populate.py", line 187, in tag_untagged File "app/db/sqlite/tracks.py", line 69, in insert_many_tracks File "app/db/sqlite/tracks.py", line 52, in insert_one_track File "app/db/sqlite/tracks.py", line 48, in force_surrogatepass UnicodeDecodeError: 'utf-16-le' codec can't decode bytes in position 136-137: illegal encoding

[cwilvx]

@volcs0

Looks like another unicode issue. If I attempt another patch, I'll just be shooting arrows in the dark.

Since I can't reproduce the issue on my end, I've pushed a new image on tag volcs0_v2 that will print tags of the tracks causing the issues.

Pull it, run it and send over the JSON printed on your terminal so that I can use the data to fix the unicode issues.

docker pull geoffrey45/swingmusic.v1.3.0.beta:volcs0_v2

docker run --name swingmusicv130betav2 -p 1970:1970 \
  -v /path/to/music:/music \
  -v /path/to/config:/config \
  --restart unless-stopped \
  geoffrey45/swingmusic.v1.3.0.beta:volcs0_v2

The App version should read something like:

SwingMusic 1.3.0 :volcs0 (v2) 

Paste the JSON on https://pastebin.com and share the link. I'll work on it tomorrow morning.

Edit: You don't have to use pastebin, you can also paste them into a .txt file and attach it.

[volcs0]

Okay thanks. I'll try this.

Any ideas why when I do a few albums the artwork is fine, but when I try to do the entire library it doesn't work?

[cwilvx]

@volcs0

Yeah ... the errors prevent the app from getting to the point where the thumbnails are extracted. The app basically stops executing further instructions after encountering unhandled errors.

With the subset you used (the ones where thumbnails are OK) the files' tags or filenames don't have the characters causing errors. But for the whole library, at least one file has messy characters, which are causing the unhandled errors.

So the thumbnails can still miss if you select a tiny group of files containing some specific files with the messy characters in tags or filenames.

This will be fixed soon once you send the JSON.

[volcs0]

So, this time, it actually went through and grabbed the album and artist art. It looks great. Here is the log output, presumably of the tracks it tripped over. https://pastebin.com/xb1dme1i

It seems to be stuck in a loop doing this now: https://imgur.com/a/r0C2eDS Over and over.

Anyway, what's there looks and works great. (Separate issue, any plans for an Android client - it doesn't really work well in the browser on mobile).

Thanks - great work. It's so fast.

[cwilvx]

I handled the errors by printing the faulty tags in the terminal instead of trying to add them to the database. So the printed tracks were not added to your library.

Thanks for the logs. I'll try to recreate the errors on my end so that I can fix them properly.

About the loop, it's a background task trying to make sure everything is okay. It checks for new files and tries to extract/download missing data. About 5 rounds of doing the loop was enough to process everything. Some data can't be extracted/downloaded because it's unavailable, so the loop will run on forever.

If you are sure your library is processed, you can disable the bg task by passing the -nps (no periodic scans) flag when running the app.

docker run --name swingmusicv130betav2 -p 1970:1970 \
  -v /path/to/music:/music \
  -v /path/to/config:/config \
  --restart unless-stopped \
-  geoffrey45/swingmusic.v1.3.0.beta:volcs0_v2
+  geoffrey45/swingmusic.v1.3.0.beta:volcs0_v2 -nps

About the native mobile client, not sure unless I start making money from this app to pay for it. I'm thinking of ways to monetize this app, so that I can be able to offer the best experience. ie. the native mobile clients, personalized mixes based on listening activity, music discovery, etc. A lot of time (lots of time) and costs will be incurred to make this happen.

I'll start by trying a donations page on the website, if that doesn't work, I'll have go monthly subscription mode.

In the meantime, you can use the client on mobile as a Progressive web app. If you're using chrome, click the 3 dots on the top right corner and select Add to Home screen. Then open it from your home screen. It should be more stable than it was in the browser. Some features will be missing, look ugly or misbehave on the PWA.

I have spent a lot of time working on this project. It has to be fast. That's the idea. Thanks for the compliment.

Anyway, I'm also working on the docs on the website, be sure to check them out once v1.3.0 officially drops.

[volcs0]

This seems stable now. Almost everything has art, and when I add new music, it gets picked up. I don't see any ongoing jobs. Thanks for your help. I'm looking forward to future improvements and features and will definitely be a supporter.

And I still can't believe the speed of search. Amazing.

[cwilvx]

Hello @volcs0

The new release is out. You can pull the latest docker image from Github Packages (see Readme). If you'd like to donate, visit the support us page.

Changelog since previous beta (geoffrey45/swingmusic.v1.3.0.beta:volcs0_v2)

• Join EP & Singles in artist page
• Make artist page faster (removed 100+ lines of 💩 code)
• Make artist page with many albums faster (rewrite logic for calculating album dates to improve time complexity)
• Show newly created playlist in playlist list page
• TTY progress bars look a tiny bit consistent

Bug fixes

• Playlist with no tracks breaking playlist page
• Artist discography page not working
• Albums with no tracks missing from that album artist page
• Ed Sheeran tracks being in a single album (due to their album names being only one non-alpha-numeric character ... fuck Ed Sheeran) you might lose some favorites
• Some soundtrack albums being labeled as normal albums in album page
• Incorrect text balancing on playlist page header using a square image
• Merging albums not working