Incorrect images

[fuelen]

I imported a CSV file with my last.fm data. And it seems like for some songs/artists I receive incorrect images Here is a screenshot:

All images on the screenshot are totally incorrect, they are from different artists. For some tracks, images are absent at all.

Tracks from the screen in text:

#1 |   Злам – Мій дім. Моя земля.
#2 |   Злам – Забути й стерти
#3 |   Злам – Судна зi дна
#4 |   Злам – Дорогою слiпих

related issue: https://github.com/krateng/maloja/issues/224

[krateng]

Does this only happen for artists in Cyrillic? Do you have any Latin ones to compare?

[fuelen]

I haven't noticed this with Latin ones. It is worth to note, that not all Cyrillic artists have this issue.

[SirMartin]

I am getting also wrong images right now in the #next-minor-version.

I have 2 builds of Maloja, I was using until now for some weeks the 3.1.5 version from Dockerhub, and I was trying to import the Spotify history, so I created a new one from #next-minor-version branch.

But I found, both using the same data, that the new one, is taking different images, and for some are wrong ones.

The 3.1.5

The #new-minor-version one

The failing artists, are latin characters, nothing cyrillic here.

[krateng]

Which image providers are you using? Also make sure to remove the imagecache database and restart the server.

[SirMartin]

I have setted Spotify in both, same API key

[krateng]

Alright I think I might have found the error, let me know if it works.

[SirMartin]

Alright I think I might have found the error, let me know if it works.

It works correctly. Great !!!

[SirMartin]

But wait a moment, it works correctly for artists, but as far as I can see, it makes still weird things for songs

As you can see there is some albums that have not found a cover, but in Spotify it found it.

The spotify version.

[krateng]

That's just a limitation of the Spotify search I'm afraid. In this case for example the API returns the correct result when ordering the artists one way, but not when ordering them the other way.

[SirMartin]

Ok, I am trying to mount another docker with the next_minor_version of Maloja plus another multiscrobbler, to try to scrobble songs to both Malojas, and see. Because for now, the old one, shows good images, and the _next_minorversions is showing wrong covers sometimes.

[SirMartin]

Ok, I have working now, and scrobbling same Spotify account to tow different multiscrobblers sending to two different versions of Maloja, one the "official" and the other one, from this morning of "next_minor_version", and the images pulled are different for artists.

Some of the artists, are different version, but at least is the right artist, but others are wrong in the new version but correct in the "production" one.

This is the production version.

This is the _next_minorversion

[krateng]

Which ones are incorrect?

[SirMartin]

For example the man with the cowboy hat, is not Ede. Ede is a girl.

Here is from her Spotify page.

[krateng]

Seems like this is just the first result Spotify returns. Not sure why you don't get the same in the old version, maybe that's the Deezer fallback?

[SirMartin]

Not sure what it is, but I can see some issues with images between both versions. Now after I played some music and get more songs scrobbled, some of the images in the new version are not appearing, but in fact other images are more correctly in the new version than in the old one.

Is any way to clean all the existing metadata for images, and make maloja start getting everything again? Or is too hard?

Here are the images that disappeared, and for example, the one with the green tick, is right here, and in the production one, just shows the album picture.

Also, I am getting the feeling that the new one is much slower than the "production" one, is something local from mine, or still needs improvement. For example, if I search for an artist or even just refreshing the main page, it takes a lot more time, even some timeouts in the new one.

[krateng]

If there's no image at all, you just need to give it more time to fetch it - it shows a placeholder if it actually hasn't found an image. If you wanna refresh everything, delete cache/imagecache.sqlite

[SirMartin]

Ok, I will try later, so can start both containers to get images freshly and see what happens

[SirMartin]

I removed the cache db from both containers and let's see how long it takes, but for now, the old version is much much faster to load new images than the other one. I am getting error 500 all around, for each image that cannot find, is that normal?

And in the same amount of time that is the difference, and again, is taking some different results for some images.

New Version

Old Version

PS: I don't see that the cache database is created again.

[krateng]

Yes, loading new images should be slower than before; and the images simply being different from the old version isn't a problem unless they are actually wrong.

As for the 500 status, can you post the error that the server process throws?

[SirMartin]

I think the error is this.

In old version:

[SQL: INSERT INTO artists (id, url, expire, raw) VALUES (?, ?, ?, ?)] [parameters: ('6739', 'https://e-cdns-images.dzcdn.net/images/artist/8250a712cd8fc551737ae2421ca5c270/250x250-000000-80-0-0.jpg', 1703342830, DataURI('data:image/jpeg;charset=ascii;base64,/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAYEBQYFBAYGBQYHBwYIChAKCgkJChQODwwQFxQYGBcUFhYaHSUfGhsjHBYWICwgIyYnKSop ... (15468 characters truncated) ... hgi9uA3wV+ogoKRpOv8ACvWoejaSqdhl/FvxAYcoPsPzcFEtxqbfJDFCyrIuMQyRVnq4gus+JhuLue2GyGyG/Q3jqcTBDdv1LVVVXKs4j/hzGHoIAFJR0ktTjR8vRrBlHU0I+hd5xjhOPT//2Q=='))] (Background on this error at: https://sqlalche.me/e/20/e3q8) [images] Downloaded https://e-cdns-images.dzcdn.net/images/artist/06276fa064dac7b61f191fdebe56c594/250x250-000000-80-0-0.jpg for local caching Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1964, in _exec_single_context self.dialect.do_execute( File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 747, in do_execute cursor.execute(statement, parameters) sqlite3.OperationalError: attempt to write a readonly database The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/bin/bottle.py", line 876, in _handle return route.call(**args) File "/usr/bin/bottle.py", line 1759, in wrapper rv = callback(*a, **ka) File "/usr/lib/python3.9/site-packages/maloja/server.py", line 126, in dynamic_image result = resolve_artist_image(keys['id']) File "/usr/lib/python3.9/site-packages/maloja/images.py", line 174, in resolve_artist_image set_image_in_cache(artist_id,'artists',result['value']) File "/usr/lib/python3.9/site-packages/maloja/images.py", line 77, in set_image_in_cache result = conn.execute(op) File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1414, in execute return meth( File "/usr/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 485, in _execute_on_connection return connection._execute_clauseelement( File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1638, in _execute_clauseelement ret = self._execute_context( File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1842, in _execute_context return self._exec_single_context( File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1983, in _exec_single_context self._handle_dbapi_exception( File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2325, in _handle_dbapi_exception raise sqlalchemy_exception.with_traceback(exc_info[2]) from e File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1964, in _exec_single_context self.dialect.do_execute( File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 747, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) attempt to write a readonly database [SQL: INSERT INTO artists (id, url, expire, raw) VALUES (?, ?, ?, ?)] [parameters: ('6737', 'https://e-cdns-images.dzcdn.net/images/artist/06276fa064dac7b61f191fdebe56c594/250x250-000000-80-0-0.jpg', 1703342830, DataURI('data:image/jpeg;charset=ascii;base64,/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAYEBQYFBAYGBQYHBwYIChAKCgkJChQODwwQFxQYGBcUFhYaHSUfGhsjHBYWICwgIyYnKSop ... (9240 characters truncated) ... 7i5xFdRcQY2xaDqBlI4lTcSeJozxAZxCdEBeiA6IS9EJeiA6IHRCtwQM4IHRAzggZwfaA0GoXMPtAFeMcQw1CR3EehrqdQ+0saPtKGj7SlmD7QNBAvRvqUvR9pS2DcBTBzAvRAK0RDogFGCf/9k='))] (Background on this error at: https://sqlalche.me/e/20/e3q8)

In new version I am getting these:

[parameters: (50, 1698158941)] (Background on this error at: https://sqlalche.me/e/20/e3q8) Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context self.dialect.do_execute( File "/usr/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute cursor.execute(statement, parameters) sqlite3.OperationalError: no such table: albums The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/bottle.py", line 876, in _handle return route.call(**args) File "/usr/lib/python3.10/site-packages/bottle.py", line 1759, in wrapper rv = callback(*a, **ka) File "/usr/lib/python3.10/site-packages/maloja/server.py", line 122, in dynamic_image result = image_request(**{k:int(keys[k]) for k in keys}) File "/usr/lib/python3.10/site-packages/maloja/images.py", line 283, in image_request result = get_image_from_cache(artist_id=artist_id,track_id=track_id,album_id=album_id) File "/usr/lib/python3.10/site-packages/maloja/images.py", line 88, in get_image_from_cache result = conn.execute(op).all() File "/usr/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1416, in execute return meth( File "/usr/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 516, in _execute_on_connection return connection._execute_clauseelement( File "/usr/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1639, in _execute_clauseelement ret = self._execute_context( File "/usr/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1848, in _execute_context return self._exec_single_context( File "/usr/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1988, in _exec_single_context self._handle_dbapi_exception( File "/usr/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2343, in _handle_dbapi_exception raise sqlalchemy_exception.with_traceback(exc_info[2]) from e File "/usr/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context self.dialect.do_execute( File "/usr/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: albums [SQL: SELECT albums.id, albums.url, albums.expire, albums.local, albums.localproxyurl  FROM albums WHERE albums.id = ? AND albums.expire > ?]

Not sure the one from the new version is right, I got from the Docker error log, if its another specific error log, tell me and I will check.

About if they are right or not, not sure, some images in both versions are not the best ones, but ...

[krateng]

Can you run the latest version and see if this still happens?

[SirMartin]

Great job, it works perfectly. And works really fast. In the first load shows all the artist, it took some extra refreshes for tracks and albums. The only thing, but I can imagine is related with some issues with Spotify API, is that gets some 503 errors, until all the images were loaded.

Also the performance is amazing, even better than the "production" one. Great work!

Do you have an estimated date, to release the version 3.2?

[krateng]

Nice!

Probably gonna release it by end of the week or early next week if no complicated issues pop up, but no promises.

@fuelen does this work for you as well?

[fuelen]

Unfortunately, I cannot check it right now from the source, so I'll wait for the release :) Feel free to close the issue, it can be reopened later if needed.

[fuelen]

I've updated to the latest version from dockerhub. It is much-much better now, but not perfect :slightly_smiling_face: For an artist from my first screenshot, artist image is not valid. Last.fm page of the band https://www.last.fm/music/%D0%97%D0%BB%D0%B0%D0%BC for reference I found this line in logs:

Got artist image Злам from Deezer