jellyfin / jellyfin

The Free Software Media System
https://jellyfin.org
GNU General Public License v2.0
33.73k stars 3.09k forks source link

With ID3v1.1 tagged MP3, Album titles longer than the ID3v1.1 max 30 bytes are not identified by the scanner. #12720

Closed prahal closed 4 days ago

prahal commented 5 days ago

This issue respects the following points:

Description of the bug

I have an album whose mp3 I bought long ago are tagged with ID3v1.1 and the album title is longer than 30 bytes thus truncated in the ID3v1.1 Album field to "Artificial Animals Riding On N" instead of its full title "Artificial Animals Riding On Neverland". The issue is the metadata scanner takes for granted that the Album name it gets from the ID3v1.1 Album tag is the full title and even though the mp3 is in a directory named "Artificial Animals Riding On Neverland (2007)" it set the Jellyfin album title as "Artificial Animals Riding On N" from the ID3v1.1.

Could Jellyfin metadata extractor check the parent folder name if structured as "Artist/Album/xx - song.mp3" to see the album title could have been truncated for ID3v1.1 tags? (or any that potentially truncate fields) Ie from "Aaron/Artificial Animals Riding On Neverland (2007)/".

In fact logs show: [20:21:23] [ERR] [66] MediaBrowser.Providers.Manager.ProviderManager: Error refreshing item System.InvalidOperationException: The data is NULL at ordinal 1. This method can't be called on NULL values. Check using IsDBNull before calling.

Still this bug report should focus on the jellyfin metadata keeping only the truncated album name as when I edit jellyfin metadata to the full album name and then "Refresh metadatas" > "Replace all metadata" the truncated version repalce my fixed version (even though the updating of the metadata triggers the Invalid OperationException). So I prefer if this bug deal with the keeping of the truncated album name instead of the exception, as its seems the metadata manages to update even with the exception.

Sidenote: I am also interested in the ordinal exception but will try to open a bug report for that. This issue returns errors 500 to Kodi jellyfin plugin which itself hang the jellyfin libraries update, but it is another issue. I will try to open another bug report for this exception. But it might be that once the album title is complete and metadata grabbed from scrapers the exception will be gone (it would still deserve to get fixed but would only be a bug in the error path).

So both issues might be related but this bug report is about Jellyfin giving too high implortance to ID3v1.1 when the folder name is more correct. Maybe ID3v1.1 fields could be not trusted when the max field bytes is reached (it would tag the title which are exactly 30 bytes as unreliable but that is I believe ok).

About the Ordinal Exception I already tried the FixAudioDB hack in migrations.xml

Reproduction steps

  1. Add an mp3 file with an ID3v1.1 field which album title is longer than 30 bytes
  2. Let Jellyfin scan its metadata or trigger a "Update metadata" > "Replace all metadatas"
  3. check the jellyfin album page generated with the truncated album title and without any scraped metadata due to truncated title

What is the current bug behavior?

No metadata is grabbed from scanners and thus it only show the bare ID3v1.1 metadata from the mp3 and no pictures.

What is the expected correct behavior?

Show the full album title and grab pictures and all details from external sources

Jellyfin Server version

10.9.11+

Specify commit id

No response

Specify unstable release number

No response

Specify version number

No response

Specify the build version

10.9.11

Environment

- OS: Debian 12
- Linux Kernel: 6.9.3
- Virtualization: Docker
- Clients: Browser
- Browser: Firefox 130.0.1
- FFmpeg Version: 6.0.1-8-bookworm
- Playback Method: unrelated to the issue
- Hardware Acceleration: none
- GPU Model: Mali T860
- Plugins: MusicBrainz, Bookshelf, OPDS feeds, Cover Art Archive, AudioDB, DLNA, IMVDB, KodiSyncQueue, Local Intros, NextPVR, OMDb, Open Subtitles, Playback Reporting, Reports, Studio Images, TMDb, TheTVDB, VGMdb, YoutubeMetadata
- Reverse Proxy: none
- Base URL: https://helios64.prahal.homelinux.net:8920/
- Networking: Host
- Storage: local

Jellyfin logs

logs when I click "Refresh Metadatas" > "Replace all metadatas"

[20:21:16] [INF] [50] Microsoft.AspNetCore.Hosting.Diagnostics: Request starting HTTP/2 GET https://helios64.prahal.homelinux.net:8920/Users/f0f250cf42a24340b5634087be9de6ad/Items/10738bcfba0d5c5f319bd98942b75ffc - null null
[20:21:16] [INF] [50] Microsoft.AspNetCore.Routing.EndpointMiddleware: Executing endpoint 'Jellyfin.Api.Controllers.UserLibraryController.GetItemLegacy (Jellyfin.Api)'
[20:21:16] [INF] [50] Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker: Route matched with {action = "GetItemLegacy", controller = "UserLibrary"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.ActionResult`1[MediaBrowser.Model.Dto.BaseItemDto]] GetItemLegacy(System.Guid, System.Guid) on controller Jellyfin.Api.Controllers.UserLibraryController (Jellyfin.Api).
[20:21:16] [INF] [50] Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor: Executing ObjectResult, writing value of type 'MediaBrowser.Model.Dto.BaseItemDto'.
[20:21:16] [INF] [50] Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker: Executed action Jellyfin.Api.Controllers.UserLibraryController.GetItemLegacy (Jellyfin.Api) in 7.8942ms
[20:21:16] [INF] [50] Microsoft.AspNetCore.Routing.EndpointMiddleware: Executed endpoint 'Jellyfin.Api.Controllers.UserLibraryController.GetItemLegacy (Jellyfin.Api)'
[20:21:16] [INF] [50] Microsoft.AspNetCore.Hosting.Diagnostics: Request finished HTTP/2 GET https://helios64.prahal.homelinux.net:8920/Users/f0f250cf42a24340b5634087be9de6ad/Items/10738bcfba0d5c5f319bd98942b75ffc - 200 null application/json; charset=utf-8 10.3877ms
[20:21:23] [INF] [68] Microsoft.AspNetCore.Hosting.Diagnostics: Request starting HTTP/2 POST https://helios64.prahal.homelinux.net:8920/Items/10738bcfba0d5c5f319bd98942b75ffc/Refresh?Recursive=true&ImageRefreshMode=FullRefresh&MetadataRefreshMode=FullRefresh&ReplaceAllImages=false&ReplaceAllMetadata=true - null 0
[20:21:23] [INF] [68] Microsoft.AspNetCore.Cors.Infrastructure.CorsService: CORS policy execution successful.
[20:21:23] [INF] [68] Microsoft.AspNetCore.Routing.EndpointMiddleware: Executing endpoint 'Jellyfin.Api.Controllers.ItemRefreshController.RefreshItem (Jellyfin.Api)'
[20:21:23] [INF] [68] Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker: Route matched with {action = "RefreshItem", controller = "ItemRefresh"}. Executing controller action with signature Microsoft.AspNetCore.Mvc.ActionResult RefreshItem(System.Guid, MediaBrowser.Controller.Providers.MetadataRefreshMode, MediaBrowser.Controller.Providers.MetadataRefreshMode, Boolean, Boolean) on controller Jellyfin.Api.Controllers.ItemRefreshController (Jellyfin.Api).
[20:21:23] [INF] [68] Microsoft.AspNetCore.Mvc.StatusCodeResult: Executing StatusCodeResult, setting HTTP status code 204
[20:21:23] [INF] [68] Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker: Executed action Jellyfin.Api.Controllers.ItemRefreshController.RefreshItem (Jellyfin.Api) in 1.3119ms
[20:21:23] [INF] [68] Microsoft.AspNetCore.Routing.EndpointMiddleware: Executed endpoint 'Jellyfin.Api.Controllers.ItemRefreshController.RefreshItem (Jellyfin.Api)'
[20:21:23] [INF] [68] Microsoft.AspNetCore.Hosting.Diagnostics: Request finished HTTP/2 POST https://helios64.prahal.homelinux.net:8920/Items/10738bcfba0d5c5f319bd98942b75ffc/Refresh?Recursive=true&ImageRefreshMode=FullRefresh&MetadataRefreshMode=FullRefresh&ReplaceAllImages=false&ReplaceAllMetadata=true - 204 null null 5.3399ms
[20:21:23] [ERR] [66] MediaBrowser.Providers.Manager.ProviderManager: Error refreshing item
System.InvalidOperationException: The data is NULL at ordinal 1. This method can't be called on NULL values. Check using IsDBNull before calling.
   at Microsoft.Data.Sqlite.SqliteDataRecord.GetNull[T](Int32 ordinal)
   at Microsoft.Data.Sqlite.SqliteValueReader.GetBlob(Int32 ordinal)
   at Microsoft.Data.Sqlite.SqliteDataRecord.GetCachedBlob(Int32 ordinal)
   at Microsoft.Data.Sqlite.SqliteDataRecord.GetStream(Int32 ordinal)
   at Emby.Server.Implementations.Data.SqliteItemRepository.GetItem(SqliteDataReader reader, InternalItemsQuery query, Boolean enableProgramAttributes, Boolean hasEpisodeAttributes, Boolean hasServiceName, Boolean queryHasStartDate, Boolean hasTrailerTypes, Boolean hasArtistFields, Boolean hasSeriesFields, Boolean skipDeserialization)
   at Emby.Server.Implementations.Data.SqliteItemRepository.GetItemList(InternalItemsQuery query)
   at MediaBrowser.Controller.Entities.Folder.AddChildrenToList(Dictionary`2 result, Boolean includeLinkedChildren, Boolean recursive, Func`2 filter)
   at MediaBrowser.Controller.Entities.Folder.GetRecursiveChildren(Func`2 filter, Boolean includeLinkedChildren)
   at MediaBrowser.Controller.Entities.Folder.GetRecursiveChildren(Func`2 filter)
   at MediaBrowser.Controller.Entities.Audio.MusicAlbum.GetLookupInfo()
   at MediaBrowser.Providers.Manager.MetadataService`2.RefreshMetadata(BaseItem item, MetadataRefreshOptions refreshOptions, CancellationToken cancellationToken)
   at MediaBrowser.Controller.Entities.BaseItem.RefreshMetadata(MetadataRefreshOptions options, CancellationToken cancellationToken)
   at MediaBrowser.Providers.Manager.ProviderManager.RefreshItem(BaseItem item, MetadataRefreshOptions options, CancellationToken cancellationToken)
   at MediaBrowser.Providers.Manager.ProviderManager.StartProcessingRefreshQueue()

FFmpeg logs

No response

Client / Browser logs

No response

Relevant screenshots or videos

No response

Additional information

No response

gnattu commented 5 days ago

I don't think this is jellyfin's problem in this case AND I think user should fix their tagging if it violates the spec in such way. I don't even think music tagger should allow such tag to exist because it will likely cause undefined behavior for a lot of cases.

prahal commented 5 days ago

@gnattu by "fix the tagging" do you mean that the user should migrate the tagging to a higher ID3 tag version that supports more bytes for long album titles? I mean the fact the title does not fit into the max 30 bytes Album field of the ID3v1, v1.1 spec does not violate the spec. Please clarify, I don't understand your reply. "Artificial Animals Riding On N"is 30 bytes and thus the title cannot be written in its entirety. Sorry if my bug report was not clear.

Or maybe jellyfin could stop parsing ID3v1 and ID3v1.1 as they don't support medium sized album titles. This is music that I bought around 2006.

gnattu commented 5 days ago

It is not hard to re-tag your music with a higher version ID3 tag isn't it?

felix920506 commented 4 days ago

Closing as won't fix since your media tags do not conform to the proper specification of that format. Please use a newer version of ID3 that supports longer names.

prahal commented 2 days ago

@felix920506 I disagree that this bug should be closed. If Jellyfin does not support ID3v1.1 tags, then... Jellyfin should not grab data from these obsolete tag standard. If Jellyfin was getting the album title from the parent directory name and not from the ID3v1.1 tags, then all would be fine.

If Jellyfin does not support ID3v1.1 tag because of too old, then it should not grab data from these ID3v1.1 tags but only from newer standards. There is still a bug.

@gnattu these tags are not from me, but from a music shop when I bought these titles around 2006 (I believe back then ID3v1.1 was not obsolete). I believe I will end up tagging them to a newer format. But I could not know before felix told me that Jellyfin did not support these old tag format. Also, I believe Jellyfin should not parse the tag format it does not support. That is, if Jellyfin does not support ID3v1.1 then it should not parse them. Then it might even get the data from the folder names which is correct, and thus I should get the metadata without tagging with a newer format.

Please reopen. If only to retitle "Jellyfin should not parse ID3v1.1 tag it does not support"

PS: it is not hard to re-tag a few files. It is harder to check every music files I own to see if they have new enough tag a format. I have no such tool to parse a library for old tag format.

gnattu commented 2 days ago

It’s not a Jellyfin issue, nor is it about Jellyfin not supporting ID3v1. Jellyfin does support ID3v1—it's just that ID3v1 doesn't support titles that long. I hope you can see the difference between these two points. Additionally, we are not going to parse albums from folder names. Music files contain all necessary metadata within the file, and adjusting the parsing logic to accommodate this specific case would disrupt the experience for many other users. What would we say to users asking why album titles aren't pulled from ID3v1 tags anymore? Because it "works for that specific user"?

prahal commented 1 day ago

@gnattu you reply to my reply to @felix920506 about:

Closing as won't fix since your media tags do not conform to the proper specification of that format. Please use a newer version of ID3 that supports longer names. I meant that "my media tags do conform to the proper specification of that format". That if ID3v1.1 tags are not mean to be supported for album titles more than 30 bytes then Jellyfin should not parse these tags at all instead of getting data from them and then telling that is not a jellyfin issue because this ID3v1.1 media tags are not supported.

But could be you did not see @felix920506 reply above I was replying to.

I am not telling that ID3v1.1 tags are not supported by Jellyfin. I am telling that if as @felix920506 tells they are not supported by Jellyfin and thus no workaround for album titles longer than supported by ID3v1.1 tags would be accepted because this specification is not supported, then Jellyfin should not parse these ID3v1.1 tags. I know very well that Jellyfin supports ID3v1.1 tags and that ID3v1.1 do not support album titles longer than 30 bytes as I explained in my initial post.

I asked if Jellyfin supports ID3v1.1 if a workaround for title 30 bytes or more could be added and if as @felix920506 tells they are not supported thus not workaround would be accepted, that ID3v1.1 tag should not be parsed.

we are not going to parse albums from folder names. Music files contain all necessary metadata within the file, and adjusting the parsing logic to accommodate this specific case would disrupt the experience for many other users. I thought that was already supported, as is done by movies. Thank you for the clue.

So the workaround would then be to not import tags longer or equal than 30 bytes as they are likely invalid and Jellyfin will be unable to do anything meaningful with them and log that in the Jellyfin logs. Better not import anything than truncated or inconsistent data in my opinion.

What would we say to users asking why album titles aren't pulled from ID3v1 tags anymore? Because it "works for that specific user"? I initially asked for a workaround for title "equal or longer than 30 bytes" which are already truncated. This would not break anything that is already broken and for which Jellyfin already is unable to grab any external metadata. But felix told me it would not be accepted: your media tags do not conform to the proper specification of that format. Please use a newer version of ID3 that supports longer names. Though I admit it is not clear to me what is meant by "your media tag do no conform to the proper specification of that format". Truncating titles is part of the ID3v1.1 specification. I mean there were no better specification when these medi files were made (these are the vendor media tags)

I was asking not to parse ID3v1.1 if they are not meant to be supported as @felix920506 told me. If they are meant to be supported then my initial request was in now way to stop parsing them but to add a workaround for titles that are too equals or longer than the max size of the field in the specification. Which I am still asking for. I am only asking ID3v1.1 is told not to be supported for Jellyfin to stop parsing ID3v1.1.

gnattu commented 1 day ago

How could we know if the field is truncated and not exact 30 bytes long? What you are asking for does not make any sense.

prahal commented 1 day ago

We could consider do so even if the title is only 30 bytes (and consider that if there are 30 bytes then either the title is 30 bytes or more). That would make Jellyfin not support exactly 30 bytes titles but I see no better way. Though I don't have statistics to know if there are a log of exactly 30 bytes long titles.

There is another option. Maybe data provider could support the 30 bytes truncated titles as I believe most truncated titles are still unique. Then the issue would be gone. But in my opinion my initial request is valid. THat is to have Jellyfin gracefully handle ID3v1.1 truncated titles, either by warning the user to upgrade to a newer standard and de-truncate his tags or by any other mean.

I mean it took me quite a while to discover why my media external data was not grabbed. Jellyfin should not request for user to find out for themselves that their media tags are truncated, that this is due to the ID3v1.1 standard and that they should upgrade to a newer standard. This is pretty high a bar. Jellyfin should warn the user each time he requests for these ID3v1.1/iD3v1 music files to be parsed for metadata. So at least reading the logs tells what is wrong. I mean I have thousands of music files spanning 20 years, in fact many do not get covers or external data via Jellyfin. But I believe for many different reasons. That would help a lot to get hints as to why in Jellyfin logs (and maybe in the UI, ie by not importing tags if they are likely broken for Jellyfin use, or best would be for data providers to support 30 bytes truncated titles even if there maybe be title conflicts, but unlikely with the same artist, same year, etc

prahal commented 1 day ago

Or even better if there was a way to guess the non-truncated title by a request, with artist, year, truncated title, etc. to the data providers and replace the Jellyfin title with this one while also adding covers and external data. Though that might be a very low priority, I still believe this is a valid bug report.

I searched and found that MusicBrainz LuceneSearch wildcard could not be used for phrases like "Artificial Animals Riding On N*", but the opposite ie truncate further do work (though I don't know if Jellyfin could truncate the last potentially truncated word),

"Artificial Animals Riding On N" returns nothing https://musicbrainz.org/search?query=%22Artificial+Animals+Riding+On+N%22&type=recording&limit=25&method=indexed but "Artificial Animals Riding On" works and returns the same results as "Artificial Animals Riding On Neverland" https://musicbrainz.org/search?query=%22Artificial+Animals+Riding+On+N%22&type=recording&limit=25&method=indexed

This is really not top priority to support truncated titles by truncating further, these are just of interest data.

Having a way to tell that the title is truncated in the media file tags would be great. Be it by a warning in the log or in the UI that the title is 30 bytes for an ID3v1.1 or below ID3 tag and thus is likely truncated still.

Thanks a lot for the input, I really thought Jellyfin was parsing the directory hierarchy already, as it does for movies. If this is a wontfix then be it.

gnattu commented 1 day ago

Having a way to tell that the title is truncated in the media file tags would be great. Be it by a warning in the log or in the UI that the title is 30 bytes for an ID3v1.1 or below ID3 tag and thus is likely truncated still.

This is added in linked PR.