Open discussion around various metadata containers

[gotson]

Following discussions in #82, this issue is to gather information about the different metadata containers, along with their limitations.

This issue is not about which metadata container to use in Komga, which has its own internal model stored in database. It's a reflexion and thought process about the overall universe of metadata for comics.

Having a single standard would obviously be beneficial, with some caveats:

• The famous xkcd comic about standards
• Even with the best standard, adoption remains key. Not being able to convince people to use it would mean a useless standard.

Nevertheless, it doesn't hurt to summarize the market offerings along with their limitations and adoption.

---

Existing formats:

• ComicRack (known as ComicInfo.xml)
  • Status: part of ComicRack, which is not developed anymore. The single developer, cYo, disappeared from the internet. ComicRack is/was also a shareware model, which could limit the steering capability of the format.
  • Format: XML, usually stored inside the archive at the root
  • There is a XML schema available, but there is no versioning (and at least 2 versions of the schema) of the schema, nor usage of XML namespaces
  • Limitations:
  • Some fields are missing or limited. Example:
    • There is a StoryArc field, but no field to indicate the ordering of the book in this StoryArc. It is also impossible to specify more than one StoryArc per book.
• CoMet
  • Status: unknown. Seems to be a one-man project without actual usage in applications. Not updated since 2015.
  • Format: XML, apparently stored inside the archive at the root
  • There should be a XML schema available, but I can't find it. There is also no versioning of the schema, but there are XML namespaces defined.
  • Limitations:
  • Some fields are missing or limited. Example:
    • No difference between Series and Story Arc
• ACBF
  • Status: has no history on their wiki since 2018, but there is a commit from Dec 2019 in their repo. Not sure who uses this format either.
  • Format: XML
  • There is a XML schema available for download, it is versioned and uses XML namespaces.
  • Limitations: no yet evaluated
• OPF
  • Status: industry standard used in ebooks. It relies on the Dublin Core Metadata Initiative for the metadata fields.
  • Format: XML, with schema, versioned, using namespaces.
  • Limitations: not yet evaluated

[jevcleem]

I've been reading about metadata being stored in the zip comment (Source: https://wiki.mobileread.com/wiki/CBR_and_CBZ#Metadata). Apparently it's not a common practice but this would have some benefits imo. The zip comment is an uncompressed section of the file such that unzipping is not necessary to change it. I think writing a json object to the comment section of the zip file would be easy to implement and computationally efficient. Positives:

• Easy to implement
• Efficient and fast (due to unzipping not necessary)

Negatives:

• Additional dependecy necessary for writing/reading to zip comment
• New standard
• Metadata lost when unzipping and zipping again
• Not possible if not a zip file, e.g. epubs/pdf

Other suggestions:

• I think if the xml route is chosen, it would be best to store it outside the zip file.
• Maybe write the software in such a way that the metadata implementation can be a server setting.

[gotson]

Thanks for the input. You are mentioning 2 things here:

• the location of data, the zip comment, which could actually hold any of the XML mentionned above.
• the serialization format, json. Actually JSON is poorly equipped compared to XML for schema definition and validation. They do achieve the same purpose though : serialization.

Those 2 aspects are important, but to me the most important one is about the content, or semantics, ie what can you represent with those formats. 2 mentioned above already are not exhaustive enough to represent the information accurately.

I think I should start to gather data about what a complete model would look like for comics, and compare it to the existing formats to see the gaps.

[theotocopulitos]

I've read other threads on this topic, and know about your thoughts regarding comicinfo.xml . But having been around this world even before Comicrack became a reality, I was the author of the first comicvine scrapper implented in Comicrack, I would like to add some thoughts for you to ponder:

• comicinfo.xml is, no doubt, incomplete, badly implemented and not documented...

• however, it is the 'de-facto' standard... almost every comic related project reads it and many write it (namely, comictagger, comixed, ubooquity, mylar...)

• thousands of users used this protocol: convert cbr to cbz, scrape from CV, and embed comicinfo in the cbz...

• this allows for data portability when you switch computer, reader, organizer...

Many efforts were made to find a better replacement for comicinfo.xml, so far, without any success. I guess not all of them were backed by a code as functional and polished as yours,,, So this might be the place to tackle such endeavor...

So, I get your points: comicinfo is far from optimum, you are exposing an API that would allow to get the info to anyone and deal it however we like... but in the interim, while a better option is created, I feel full support to read/write comicinfo would be most welcome by the community,

[gotson]

This issue is not much to talk about reading or writing from those formats, but more to discuss the formats themselves.

As for comicinfo.xml, Komga can already read it (#54). Writing is logged under #82.

[gotson]

Closing in favor of https://github.com/anansi-project/rfcs