Documentation

[rishighan]

Hello, in researching for a nodejs-based OPDS server/feed generator, I came across this repo. Would it be possible to add documentation on how to spawn a server and generate a feed?

[bluelovers]

---

https://github.com/bluelovers/novel-opds-now/blob/master/lib/site/demonovel/opds.ts https://github.com/bluelovers/novel-opds-now/blob/master/server/opds/demonovel.ts https://github.com/bluelovers/novel-opds-now/blob/master/server/search.ts

---

https://github.com/bluelovers/ws-calibre/blob/master/packages/calibre-server/lib/opds/db.ts https://github.com/bluelovers/ws-calibre/blob/master/packages/calibre-server/lib/opds/index.ts

---

[rishighan]

Thanks, but how do I get this into my project? Is this package published to npm? Can I use it to generate a feed of cbz and cbr publications?

[bluelovers]

they all in npm

opds is xml, it can be let u feed any books

[rishighan]

I added ws-calibre through yarn add bluelovers/ws-calibre and yarn add bluelovers/novel-opds-now as dependencies to my project. How do I start the server though? Seems like it is a standalone server, with its own npm scripts.

Can you, maybe outline step-by-step, how to:

1. Pass my collection of comic books to the calibre server
2. Start the server and get the OPDS feed

[bluelovers]

import { Router } from 'express';
import { buildAsync } from 'calibre-opds';
import initMain from 'calibre-opds/lib/index';
import { async as FastGlob } from '@bluelovers/fast-glob/bluebird';
import { basename, extname } from 'path';
import { Entry, Feed } from 'opds-extra/lib/v1';
import { EnumLinkRel, EnumMIME } from 'opds-extra/lib/const';
import { Link } from 'opds-extra/lib/v1/core';
import express, { Express, Request, Response } from 'express';
import { AddressInfo } from 'net';

const app = express();

app.use(opdsRouter());

const _app = app.listen(function ()
{
    let address = _app.address() as AddressInfo;

    console.log(address)
});

export function opdsRouter()
{
    const router = Router();
    const path_of_books = __dirname;

    router.use('/opds', async (req, res, next) =>
    {
        return buildAsync(initMain({
            title: `title`,
            subtitle: `subtitle`,
            icon: '/favicon.ico',
        }), [

            async (feed: Feed) =>
            {

                feed.books = feed.books || [];

                await FastGlob([
                    '*.cbr',
                    '*.cbz',
                ], {
                    cwd: path_of_books,
                })
                    .each(file =>
                    {

                        let ext = extname(file)

                        let title = basename(file, ext);

                        /**
                         * make ur download url
                         */
                        let href: string;
                        /**
                         * mime for file
                         */
                        let type: string;

                        let entry = Entry.deserialize<Entry>({
                            title,
                            links: [
                                {
                                    rel: EnumLinkRel.ACQUISITION,
                                    href,
                                    type,
                                } as Link,
                            ],
                        })

                        feed.books.push(entry)
                    })
                ;

                return feed
            },

        ]).then(feed =>
        {

            res.setHeader('Content-Type', 'application/xml');
            return res.end(feed.toXML())
        })

    })

    return router
}

[rishighan]

Thanks so much for sharing this!

1. Got a bunch of tslint warnings on @bluelovers/fast-glob/bleubird.d.ts about Can't use default as a variable name I just changed the names in the file from the locally installed node_modules
2. Seems as though I can't get past feed.books.push(entry) Keeps saying error TS2532: Object is possibly 'undefined'

[bluelovers]

1

try disable that rule

2

u need do ur own

/**
                         * make ur download url
                         */
                        let href: string = xxxx;
                        /**
                         * mime for file
                         */
                        let type: string = xxxx;

[rishighan]

Thanks, I actually put an error boundary around it:

if (!isUndefined(feed) && !isUndefined(feed.books)) {
    feed.books.push(entry);
}

This worked, and feed.books is populated.

How do I pass comic book cover information to the entry?

[bluelovers]

https://github.com/bluelovers/ws-opds/blob/7f683bae3a6390abf8ac48b998baf9eaddd3547f/packages/opds-extra/lib/v1/schemas.ts#L111

[rishighan]

Thanks, I made a lot of progress! Do you have any pointers on how to get this working with a supported iOS/Android/macOS/Windows app?

I want to test the OPDS feed; so far I have tried doing it with Panels on iOS and it's not able to download the cbrs. I can't debug it. My guess is that the paths to my comics are wrong, I tried relative and absolute paths, even tried serving the comics as static assets and using those paths... no dice.

Have you had any success serving this to a consuming app on any platform?

[bluelovers]

https://github.com/bluelovers/ws-calibre/blob/73a3001d5426f1ff5e8a69dfc574e0f543a0b823/packages/calibre-server/lib/server.ts#L89

https://github.com/bluelovers/ws-calibre/blob/73a3001d5426f1ff5e8a69dfc574e0f543a0b823/packages/calibre-server/lib/static.ts#L18

https://github.com/bluelovers/novel-opds-now/blob/cc8efc1ac988cc0d306c447bae9c6f84d4178518/server/router/calibre/core.ts#L73

[bluelovers]

https://play.google.com/store/apps/details?id=com.flyersoft.moonreader

[rishighan]

https://github.com/bluelovers/ws-calibre/blob/73a3001d5426f1ff5e8a69dfc574e0f543a0b823/packages/calibre-server/lib/server.ts#L89

https://github.com/bluelovers/ws-calibre/blob/73a3001d5426f1ff5e8a69dfc574e0f543a0b823/packages/calibre-server/lib/static.ts#L18

https://github.com/bluelovers/novel-opds-now/blob/cc8efc1ac988cc0d306c447bae9c6f84d4178518/server/router/calibre/core.ts#L73

Should I do something different with my comic paths, I'm not sure I understand. Isn't 'calibre-opds' taking care of generating those paths for me?

[rishighan]

Is there a requirement for me to create a calibre database from my comic books or simply a folder of my comics will do?

[bluelovers]

Should I do something different with my comic paths

yes, u need make ur own download path, but his code is easy way

https://github.com/bluelovers/ws-calibre/blob/73a3001d5426f1ff5e8a69dfc574e0f543a0b823/packages/calibre-server/lib/server.ts#L89

http://expressjs.com/en/starter/static-files.html

app.use(`${p}`, staticExtra(row._fulldir, {
                extensions: [EnumDataFormatLowerCase.EPUB],
                index: false,
            }));

calibre database

calibre database from https://calibre-ebook.com/

[rishighan]

So if I understand correctly, and forgive me if I didn't earlier, I am supposed to:

1. Create static routes for the comics?
2. Create a Calibre database? This part has me really confused. My app already has its own internal model to represent comics. I store the comic metadata in mongo.
3. Calibre OPDS requires that the data be in the Calibre database, correct? How do I do that?

[bluelovers]

yes , or any other way make download path

https://github.com/bluelovers/novel-opds-now/blob/6e455ffdd7a602ec2fb0a22117d574c294bfe01f/server/router/calibre/core.ts#L73

2

u can just save ur comic info to json, no need use calibre-server

3

only calibre-server requires Calibre database

[rishighan]

Okay, couple of follow-up questions:

1. Do I need calibre-portable on my path? I am using macOS Big Sur.
2. Seems like dbList is initialized outside the route registration and depends on calibrePaths

const dbList = await buildLibraryList({
    calibrePaths,
    cwd: calibrePaths[0],
});

let calibrePaths: string | string[] = envCalibrePath(process.env);

What code do I actually need to build the paths?

[bluelovers]

1.

u can set by https://github.com/bluelovers/ws-calibre/blob/73a3001d5426f1ff5e8a69dfc574e0f543a0b823/packages/calibre-server/bin/calibre-server.ts#L21

2

calibrePaths is string[]

the path should be have

• calibrePath
• • xxx/metadata.db

it will auto search all library

[bluelovers]

[rishighan]

1. Ah, I don't think I can use calibre-portable then, since I am on macOS?
2. Can I do it with normal calibre installation on macOS?
3. Do I even need calibre?
4. Can I just use a normal static path, like here: https://github.com/rishighan/threetwo/blob/master/src/server/index.ts#L40 ?
5. I tried with that static path, in Panels (iOS) and the download always fails

[bluelovers]

1. 

2

yes

3

not really need, u just need make data for build opds feed

https://github.com/bluelovers/ws-opds/blob/7f683bae3a6390abf8ac48b998baf9eaddd3547f/packages/opds-extra/lib/v1/schemas.ts#L111

4

url for router url

u need make router that can parse url u make

5

same as 4, but u make wrong url

[rishighan]

A router that can parse the URL? Can you look at my code and create a gist or pastebin or something that can show me what I'm missing?

[bluelovers]

Index: src/server/index.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/server/index.ts b/src/server/index.ts
--- a/src/server/index.ts   (revision 51c0e120a76929ec22665a71d5f712f4b168c98c)
+++ b/src/server/index.ts   (date 1626682039017)
@@ -1,6 +1,6 @@
 import express, { Request, Response, Router, Express } from "express";
 import bodyParser from "body-parser";
-import path, { basename, extname } from "path";
+import { basename, extname, join } from "path";

 import { buildAsync } from "calibre-opds";
 import initMain from "calibre-opds/lib/index";
@@ -9,6 +9,9 @@
 import { Entry, Feed } from "opds-extra/lib/v1";
 import { Link } from "opds-extra/lib/v1/core";
 import { isUndefined } from "lodash";
+import { lookup } from 'mime-types';
+import { readFile } from 'fs';
+import { responseStream } from 'http-response-stream';

 // call express
 const app: Express = express(); // define our app using express
@@ -27,7 +30,7 @@
       [
         async (feed: Feed) => {
           feed.books = feed.books || [];
-          await FastGlob(["*.cbr", "*.cbz"], {
+          await FastGlob(["*.cbr", "*.cbz", "*.cb7", "*.cba", "*.cbt"], {
             cwd: path_of_books,
           }).each((file) => {
             const ext = extname(file);
@@ -37,13 +40,13 @@
              * make ur download url
              */
             const href = encodeURI(
-              `/Users/rishi/work/threetwo/src/server/comics/${file}`,
+              `/file/${file}`,
             );

             /**
              * mime for file
              */
-            const type = "application/octet-stream";
+            const type = lookup(ext) || "application/octet-stream";

             const entry = Entry.deserialize<Entry>({
               title,
@@ -72,6 +75,20 @@
     });
   });

+  router.use("/file/*", async (req, res) => {
+    const file: string = req.params[0];
+    const ext = extname(file);
+
+    if ([".cbr", ".cbz", ".cb7", ".cba", ".cbt"].includes(ext)) {
+      const content = await readFile(join(path_of_books, file))
+      const mime = lookup(ext) || "application/octet-stream";
+      res.set('Content-Type', mime);
+      return responseStream(res, content)
+    }
+
+    res.status(404).end(`'${file}' not exists`)
+  })
+
   return router;
 }

unnamed.zip

[rishighan]

Thanks it worked! Let me know if you want me to add documentation to the module.

[bluelovers]

Thanks it worked! Let me know if you want me to add documentation to the module.

0.0 it will be good, im bad for doc