audnexus

An audiobook data aggregation API, combining multiple sources of data into one, consistent source.

📝 Table of Contents

• About
• Getting Started
• Deployment
• Usage
• Built Using
• TODO
• Contributing
• Authors
• Acknowledgments

🧐 About

Nexus - noun: a connection or series of connections linking two or more things.

Looking around for audiobook metadata, we realized there's no solid (or open) single source of truth. Further, some solutions had community curated data, only to close their API. As such, this project has been created to enable development to include multiple sources of audiobook content in one response.

This project also makes integration into existing media servers very streamlined. Since all data can be returned with 1-2 API calls, there's little to no overhead processing on the client side. This enables rapid development of stable client plugins. Audnexus serves as a provider during the interim of waiting for a community driven audiobook database, at which time audnexus will be a seeder for such a database.

🏁 Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Prerequisites

• There are 2 ways to deploy this project - for the purposes of this project, this guide will only cover Docker deployment:
  • Docker Swarm
  • Directly, via pnpm run or pm2
  • Mongo 4 or greater
  • Node/NPM 16 or greater
  • Redis
  • Registered Audible device keys, ADP_TOKEN and PRIVATE_KEY, for chapters. You will need Python and audible for this. More on that here

Installing locally

• Install Mongo, Node and Redis on your system
• pnpm install from project directory to get dependencies
• Set ADP_TOKEN and PRIVATE_KEY environment variables as mentioned above if you are using the chapters endpoint.
• pnpm run watch-debug to start the server

Test an API call with

http://localhost:3000/books/${ASIN}

🔧 Running the tests

Tests for this project use the Jest framework. Tests can be done locally in a dev environment:

• pnpm test

After the tests have run, you may also browse the test coverage. This is generated in coverage/lcov-report/index.html under the project directory.

🎈 Usage

API usage documentation can be read here: https://audnex.us/

Pre-rendered HTML documentation is also included in docs/index.html.

HTML can be re-generated from the spec, using:

pnpm run build-docs

🚀 Deployment

Once you have Docker Swarm setup, grab the docker-compose.yml from this repo, and use it to start the stack. Using something like Portainer for a Swarm GUI will make this much easier.

The stack defaults to 15 replicas for the node-server container. Customize this as needed.

Environment variables to add:

• NODE_ADP_TOKEN: Aforementioned ADP_TOKEN value
• NODE_MAX_REQUESTS: Maximum amount of requests per 1 minute period from a single source (default 100)
• NODE_MONGODB_URI: MongoDB connection URL, such as mongodb://mongo/audnexus
• NODE_PRIVATE_KEY: Aforementioned PRIVATE_KEY value
• NODE_REDIS_URL: Redis connection URL, such as redis://redis:6379
• NODE_UPDATE_INTERVAL: Frequency (in days) to run scheduled update tasks (default 30). Update task is also run at startup.
• NODE_UPDATE_THRESHOLD: Minimum number of days after an item is updated, to allow it to check for updates again (either scheduled or param).
• TRAEFIK_DOMAIN: FQDN for the API server
• TRAEFIK_EMAIL: Email to register SSL cert with

Once the stack is up, test an API call with

https://${TRAEFIK_DOMAIN}/books/${ASIN}

Set up DB indexes to keep item lookups fast and to support searches.

1. Connect to the DB either from inside the mongodb container terminal or a MongoDB Compass/MongoSH session.

2. Switch to the correct DB:

   use audnexus

3. Create the recommended indexes:

   db.authors.createIndex( { asin: 1, region: 1 } )

   db.books.createIndex( { asin: 1, region: 1 } )

   db.chapters.createIndex( { asin: 1, region: 1 } )

   db.authors.createIndex( { name: "text" } )

⛏️ Built Using

• Fastify - Server Framework
• MongoDB - Database
• NodeJs - Server Environment
• Papr - Databse connection
• Redis - Cached responses

✍️ Authors

• @djdembeck - Idea & Initial work

🎉 Acknowledgements

• Huge thanks to mkb79 and their audible project for a great starting point.
• macr0dev for introducing us to scraping.
• seanap for passionately standardizing audiobook organization.
• Bookcamp for giving us a reason to have awesome audiobook data.