Docker deployment with pubsub autocook

davidemerli commented 1 year ago

Hi! I noticed that the readme is a bit outdated in some parts (like with some versions or ports specifications) and I tried to deploy Pandora with compose; I am trying to use a local object store.

Unfortunately I am having problems due to the cooking server never receiving any events.

I took some configurations from here and from the cooking server repo, and looked up into the source code for anything else that I had trouble configuring.

My current configuration is this one:

 |-pandora_recordings
 |-components
 | |-pub-sub.yml
 | |-state-store.yml
 | |-autocook.yml
 | |-object-store-local.yml
 |-compose.yml

with compose being:

 services:
  # The bot itself, record into raw, unusable files
  pandora:
    image: sotrx/pandora:2.3.3
    container_name: pandora
    restart: always
    environment:
      # Discord bot token
      - PANDORA_TOKEN=<TOKEN>
      # Prefix for text-based command
      - COMMAND_PREFIX=!
      # Dapr component for state storage
      - STORE_NAME=statestore
      - OBJECT_STORE_NAME=object-store-local
      - DISABLE_INTERACTION=true
      - PUBSUB_NAME=pubsub
      - DAPR_HTTP_PORT=3500
      - DAPR_SERVER_PORT=50051
    volumes:
      - ./pandora_recordings:/rec
    networks:
      - discord_recordings
  # Dapr sidecar, defining runtime implementations
  pandora-dapr:
    image: "daprio/daprd:edge"
    command:
      [
        "./daprd",
        "--app-id",
        "pandora",
        "--dapr-http-port",
        "3500",
        "--app-port",
        "50051",
        "--dapr-grpc-port",
        "50002",
        "--components-path",
        "/components",
      ]
    # In docker-compose, you have to provide components by sharing a volume
    # this is the dapr/components directory
    volumes:
      - "./components/:/components"
      - ./pandora_recordings:/rec
    depends_on:
      - pandora
    network_mode: "service:pandora"

  # Converts the raw files into audio files
  pandora-cooking-server:
    image: sotrx/pandora-cooking-server:2.4.4
    container_name: cooking-server
    environment:
      - OBJECT_STORE_NAME=object-store-local
      - PUBSUB_NAME=pubsub
    ports:
      - "3004:3004"
    restart: always
    volumes:
      - pandora_recordings:/app/rec
    networks:
      - discord_recordings

  # State store
  redis:
    container_name: redis
    image: "redis:alpine"
    networks:
      - discord_recordings

# Storing the recordings
volumes:
  pandora_recordings:

# Default docker network doesn't always provide name resolution
# so we create a new one
networks:
  discord_recordings:

And here's the DAPR files: autocook.yml

 # This subscription reacts on the topic `stoppedRecordingDiscord` and make the cooking server process
# the corresponding record.
# The `stoppedRecordingDiscord` event is fired by Pandora when it has successfully finished to record a voice channel
apiVersion: dapr.io/v1alpha1
kind: Subscription
metadata:
  name: autocook
spec:
  topic: stoppedRecordingDiscord
  route: /
  pubsubname: pubsub
scopes:
  - cooking-server

object-store-local.yml

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: object-store-local
spec:
  type: bindings.localstorage
  version: v1
  metadata:
    - name: rootPath
      value: /rec

pubsub.yml

 #./components/pubsub.yaml
# A sample pubsub component using Redis
# as a backend
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: pubsub
spec:
  type: pubsub.redis
  version: v1
  metadata:
    - name: redisHost
      value: redis:6379
    - name: redisPassword
      value: ""

state-store.yml

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: statestore
spec:
  type: state.redis
  version: v1
  metadata:
    - name: redisHost
      value: redis:6379
    - name: redisPassword
      value: ""

and finally here's the logs from docker:

Attaching to cooking-server, pandora, pandora-pandora-dapr-1, redis
redis                   | 1:C 04 Jan 2023 21:29:28.107 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis                   | 1:C 04 Jan 2023 21:29:28.107 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=1, just started
redis                   | 1:C 04 Jan 2023 21:29:28.107 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis                   | 1:M 04 Jan 2023 21:29:28.107 * monotonic clock: POSIX clock_gettime
redis                   | 1:M 04 Jan 2023 21:29:28.108 * Running mode=standalone, port=6379.
redis                   | 1:M 04 Jan 2023 21:29:28.108 # Server initialized
redis                   | 1:M 04 Jan 2023 21:29:28.109 * Ready to accept connections
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.438519822Z" level=info msg="starting Dapr Runtime -- version edge -- commit 1c95ad119a4257d1f0f1403eda0aced56c3fe848" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.438637946Z" level=info msg="log level set to: info" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.438927379Z" level=info msg="metrics server started on :9090/" app_id=pandora instance=273d5bdec16c scope=dapr.metrics type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.440052242Z" level=info msg="loading default configuration" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.461733731Z" level=info msg="Resiliency configuration loaded." app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.462047504Z" level=info msg="standalone mode configured" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.462082511Z" level=info msg="app id: pandora" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.462879235Z" level=info msg="mTLS is disabled. Skipping certificate request and tls validation" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.463135088Z" level=info msg="Dapr trace sampler initialized: DaprTraceSampler(P=0.000100)" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.464077359Z" level=info msg="local service entry announced: pandora -> 172.19.0.4:46605" app_id=pandora instance=273d5bdec16c scope=dapr.contrib type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.464161601Z" level=info msg="Initialized name resolution to mdns" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.464258936Z" level=info msg="loading components" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.475410119Z" level=info msg="successful init for output binding object-store-local (bindings.localstorage/v1)" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.475516178Z" level=info msg="component loaded. name: object-store-local, type: bindings.localstorage/v1" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.480977015Z" level=info msg="component loaded. name: pubsub, type: pubsub.redis/v1" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.482121235Z" level=info msg="waiting for all outstanding components to be processed" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.484598374Z" level=info msg="component loaded. name: statestore, type: state.redis/v1" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.484668814Z" level=info msg="all outstanding components processed" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.484689331Z" level=info msg="gRPC proxy enabled" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.485170116Z" level=info msg="enabled gRPC tracing middleware" app_id=pandora instance=273d5bdec16c scope=dapr.runtime.grpc.api type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.485303359Z" level=info msg="enabled gRPC metrics middleware" app_id=pandora instance=273d5bdec16c scope=dapr.runtime.grpc.api type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.485417338Z" level=info msg="API gRPC server is running on port 50002" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.48625731Z" level=info msg="enabled metrics http middleware" app_id=pandora instance=273d5bdec16c scope=dapr.runtime.http type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.486322296Z" level=info msg="enabled tracing http middleware" app_id=pandora instance=273d5bdec16c scope=dapr.runtime.http type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.487467421Z" level=info msg="http server is running on port 3500" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.487577336Z" level=info msg="The request body size parameter is: 4" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.487701951Z" level=info msg="enabled gRPC tracing middleware" app_id=pandora instance=273d5bdec16c scope=dapr.runtime.grpc.internal type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.487844087Z" level=info msg="enabled gRPC metrics middleware" app_id=pandora instance=273d5bdec16c scope=dapr.runtime.grpc.internal type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.488708324Z" level=info msg="internal gRPC server is running on port 46605" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:28.489301723Z" level=info msg="application protocol: http. waiting on port 50051.  This will block until the app is listening on that port." app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora                 | 2023-01-04T21:29:28: PM2 log: Launching in no daemon mode
pandora                 | 2023-01-04T21:29:28: PM2 log: App [main:0] starting in -fork mode-
pandora                 | 2023-01-04T21:29:28: PM2 log: App [main:0] online
cooking-server          | info: Using Remote Object storage "object-store-local" !
cooking-server          | info: Using message broker "pubsub" !
cooking-server          | info: ⚡️[server]: Server is running at port 3004
pandora                 | Dapr params : http port -> "3500", server port -> "50051"
pandora                 | Disabling interactions
pandora                 | info: Object store used, record will be uploaded on the storage backend
pandora                 | Up and running
pandora                 | 2023-01-04T21:29:30.740Z INFO [HTTPServer, HTTPServerImpl] [Topic = startRecordingDiscord] Registered Subscription with routes: default
pandora                 | 2023-01-04T21:29:30.749Z INFO [HTTPServer, HTTPServerImpl] [Topic = stopRecordingDiscord] Registered Subscription with routes: default
pandora                 | 2023-01-04T21:29:30.754Z INFO [HTTPServer, HTTPServer] Listening on 50051
pandora                 | 2023-01-04T21:29:30.755Z INFO [HTTPServer, HTTPServer] Registering undefined PubSub Subscriptions
pandora-pandora-dapr-1  | time="2023-01-04T21:29:30.762390992Z" level=info msg="application discovered on port 50051" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora-pandora-dapr-1  | time="2023-01-04T21:29:30.770383799Z" level=info msg="application configuration loaded" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora                 | 2023-01-04T21:29:30.774Z INFO [HTTPServer, HTTPServer] Registered undefined PubSub Subscriptions
pandora-pandora-dapr-1  | time="2023-01-04T21:29:30.782833879Z" level=info msg="app is subscribed to the following topics: [startRecordingDiscord stopRecordingDiscord] through pubsub=pubsub" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora                 | 2023-01-04T21:29:30.779Z INFO [HTTPServer, HTTPServer] Registered undefined PubSub Subscriptions
pandora-pandora-dapr-1  | time="2023-01-04T21:29:30.787457477Z" level=info msg="dapr initialized. Status: Running. Init Elapsed 2325ms" app_id=pandora instance=273d5bdec16c scope=dapr.runtime type=log ver=edge
pandora                 | 2023-01-04T21:29:30.797Z INFO [HTTPClient, HTTPClient] Sidecar Started
pandora                 | debug: [Controller PUBSUB] :: Registrations complete
pandora                 | Up and running
pandora                 | Up and running
pandora                 | debug: [Controller INTERACTIONS] :: Slash commands registration complete
pandora                 | 2023-01-04T21:29:36.577Z INFO [HTTPClient, HTTPClient] Sidecar Started
pandora                 | info: State is clean, no pending recording.
pandora                 | info: [Controller INTERACTIONS] :: Starting a new recording...
pandora                 | debug: Record parameters : {"voiceChannelId":"<VOICE_CHANNEL_ID>"}
pandora                 | debug: [Main] :: Record started with id 280147119
pandora                 | info: [Controller INTERACTIONS] :: Ending recording
pandora                 | info: Uploading the records...
pandora                 | 2023-01-04T21:29:51.949Z INFO [HTTPClient, HTTPClient] Sidecar Started
pandora                 | 2023-01-04T21:29:51.961Z INFO [HTTPClient, HTTPClient] Sidecar Started
pandora                 | 2023-01-04T21:29:51.971Z INFO [HTTPClient, HTTPClient] Sidecar Started
pandora                 | 2023-01-04T21:29:51.977Z INFO [HTTPClient, HTTPClient] Sidecar Started
pandora                 | 2023-01-04T21:29:51.985Z INFO [HTTPClient, HTTPClient] Sidecar Started
pandora                 | info: Recording ended successfully!

Thanks in advance for the help, if needed I can contribute to update the README configurations

davidemerli commented 1 year ago

Side note, I don't know if this is relevant or not but when the files get saved through Dapr local object store, the file names are mapped to random uuids

Since the requests to the server hosted on 3004 use filenames to retrieve stuff, this might be of interest

SoTrxII commented 1 year ago

Hey, Good catch, it's been quite a while since I've done this exemple and it is indeed outdated.

So there are two distincts problems :

For the uuid, the problem is there. I only ever use S3 like storage so I didn't see that the metadata attribute name to pass to Dapr was different for S3 (attribute is key) and local storage (attribute is fileName). Not providing the right attribute when uploading a file makes dapr use a random uuid instead. This should be an easy fix. I'll patch this.
Regarding the "autocooking", this is actually expected behaviour. The "autocooking" yaml file makes the cooking server listen to the "stoppedRecordingDiscord" event on the chosen pubsub. However this event is only fired by Pandora if the recording was started by the pubsub method. (The event is fired there and using a command (ex: !record) signalState is a NO-OP.) The intended usage with commands for users to retrieve their recordings using their browser to navigate to the cooking server exposed url (http://localhost:3004/*recording-id* in your case).

That being said, I'm quite curious. Did you have a specific goal in mind using both the autocook and commands at the same time ? I've never though of this usage.

davidemerli commented 1 year ago

Ok, I tried the new image and the files get correctly named.

I had to remove the object store from the env configurations for the cooking server and let it access files directly from a local volume.

Just a couple more things I found while trying things out

without the OBJECT_STORE_NAME in the Pandora environment, bindings for the Inversify initialization are skipped if I remember correctly and there is something trying to access ObjectStore which leads to errors

Error: No matching bindings found for serviceIdentifier: Symbol(ObjectStore)
pandora                 |     at _validateActiveBindingCount (/app/node_modules/inversify/src/planning/planner.ts:113:15)
pandora                 |     at _getActiveBindings (/app/node_modules/inversify/src/planning/planner.ts:91:3)
pandora                 |     at _createSubRequests (/app/node_modules/inversify/src/planning/planner.ts:146:22)
pandora                 |     at plan (/app/node_modules/inversify/src/planning/planner.ts:240:5)
pandora                 |     at /app/node_modules/inversify/src/container/container.ts:623:25
pandora                 |     at Container._get (/app/node_modules/inversify/src/container/container.ts:574:37)
pandora                 |     at Container._getButThrowIfAsync (/app/node_modules/inversify/src/container/container.ts:580:25)
pandora                 |     at Container.get (/app/node_modules/inversify/src/container/container.ts:325:17)
pandora                 |     at Object.<anonymous> (/src/inversify.config.ts:183:17)
pandora                 |     at Module._compile (node:internal/modules/cjs/loader:1165:14)

Another things that looks a bit confusing to me (but I am not really that familiar with Discord namings) is the DISABLE_INTERACTIONS config

If I put any value into the DISABLE_INTERACTIONS, the result is that I am not able to use slash commands anymore, while the commands with the specified prefix keep on working. Is this the intended behavior or is the other way around?

davidemerli commented 1 year ago

Anyways, for my use case for the autocook: I wanted to make a webapp able to play recordings coming from discord Specifically I wanted to:

start a recording
end the recording
link the recording to the application automatically
be able to play the audio when needed, possibly by separating the audio tracks for the users

So for this use case I wanted to automatically process the audio files, store them in a volume and access them from the webapp without having to process them further or call the cooking server.

"autocook" seemed to be something I could work with for this use case :P

Automatic linking may be something I can only do by forking the bot for my own needs, and is honestly an hassle. I think I will just create a form field on the app frontend where to paste the recording ID. Taking this into consideration I could just make a GET to the cooking server when the recording ID is pasted in the frontend, and avoid useless computation if not needed. (this was quite a stream of though, if you want to discuss things further we may find a more suitable place to chat wrt github issues)

SoTrxII commented 1 year ago

Hey, some news on this. I've been quite busy lately, and just got back to this. I hope you your workaround method worked out. I'm keeping this open as this is indeed something I could improve Pandora with !

SoTrxII / Pandora

Docker deployment with pubsub autocook #5