bitmagnet-io / bitmagnet

A self-hosted BitTorrent indexer, DHT crawler, content classifier and torrent search engine with web UI, GraphQL API and Servarr stack integration.
https://bitmagnet.io/
MIT License
2.07k stars 80 forks source link

ENH: Classification result protobuf includes content id and source #266

Closed rraymondgh closed 2 weeks ago

rraymondgh commented 3 weeks ago

Add

  optional string contentId = 12;
  optional string contentSource = 13;

to result object in classifier.

This enables using tmdbid to add tags to torrents. For example:

# define active series:
extensions:
  sonarr_active:
    [[REDACTED]]

# extend the default workflow with a custom workflow to tag torrents containing interesting documents:
workflows:
  custom:
    # first run the default workflow:
    - run_workflow: default
    # then add the tag to any torrents containing active in sonarr
    - if_else:
        condition: 
          and:
            - "result.contentType == contentType.tv_show"
            - "result.contentSource == 'tmdb' "
            - "result.contentId in extensions.sonarr_active  "
        if_action:
          add_tag: sonarr-active

I see that protobuf message can be extended in one one these ways

  1. extension of existing message where content is additional attributes being added this is the selected option
  2. extension of existing message where content is type, source, id tuple that makes up the whole ref. Leave contentType in place to maintain backwards compatibility with any classifier.yml that has been built by community. This would mean type is duplicated in the message (as contentType and content.type)
  3. add content as type, source, id, remove contentType. Most consistent message but would break any existing classifier.yml that uses contentType

update

have concluded on two new attributes: contentId and contentSource

mgdigital commented 3 weeks ago

Looks like you don't have the expected version numbers for the protobuf generator: https://github.com/bitmagnet-io/bitmagnet/actions/runs/9332593211/job/25689424444?pr=266#step:6:9 The easiest way to install correct versions would be with the Nix dev shell: https://bitmagnet.io/internals-development/tech-stack.html#nix-dev-shell But the output is otherwise identical so the numbers could just be changed manually...

rraymondgh commented 3 weeks ago

Have built a nixos vm to be able to use nix. A bit more needed for me to get to a integrated dev environment ;-)