devopsarr / terraform-provider-prowlarr

Terraform Prowlarr provider
https://registry.terraform.io/providers/devopsarr/prowlarr
Mozilla Public License 2.0
6 stars 1 forks source link

Providing YggTorrent indexer failing #210

Closed RaSamourai closed 9 months ago

RaSamourai commented 9 months ago

Hi :) (It's my first issue, I'll give as much details as I can. I hope it will be acceptable for you guys)

Technical context :

Details :

I can use the provider to provision my indexer proxy (FlareSolverr). By doing that, I can MANUALLY add my YggTorrent indexer whitout trouble. (if FlareSolverr is not in indexers proxies, it doesn't work). So I manually delete my YggTorrent indexer and then I try to provision it using Terraform.

Issue :

I've been trying to figure out how to use Terraform provider to provision indexers in Prowlarr using this provider : (devopsarr/prowlarr in version 2.3.0)

Code/Config :

Here is my providers.tf config :

terraform {
  required_providers {
    kubernetes = {
      source = "hashicorp/kubernetes"
      version = "2.25.2"
    }
    prowlarr = {
      source  = "devopsarr/prowlarr"
      version = "2.3.0"
    }
provider "prowlarr" {
    url     = "https://URL-TO-PROWLARR"
    api_key = "PROWLARR-API-KEY"
}

Here is my indexer.tf :

resource "prowlarr_indexer" "YGG" {                                                                    
  enable          = true
  name            = "YGG"
  implementation  = "YggTorrent"
  config_contract = "YggTorrentSettings"
  protocol        = "torrent"
  app_profile_id  = 1
  priority        = 2
  tags            = [1]
  fields = [
    {
      name       = "username"
      #name       = "usernameSettings"
      text_value = "${var.username}"
    },
    {
      name       = "password"
      #name       = "passwordSettings"
      sensitive_value = "${var.password}"
    },
    {
      name       = "apiKey"
      text_value = "${var.apiKey}"
    },
  ]
}

I've tried a lot of syntax and I can't get any thing done :'(

Questions :

I have several questions :

PostScriptum :

I searched in Prowlarr's official documentation, I didn't find anything that can get my out of trouble. I read about something named Cardigann that is the template manager used in Prowlarr, which supports YggTorrent according to the GitHub page. And finally, I've been googling a lot and I can't found anything to solve my problem (seems people using Terraform to manage Prowlarr are very few).

Thanks in advance for your attention and your time. I hope I don't bother you and just act like a noob, I really did my best :)

Fuochi commented 9 months ago

Hi @RaSamourai, thanks for opening the issue. Unfortunately Prowlarr's documentation isn't the best, and the one I wrote for the provider is even worse. Let me try to clarify some aspects.

You can use the datasource prowlarr_indexer_schemas to get a list of all indexer schema names:

data "prowlarr_indexer_schemas" "test" {
}

Then, you could use the datasourceprowlarr_indexer_schema to identify the fields needed by your indexer:

data "prowlarr_indexer_schema" "test" {
  name = "YGGtorrent"
}

Based on the previous result, you'll have the to populate all the non-empty fields (and all the booleans):

resource "prowlarr_indexer" "ygg" {
  enable          = true
  name            = "YGG"
  implementation  = "Cardigann"
  config_contract = "CardigannSettings"
  protocol        = "torrent"
  app_profile_id  = 1
  priority        = 2
  tags            = []
  fields = [
    {
      name       = "definitionFile"
      text_value = "yggtorrent"
    },
    {
      name       = "username"
      text_value = "test"
    },
    {
      name       = "password"
      text_value = var.password
    },
    {
      name         = "type"
      number_value = 1
    },
    {
      name       = "enhancedAnime"
      bool_value = false
    },
    {
      name       = "enhancedAnime4"
      bool_value = false
    },
    {
      name       = "filter_title"
      bool_value = false
    },
    {
      name       = "multilang"
      bool_value = false
    },
    {
      name       = "vostfr"
      bool_value = false
    },
    {
      name       = "strip_season"
      bool_value = true
    },
    {
      name         = "category"
      number_value = 10
    },
    {
      name        = "flaresolverr"
      sting_value = "This site may use Cloudflare DDoS Protection, therefore Prowlarr may require <a href=\"https://github.com/FlareSolverr/FlareSolverr\" target=\"_blank\">FlareSolver</a> added as a proxy in Prowlarr (Settings => Indexers) to access it."
    },
    {
      name         = "info_category"
      string_value = "Only select a <i>Category</i> <b>OR</b> a <i>Sub-category</i>, leaving the other as <i>Tous</i>."
    },
    {
      name         = "multilanguage"
      number_value = 1
    },
    {
      name         = "sort"
      number_value = 1
    },
    {
      name         = "subcategory"
      number_value = 52
    },
    {
      name         = "baseSettings.limitsUnit"
      number_value = 0
    },
  ]
}

I tried to make the provider the most generic I could, so it should work for every indexer, but I understand it's a bit tricky, and not bug free.

Thanks to this ticket I noticed I have to remove the info fields from the equation (like flaresolverr and info_category) I'll try to do that in next release. Please, let me know if it's clearer now.

RaSamourai commented 9 months ago

Hi @Fuochi :D

Thanks a lot for your answer, it helped :D :D :D I managed to make it work, but not exactly as you said.

Sorry for the delay, I've worked on this all night and I slept most of the day XD

Code/Config :

Here is my datasource.tf :

data "prowlarr_indexer_schemas" "schemas" {                                                                                                                                                                       
}                      

data "prowlarr_indexer_schema" "options" {
  name = "YGGtorrent"  
} 

I manage to get the content using those commands :

# This command allowed me to get the correct value to fill data.prowlarr_indexer_schema
terraform state show module.<$MODULE-NAME>.data.prowlarr_indexer_schemas.<RESOURCE-NAME>|grep -i yggtorrent

# This command allowed me to get fill the "fields" section and the value for "implementation" & "config_contract" in the indexer resource
terraform state show module.prowlarr-provision.data.prowlarr_indexer_schema.options

Here is my indexer.tf :

resource "prowlarr_indexer" "ygg" {
  enable          = true
  name            = "YGG"
  implementation  = "Cardigann"
  config_contract = "CardigannSettings"
  protocol        = "torrent"
  app_profile_id  = 1 
  priority        = 2 
  tags            = [1]
  fields = [          
    {                 
      description = ""
      name        = "category"
      type        = "select"
      number_value = 10
    },                
    {                 
      description = ""
      name        = "definitionFile"
      type        = "textbox"
      text_value = "yggtorrent"
    },                
    {                 
      description = ""
      name        = "enhancedAnime"
      bool_value = false
      type        = "checkbox"
    },                
    {                 
      description = ""
      name        = "enhancedAnime4"
      type        = "checkbox"
      bool_value = false
    },
    {                 
      description = ""                                                                                                                                                                                            
      name        = "filter_title"
      type        = "checkbox"
      bool_value = false
    },                
    {                 
      description = ""
      name        = "flaresolverr"
      type        = "info"
      sting_value = "This site may use Cloudflare DDoS Protection, therefore Prowlarr may require <a href=\"https://github.com/FlareSolverr/FlareSolverr\" target=\"_blank\">FlareSolver</a> added as a proxy in Prowlarr (Settings => Indexers) to access it."
    },                
    {                 
      description = ""
      name        = "info_category"
      type        = "info"
      string_value = "Only select a <i>Category</i> <b>OR</b> a <i>Sub-category</i>, leaving the other as <i>Tous</i>."
    },                
    {                 
      description = ""
      name        = "multilang"
      type        = "checkbox"
      bool_value = false
    },                
    {                 
      description = ""
      name        = "multilanguage"
      type        = "select"
      number_value = 1
    },                
    {                 
      description = ""
      name        = "password"
      type        = "password"
      text_value  = "var.password"
    },                
    {                 
      description = ""
      name        = "sort"
      type        = "select"
      number_value = 1
    },                                                                                                                                                                                                            
    { 
      description = ""
      name        = "strip_season"
      type        = "checkbox"
      bool_value = true
    },               
    { 
      description = ""
      name        = "subcategory"
      type        = "select"
      number_value = 52
    },               
    { 
      description = ""
      name        = "type"
      type        = "select"
      number_value = 1
    },               
    { 
      description = ""
      name        = "username"
      type        = "textbox"
      text_value = "var.username"
    },               
    { 
      description = ""
      name        = "vostfr"
      type        = "checkbox"
      bool_value = false
    },               
    { 
      description = "Minimum seeders required by the Applications for the indexer to grab, empty is Sync profile's default"
      name        = "torrentBaseSettings.appMinimumSeeders"
      type        = "number"
      number_value = 1
    },               
    { 
      description = "Select which base url Prowlarr will use for requests to the site"
      name        = "baseUrl"
      type        = "select"
      text_value  = "https://www3.yggtorrent.qa/"
    },
    {
      description = "The number of max grabs as specified by the respective unit that Prowlarr will allow to the site"
      name        = "baseSettings.grabLimit"
      type        = "number"
      number_value = 100
    },
    {
      description = "The number of max queries as specified by the respective unit that Prowlarr will allow to the site"
      name        = "baseSettings.queryLimit"
      type        = "number"
      number_value = 50
    },
    {
      description = "The ratio a torrent should reach before stopping, empty uses the download client's default. Ratio should be at least 1.0 and follow the indexers rules"
      name        = "torrentBaseSettings.seedRatio"
      type        = "textbox"
      number_value = 1.5
    },
    {
      description = "The time a pack (season or discography) torrent should be seeded before stopping, empty is app's default"
      name        = "torrentBaseSettings.packSeedTime"
      type        = "number"
      number_value = 3600
    },
    {
      description = "The time a torrent should be seeded before stopping, empty uses the download client's default"
      name        = "torrentBaseSettings.seedTime"
      type        = "number"
      number_value = 7200
    },
    {
      description = "The unit of time for counting limits per indexer"
      name        = "baseSettings.limitsUnit"
      type        = "select"
      number_value = 1
    },
  ]
} 

Details :

Annoying lines :

I tried to get ride of the lines type = "$xxx". Turns out that when those lines are commented, Terraform returns an error. (I'm not sure if the issue is on Terraform's side or Prowlarr's side). Anyway, I have to get them here in order to make it work.

Sensitive values :

I tried to use "sensitive_value" instead of "text_value" for the password and it doesn't work. (Again, I don't know if the issue in on provider's side or on Prowlarr's side).

Here is the error :

[ERROR] vertex "module.prowlarr-provision.prowlarr_indexer.ygg" error: Provider produced inconsistent result after apply
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to module.prowlarr-provision.prowlarr_indexer.ygg, provider
│ "provider[\"registry.terraform.io/devopsarr/prowlarr\"]" produced an unexpected new value: .fields:
│ planned set element cty.ObjectVal(map[string]cty.Value{"bool_value":cty.UnknownVal(cty.Bool),
│ "name":cty.StringVal("password"), "number_value":cty.UnknownVal(cty.Number),
│ "sensitive_value":cty.StringVal("PASSWORD"), "set_value":cty.UnknownVal(cty.Set(cty.Number)),
│ "text_value":cty.UnknownVal(cty.String)}) does not correlate with any element in actual.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Uncertainties :

Thanks :

Thank you a lot for pointing me in the right direction, I can know manage my medias downloaders As Code, it's a victory to me :) I don't know if you're alone or more than 1 person to work on this, but I would be glad to write some documentation on how to use this provider :) (at least, for the feature I tried on it). Many many thanks for the work you've done :) I'll tried to create tags using Terraform and datasources to apply them dynamically to indexers and proxies. This way, everything will be fully automated. I'll get back to you if I encounter issues ;) I let you close this thread if you wish, I'll open another one if needed :)

Thanks again :)

Fuochi commented 9 months ago

Hi @RaSamourai. Happy that something worked :). Here few notes following your comment:

Annoying lines :

Firstly, I honestly have no clue on how it can work with description and type attributes since they're not defined here: https://github.com/devopsarr/terraform-provider-prowlarr/blob/f17e501c366d15961f03c17e43641f37f4d3738d/internal/provider/indexer_resource.go#L141-L174 It seems that terraform validate doesn't validate arguments in nested objects. Those 2 fields aren't used anywhere in the code, and it must work even without them. BTW, what terraform version are you using?

Sensitive values :

About the sensitive values: I added them to manage fields that are masked by Prowlarr's API. So it should work only when the API response is something like ********. If Prowlarr sends them back in clear text, I treat them as text values. I know it's a bit tricky, and the only way to know which one to use is trying. On the other hand, it provides the flexibility of using the same source code for both scenario without explicitly setting which field is which. I valued this more since there are hundreds of indexers, and I can't manage each one of them separately by myself.

Uncertainties :

This unfortunately is part of the *arr ecosystem (at least, I think). I usually check via GUI in advance when I've to deal with this kind of fields.

And... Finally, yes I'm quite alone fore now. If you'd like to jot down something i'll be happy to review. Just a note for the doc, I'm using a fork of https://github.com/hashicorp/terraform-plugin-docs to manage the doc. So, if you write a custom *.md page, put that under the /templates folder (which doesn't exists for now), and run make generate form the repo folder to generate the doc site.

I'll keep this ticket open for a couple of days, if you want to add something more. If there are no news, I'll close this on Tuesday!

BTW, many thanks for taking the time in opening the issue, and providing such an extensive feedback, it really helps.

RaSamourai commented 9 months ago

Hi @Fuochi :)

I'm back with lil update about the "annoying lines". I didn't commented them properly. I used # instead of //. Terraform is quiet new to me and I still have bash and yaml habits x)

I tested it and you were right, those lines are not necessary, my bad :)

About my Terraform version, I ran this command terraform version and here is what is got :

Terraform v1.6.6-dev
on linux_amd64
+ provider registry.terraform.io/devopsarr/prowlarr v2.3.0
+ provider registry.terraform.io/devopsarr/radarr v2.1.0
+ provider registry.terraform.io/devopsarr/sonarr v3.1.1
+ provider registry.terraform.io/hashicorp/kubernetes v2.25.2

Your version of Terraform is out of date! The latest version
is 1.7.4. You can update by downloading from https://www.terraform.io/downloads.html

I took a rapid look at the repository you're using to manage documentation... I have to say that I know nothing about it XD I'll give it a try following your instructions and if I can't manage to do it, I'll open an issue to give you what I produced and you'll be able to do whatever you want with it :)

BTW, many thanks for taking the time in opening the issue, and providing such an extensive feedback, it really helps.

You're welcome :) I'm an I.T. guy (LinkedIn calls me a DevOps Engineer XD) and I like when people give me all the information I need at once. So I do the same to them ;)