Unable to add new computes in version 0.8

[pdzionek]

Describe the bug I am not able to add any nodes with the latest plugin version. When I click "Test connection" nodes are not listed and in proxmox logs I see authentication failure. I did some debugging and I see in apache/foreman logs this behaviour:

FYI, I just added puts in ./vendor/ruby/2.5.0/gems/fog-proxmox-0.8.0/lib/fog/proxmox.rb file

App 4829 stdout: login:foreman@pve
App 4829 stdout: password:test1234 App 4829 stdout: login:foreman@pve App 4829 stdout: password:5CEE69DD:N7C3tivFbwmdQbOpM3pZ0aCYgoQ

So I guess it looks like this: When I click "Test connection" the first call to API works, but there is another call to list nodes and this call tries to use ticket as a password and this creates a problem. As I understand Proxmox API, this token should be passed as a cookie in next calls. I am not sure how you implemented it though. Passing it directly as a password probably not gonna work. https://pve.proxmox.com/wiki/Proxmox_VE_API

I am using latest foreman 1.22 rc2 and 0.8 version of this plugin. I also ran tests on proxmox 5.2, 5.3 and 5.4.

[LaurentDumont]

I was previously running 0.6 with Proxmox 5.4 which worked "okay".

Upgraded to Foreman RC1 1.22 and foreman_fog_proxmox 0.8.0 and it seems that I'm facing the same issue.

Unable to save
Expected([200, 204]) <=> Actual(401 Unauthorized) excon.error.response :body => "{\"data\":null}" :cookies => [ ] :headers => { "Cache-Control" => "max-age=0" "Connection" => "close" "Content-Length" => "13" "Content-Type" => "application/json;charset=UTF-8" "Date" => "Thu, 30 May 2019 03:32:43 GMT" "Expires" => "Thu, 30 May 2019 03:32:43 GMT" "Pragma" => "no-cache" "Server" => "pve-api-daemon/3.0" } :host => "10.10.69.11" :local_address => "10.10.99.4" :local_port => 43142 :path => "/api2/json/access/ticket" :port => 8006 :reason_phrase => "authentication failure" :remote_ip => "10.10.69.11" :status => 401 :status_line => "HTTP/1.1 401 authentication failure\r\n"

[pdzionek]

Ok, I found a solution to this problem. If it is TL:DR for you just edit for Ubuntu:

/usr/share/foreman/vendor/ruby/2.5.0/gems/fog-proxmox-0.8.0/lib/fog/proxmox.rb

and for CentOS/Redhat

/opt/theforeman/tfm/root/usr/share/gems/gems/fog-proxmox-0.8.0/lib/fog/proxmox.rb

and comment out line 82

      @api_path = uri.path
      connection_options = connection_options.merge(path_prefix: @api_path)
#     password = @credentials[:csrftoken] if credentials_has_expired?
      puts password
      retrieve_tokens(uri, connection_options, username, password) unless authenticated? && !credentials_has_expired?
    end

Long version, Proxmox documentation states that if you want to get a new ticket, you can send your old one.

NOTE: Tickets have a limited lifetime of 2 hours. But you can simple get a new ticket by passing the old ticket as password to the /access/ticket method. 

but in plugin code it tries to send :csrftoken and it should send a ticket. However, changing this to :ticket still won't solve the problem. I found out that the issue here is the endpoint that plugin tries to use (thanks to this topic https://forum.proxmox.com/threads/api-how-to-get-new-ticket.19034/ ):

With https://proxmox01:8006/api2/access/ticket it gets empty reponse

curl -k --data "username=foreman" --data "realm=pve" --data-urlencode "password=PVE:foreman@pve:5D19BF1A::P2lmGssG0m.....FEZds2rQ==" https://proxmox01:8006/api2/access/ticket | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   442    0     0  100   442      0    146  0:00:03  0:00:03 --:--:--   146

but if it is https://proxmox01:8006/api2/extjs/access/ticket it works:

curl -k --data "username=foreman" --data "realm=pve" --data-urlencode "password=PVE:foreman@pve:5D19BF1A::P2lmGssG0m.....FEZds2rQ==" https://proxmox01:8006/api2/extjs/access/ticket | jq .
{
  "data": {
    "ticket": "PVE:foreman@pve:5D19BFCA::NlypLP3o/lO1......N2RxSZ6TQS70a4E8SZrS5bBV7+AG/wYmNrvThQoNZ0mx/fCAg==",
    "cap": {
      "nodes": {
        "Sys.PowerMgmt": 1,
        "Sys.Modify": 1,
        "Sys.Console": 1,
        "Permissions.Modify": 1,
        "Sys.Syslog": 1,
        "Sys.Audit": 1
      },
      "access": {
....
}

Therefore the issue here is incorrect endpoint. However, I am not sure what is the difference between the one with or without extjs part.

To summarize, commenting out line 82 disables this mechanism and plugin just uses your password every time it needs a ticket.

[hlawatschek]

Hi @pdzionek, thanks for your info. This seems to be very helpful. But I can't find the code you are referring to. Could you please provide a patch to the latest master from fog-proxmox?

[pdzionek]

@hlawatschek I would like to add it, but I have a "small" issue. I did this change and tests on running foreman with 0.8.0 plugin installed, but I cannot find latest source for this file. In https://github.com/fog/fog-proxmox there is older version of this file. Was this repository moved somewhere else ? Or I don't see specific branch ? Probably I am just missing something obvious.

[hlawatschek]

@pdzionek Not that I am aware of. Where did you get your version?

[pdzionek]

@hlawatschek I just installed latest foreman 1.22 with ruby-foreman-fog-proxmox pkg, and I guess it ran some init script which installed fog-proxmox-0.8.0 bundle from https://rubygems.org/gems/fog-proxmox. I am not sure why this changes are here only and not in fog-proxmox git repo.

FYI, I am not Ruby developer so I am just guessing how it got in Foreman 1.22 installation.

[TekunoKage]

To summarize, commenting out line 82 disables this mechanism and plugin just uses your password every time it needs a ticket.

@pdzionek I try your suggestion but not work. Any other hint?

[brindapabari]

Hi @pdzionek I am wondering if you got the foreman_fog_proxmox plugin working for foreman 1.22. I am trying to install the plugin with this command:

gem install foreman_fog_proxmox

Successfully installed foreman_fog_proxmox-0.8.0 Parsing documentation for foreman_fog_proxmox-0.8.0 Done installing documentation for foreman_fog_proxmox after 0 seconds 1 gem installed

But when I restart httpd and go to the foreman website -->Infrastructure->Compute Resources ->Create---I cannot see Proxmox as Provider. Then I checked the Administer->About section ->Plugins ---Fog proxmox plugin is not there even.

Could you please let me know what should I do to fix this? Any pointers would be really helpful

[brindapabari]

@pdzionek

Following are the steps I followed for installing foreman_fog_proxmox (0.8) plugin in CentOS (7) for the latest foreman version (1.22) and finally had a working version! I had a difficult time figuring these out, since the given steps were Debian specific and referring to older foreman version paths. It would be very helpful if github repo had installation steps for latest Foreman and CentOS specific as well for the future upgrade of plugin. • $sudo install -y foreman foreman-compute foreman-sqlite3 foreman-asset • In /usr/share/foreman/bundler.d directory, create a Gemfile.local.rb file and add following content in it: gem 'foreman_fog_proxmox'

• System ruby version might be different from tfm-ruby version. Foreman uses tfm-ruby, therefore run the shell with following command in order to confirm that tfm version is being used. 
Use only foreman user (not root!) sudo -u foreman … If required give read write access to foreman user to the /usr/share/foreman directory

sudo -u foreman scl enable tfm bash
    ○ bash-4.2$ whoami
        foreman
    ○ bash-4.2$ set
        Check all the environment variables displayed
    ○ bash-4.2$ gem install foreman_fog_proxmox
    ○ bash-4.2$ cd  /usr/share/foreman/
    ○ Install the gem plugin:
    bash-4.2$ bundle install 
    ○ Not sure if Precompilation of plugin assets is required or not. It gave me errors but the plugin was already installed. Therefore I processed with testing the plugin functionalities of computing the proxmox resource
• Opening up Foreman website -> Compute Resources throws permission error 
• [root]$ setenforce 0

Suggestion here : It would be worth understanding that how does SELinux work in integration with Foreman? A few things work perfect on disabling selinux and don't when enabled. How should I administer or architecture the flow of permissions in entire foreman setup? Currently I have default created during foreman installatiion. • After disabling the selinux Foreman website shows the installed plugin in Administer ->About ->Plugins. Able to open the Compute Resources page. • Upon saving the Create Resource page / clicking on Test Connection of Proxmox server, it gives the authentication error similar to this open issue: https://github.com/theforeman/foreman_fog_proxmox/issues/86

Resolution provided by @pdzionek works for me: ○ Commenting out line 82 from the following file: • $vi /usr/share/foreman/.gem/ruby/gems/fog-proxmox-0.8.0/lib/fog/proxmox.rb

This worked for me and I was able to compute the proxmox resource. However, "Create Image" for the created resource failed an error. This was because the plugin initernally needs information from the "local" drive of Proxmox server. Local was disabled in our server, therefore it was throwing the error. Once the local drive was enabled, we were able to create images.

[MarvinMedia]

Ok, I found a solution to this problem. If it is TL:DR for you just edit:

/usr/share/foreman/vendor/ruby/2.5.0/gems/fog-proxmox-0.8.0/lib/fog/proxmox.rb

and comment out line 82

      @api_path = uri.path
      connection_options = connection_options.merge(path_prefix: @api_path)
#     password = @credentials[:csrftoken] if credentials_has_expired?
      puts password
      retrieve_tokens(uri, connection_options, username, password) unless authenticated? && !credentials_has_expired?
    end

Long version, Proxmox documentation states that if you want to get a new ticket, you can send your old one.

NOTE: Tickets have a limited lifetime of 2 hours. But you can simple get a new ticket by passing the old ticket as password to the /access/ticket method. 

but in plugin code it tries to send :csrftoken and it should send a ticket. However, changing this to :ticket still won't solve the problem. I found out that the issue here is the endpoint that plugin tries to use (thanks to this topic https://forum.proxmox.com/threads/api-how-to-get-new-ticket.19034/ ):

With https://proxmox01:8006/api2/access/ticket it gets empty reponse

curl -k --data "username=foreman" --data "realm=pve" --data-urlencode "password=PVE:foreman@pve:5D19BF1A::P2lmGssG0m.....FEZds2rQ==" https://proxmox01:8006/api2/access/ticket | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   442    0     0  100   442      0    146  0:00:03  0:00:03 --:--:--   146

but if it is https://proxmox01:8006/api2/extjs/access/ticket it works:

curl -k --data "username=foreman" --data "realm=pve" --data-urlencode "password=PVE:foreman@pve:5D19BF1A::P2lmGssG0m.....FEZds2rQ==" https://proxmox01:8006/api2/extjs/access/ticket | jq .
{
  "data": {
    "ticket": "PVE:foreman@pve:5D19BFCA::NlypLP3o/lO1......N2RxSZ6TQS70a4E8SZrS5bBV7+AG/wYmNrvThQoNZ0mx/fCAg==",
    "cap": {
      "nodes": {
        "Sys.PowerMgmt": 1,
        "Sys.Modify": 1,
        "Sys.Console": 1,
        "Permissions.Modify": 1,
        "Sys.Syslog": 1,
        "Sys.Audit": 1
      },
      "access": {
....
}

Therefore the issue here is incorrect endpoint. However, I am not sure what is the difference between the one with or without extjs part.

To summarize, commenting out line 82 disables this mechanism and plugin just uses your password every time it needs a ticket.

Thats works for me, tanks.

[pdzionek]

@pdzionek

Following are the steps I followed for installing foreman_fog_proxmox (0.8) plugin in CentOS (7) for the latest foreman version (1.22) and finally had a working version! I had a difficult time figuring these out, since the given steps were Debian specific and referring to older foreman version paths. It would be very helpful if github repo had installation steps for latest Foreman and CentOS specific as well for the future upgrade of plugin. • $sudo install -y foreman foreman-compute foreman-sqlite3 foreman-asset • In /usr/share/foreman/bundler.d directory, create a Gemfile.local.rb file and add following content in it: gem 'foreman_fog_proxmox'

• System ruby version might be different from tfm-ruby version. Foreman uses tfm-ruby, therefore run the shell with following command in order to confirm that tfm version is being used. 
Use only foreman user (not root!) sudo -u foreman … If required give read write access to foreman user to the /usr/share/foreman directory

sudo -u foreman scl enable tfm bash
  ○ bash-4.2$ whoami
      foreman
  ○ bash-4.2$ set
      Check all the environment variables displayed
  ○ bash-4.2$ gem install foreman_fog_proxmox
  ○ bash-4.2$ cd  /usr/share/foreman/
  ○ Install the gem plugin:
  bash-4.2$ bundle install 
  ○ Not sure if Precompilation of plugin assets is required or not. It gave me errors but the plugin was already installed. Therefore I processed with testing the plugin functionalities of computing the proxmox resource
• Opening up Foreman website -> Compute Resources throws permission error 
• [root]$ setenforce 0

Hi, For CentOS 7, after installing Foreman, I just installed

yum install tfm-rubygem-fog-proxmox.noarch tfm-rubygem-foreman_fog_proxmox.noarch 

and did a httpd restart. After that, I found proxmox in plugins and providers tab in GUI.

[pdzionek]

@TekunoKage

To summarize, commenting out line 82 disables this mechanism and plugin just uses your password every time it needs a ticket.

@pdzionek I try your suggestion but not work. Any other hint?

Hmm not really.
FYI, I updated solution for different OS, Ubuntu or CentOS. The first version was only for Ubuntu, and for CentOS/Redhat file path differs.

[tristanrobert]

@pdzionek I am working on bug fixes and enhancements. But I don't have so much time now. If you have any code solution you should test it (add vcr spec test) and send it in a pull request.

[pdzionek]

@tristanrobert Apart from the fact, it would be pretty tricky for me(but I could try :) ), I have another issue with it. Please read my quoted comment:

@hlawatschek I would like to add it, but I have a "small" issue. I did this change and tests on running foreman with 0.8.0 plugin installed, but I cannot find latest source for this file. In https://github.com/fog/fog-proxmox there is older version of this file. Was this repository moved somewhere else ? Or I don't see specific branch ? Probably I am just missing something obvious.

[tristanrobert]

duplicate #64