search

grizzthedj / smart_proxy_ipam

Foreman Smart Proxy plugin for integration with External IPAM providers
GNU General Public License v3.0
18 stars 20 forks source link

Want to install your plugin but can't succeed #20

Closed Neurozone closed 4 years ago

Neurozone commented 4 years ago

The installation instruction here are not clear: https://theforeman.org/plugins/#2.Installation

Can you make a detailed procedure please ?

Neurozone commented 4 years ago

I don't have a Gemfile.local.rb file

grizzthedj commented 4 years ago

Installing just the plugins themselves will not be enough to get this integration to work. There are also code changes required on the Foreman side that are required. Until this patch is merged into a Foreman release, the only way to use this plugin(at least right now) will be to do the following. I have not yet started on the necessary PR's for adding these plugins to foreman-installer and puppet plugins etc.

NOTE: I would highly recommend doing this in a non-production system first to ensure that there are no issues! I have not tested these steps myself on a live environment.

  1. Checkout the feature/ipam_integration branch from my fork of Foreman core(Your Foreman server will need to be running this branch).
  2. Install the foreman_ipam plugin: Run gem install foreman_ipam from the /usr/share/foreman directory on your Foreman server.
  3. Run rails db:migrate from the /usr/share/foreman directory on your Foreman server.
  4. Run rails db:seed from the /usr/share/foreman directory on your Foreman server.
  5. Create a file called Gemfile.local.rb in the /usr/share/foreman/bundler.d directory with the following contents: 
    gem 'foreman_ipam'
  6. Restart foreman service and httpd
  7. On the Smart Proxy server, create an IPAM config at: /etc/foreman-proxy/settings.d/external_ipam.yml. An example config is here
  8. Install the smart_proxy_ipam plugin: Run gem install smart_proxy_ipam from /usr/share/foreman-proxy directory on your Foreman Proxy server).
  9. Create a file called Gemfile.local.rb in the /usr/share/foreman-proxy/bundler.d directory with the following contents: 
    gem 'smart_proxy_ipam'
  10. Restart foreman-proxy service and httpd

Navigate to Infrastructure -> IPAM Dashboard in your Foreman UI. If you see phpIPAM sections being loaded on the page, then the whole integration should be working.

stephenbcollier commented 4 years ago

Ive got the connection working but getting "A Smart Proxy with an External IPAM feature enabled must be selected in the Proxies tab." when I try and assign it to a subnet It appears in Proxies drop down and in proxy config not sure where to look from here

grizzthedj commented 4 years ago

@stephenbcollier: Are you certain that your proxy has the externalipam feature enabled? You can test this by hitting http://<PROXY_URL>/features. externalipam should be returned in the response.

stephenbcollier commented 4 years ago

Yes that returns as part of the features string if I just do the call with features. I'll try the url you suggested when I get into work.

On Fri, 7 Feb. 2020, 1:24 am Chris Smith, notifications@github.com wrote:

@stephenbcollier https://github.com/stephenbcollier: Are you certain that your proxy has the externalipam feature enabled? You can test this by hitting http:///features. externalipam should be returned in the response.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/grizzthedj/smart_proxy_ipam/issues/20?email_source=notifications&email_token=AC4B4374KIHAYLHKUODTDRLRBQMTJA5CNFSM4JDOVYVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK7MTAY#issuecomment-582928771, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC4B434RLA7TFW5XKFTW5Z3RBQMTJANCNFSM4JDOVYVA .

stephenbcollier commented 4 years ago

It returned ["dhcp","dns","externalipam","httpboot","logs","pulp","puppet","puppetca","templates","tftp"] which has the expected string

stephenbcollier commented 4 years ago

image We also get a spinning top on External IPAM which may be related image

grizzthedj commented 4 years ago

I suspect what is happening is that Foreman doesn't know about the External IPAM feature. You can verify this by running bundle exec rails console from the root of the Foreman directory on the server where Foreman is installed(not the proxy).

RAILS_ENV=<YOUR_RAILS_ENV> bundle exec rails console

Once in the console, then run Feature.where(name: "External IPAM"). You should see one record returned. If you do not, then you need to seed the database with RAILS_ENV=<YOUR_RAILS_ENV> bundle exec rails db:seed.

If you think seeding the database is too risky, you can add the feature manually with: 

Feature.create(name: "External IPAM", created_at: Time.now, updated_at: Time.now).save!
stephenbcollier commented 4 years ago

I'm getting -bash-4.2$ RAILS_ENV=/usr/share/foreman bundle exec rails Could not locate Gemfile

stephenbcollier commented 4 years ago

looks like an issue with bundle

grizzthedj commented 4 years ago

My apologies - try this.

From the /usr/share/foreman directory, run foreman-rake console

grizzthedj commented 4 years ago

And prefix with RAILS_ENV=production or whatever your rails environment is

stephenbcollier commented 4 years ago

<ActiveRecord::Relation [#<Feature id: 17, name: "External IPAM", created_at: "2020-02-05 06:03:53", updated_at: "2020-02-05 06:03:53">]>

stephenbcollier commented 4 years ago

I reseeded it originally but with foreman-rake as seems to be required

grizzthedj commented 4 years ago

Having trouble reproducing this. Perhaps try restarting the foreman service and the web server?

stephenbcollier commented 4 years ago

foreman-maintain service restart has been run several times which restarts everything as far as I'm aware

stephenbcollier commented 4 years ago

I'll have a close look at the code and make sure no differences to your source, THanks so much for your help it must be early evening in Canada

grizzthedj commented 4 years ago

No worries - Glad to help. Have a few things to take care of, but I will continue to try to reproduce on my end.

grizzthedj commented 4 years ago

Another possibility is that the feature is not associated with the smart proxy in the database. In the rails console, find the ID of your IPAM smart proxy with SmartProxy.all, and the ID of the externalipam feature with Feature.all, then see if the association exists in SmartProxyFeatures with SmartProxyFeature.all.

If the association doesn't exist, then it needs to be added .. Assuming your IPAM Smart Proxy ID is: 6, and your externalipam Feature is: 2, then you would ...

SmartProxyFeature.create(smart_proxy_id: 6, feature_id: 2, capabilities: [], settings: {}).save!

I remember adding this manually way back when, but this should actually be done in the seeds. I will update the docs and seeds file when I get a chance.

stephenbcollier commented 4 years ago

That would explain a lot. thanks. I'll give it a try later this morning and let you know.

On Sat, 8 Feb. 2020, 6:23 am Chris Smith, notifications@github.com wrote:

Another possibility is that the feature is not associated with the smart proxy in the database. In the rails console, find the ID of your IPAM smart proxy with SmartProxy.all, and the ID of the externalipam feature with Feature.all, then see if the association exists in SmartProxyFeatures with SmartProxyFeature.all.

If the association doesn't exist, then it needs to be added .. Assuming your IPAM Smart Proxy ID is: 6, and your externalipam Feature is: 2, then you would ...

SmartProxyFeature.create(smart_proxy_id: 6, feature_id: 2, capabilities: [], settings: {}).save!

I remember adding this manually way back when, but this should actually be done in the seeds. I will update the docs and seeds file when I get a chance.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/grizzthedj/smart_proxy_ipam/issues/20?email_source=notifications&email_token=AC4B4357LHN2BPIBFHZRIOLRBWYKRA5CNFSM4JDOVYVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELEH2FQ#issuecomment-583564566, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC4B437HGUW7RI75ALMZJMDRBWYKRANCNFSM4JDOVYVA .

stephenbcollier commented 4 years ago

On 8/02/2020 6:23 am, Chris Smith wrote:

Another possibility is that the feature is not associated with the smart proxy in the database. In the rails console, find the ID of your IPAM smart proxy with |SmartProxy.all|, and the ID of the |externalipam| feature with |Feature.all|, then see if the association exists in SmartProxyFeatures with |SmartProxyFeature.all|.

If the association doesn't exist, then it needs to be added .. Assuming your IPAM Smart Proxy ID is: 6, and your externalipam Feature is: 2, then you would ...

|SmartProxyFeature.create(smart_proxy_id: 6, feature_id: 2, capabilities: [], settings: {}).save! |

I remember adding this manually way back when, but this should actually be done in the seeds. I will update the docs and seeds file when I get a chance.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/grizzthedj/smart_proxy_ipam/issues/20?email_source=notifications&email_token=AC4B4357LHN2BPIBFHZRIOLRBWYKRA5CNFSM4JDOVYVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELEH2FQ#issuecomment-583564566, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC4B437HGUW7RI75ALMZJMDRBWYKRANCNFSM4JDOVYVA.

The IPAM proxy is not in Feature not sure about the ... at the end though irb(main):002:0> Feature.all => #<ActiveRecord::Relation [#<Feature id: 1, name: "Templates", created_at: "2020-01-22 23:11:08", updated_at: "2020-01-22 23:11:08">,

<Feature id: 2, name: "Pulp", created_at: "2020-01-22 23:13:48",

updated_at: "2020-01-22 23:13:48">, #<Feature id: 3, name: "Pulp Node", created_at: "2020-01-22 23:13:48", updated_at: "2020-01-22 23:13:48">,

<Feature id: 4, name: "Pulp3", created_at: "2020-01-22 23:13:48",

updated_at: "2020-01-22 23:13:48">, #<Feature id: 5, name: "TFTP", created_at: "2020-01-22 23:13:48", updated_at: "2020-01-22 23:13:48">,

<Feature id: 6, name: "DNS", created_at: "2020-01-22 23:13:48",

updated_at: "2020-01-22 23:13:48">, #<Feature id: 7, name: "DHCP", created_at: "2020-01-22 23:13:48", updated_at: "2020-01-22 23:13:48">,

<Feature id: 8, name: "Puppet", created_at: "2020-01-22 23:13:48",

updated_at: "2020-01-22 23:13:48">, #<Feature id: 9, name: "Puppet CA", created_at: "2020-01-22 23:13:48", updated_at: "2020-01-22 23:13:48">,

<Feature id: 10, name: "BMC", created_at: "2020-01-22 23:13:48",

updated_at: "2020-01-22 23:13:48">, ...]>

stephenbcollier commented 4 years ago

On 8/02/2020 6:23 am, Chris Smith wrote:

Another possibility is that the feature is not associated with the smart proxy in the database. In the rails console, find the ID of your IPAM smart proxy with |SmartProxy.all|, and the ID of the |externalipam| feature with |Feature.all|, then see if the association exists in SmartProxyFeatures with |SmartProxyFeature.all|.

If the association doesn't exist, then it needs to be added .. Assuming your IPAM Smart Proxy ID is: 6, and your externalipam Feature is: 2, then you would ...

|SmartProxyFeature.create(smart_proxy_id: 6, feature_id: 2, capabilities: [], settings: {}).save! |

I remember adding this manually way back when, but this should actually be done in the seeds. I will update the docs and seeds file when I get a chance.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/grizzthedj/smart_proxy_ipam/issues/20?email_source=notifications&email_token=AC4B4357LHN2BPIBFHZRIOLRBWYKRA5CNFSM4JDOVYVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELEH2FQ#issuecomment-583564566, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC4B437HGUW7RI75ALMZJMDRBWYKRANCNFSM4JDOVYVA.

OK got this so it looks like the association is OK. It seems like the smart variables have an issue from the ruby call that fails

irb(main):025:0> feature = Feature.find(17) => #<Feature id: 17, name: "External IPAM", created_at: "2020-02-05 06:03:53", updated_at: "2020-02-05 06:03:53">

irb(main):027:0> SmartProxyFeature.all => #<ActiveRecord::Relation [#<SmartProxyFeature smart_proxy_id: 1, feature_id: 2, id: 1, capabilities: [], settings: {"pulp_url"=>"https://lab03torch01.devops.vocus.local/pulp"}>,

<SmartProxyFeature smart_proxy_id: 1, feature_id: 6, id: 2,

capabilities: [], settings: {"use_provider"=>"dns_nsupdate"}>,

<SmartProxyFeature smart_proxy_id: 1, feature_id: 1, id: 3,

capabilities: [], settings: {}>, #<SmartProxyFeature smart_proxy_id: 1, feature_id: 5, id: 4, capabilities: [], settings: {"tftp_servername"=>nil}>, #<SmartProxyFeature smart_proxy_id: 1, feature_id: 7, id: 5, capabilities: ["dhcp_filename_hostname", "dhcp_filename_ipv4"], settings: {"use_provider"=>"dhcp_isc"}>,

<SmartProxyFeature smart_proxy_id: 1, feature_id: 9, id: 6,

capabilities: [], settings: {"puppet_url"=>"https://lab03torch01.devops.vocus.local:8140", "use_provider"=>["puppetca_hostname_whitelisting", "puppetca_http_api"]}>, #<SmartProxyFeature smart_proxy_id: 1, feature_id: 8, id: 7, capabilities: [], settings: {"puppet_url"=>"https://lab03torch01.devops.vocus.local:8140", "use_provider"=>["puppet_proxy_puppet_api"]}>, #<SmartProxyFeature smart_proxy_id: 1, feature_id: 13, id: 8, capabilities: [], settings: {}>, #<SmartProxyFeature smart_proxy_id: 1, feature_id: 14, id: 9, capabilities: [], settings: {"http_port"=>8000, "https_port"=>9090}>,

<SmartProxyFeature smart_proxy_id: 1, feature_id: 17, id: 10,

capabilities: [], settings: {}>]>

irb(main):032:0> SmartProxy.with_features('External IPAM').first => #<SmartProxy id: 1, name: "lab03torch01.devops.vocus.local", url: "https://lab03torch01.devops.vocus.local:9090", created_at: "2020-01-22 23:55:25", updated_at: "2020-01-23 23:46:28", expired_logs: "0", puppet_path: "/tmp/kafo_installation20200123-10920-fkcdmg/enviro...", download_policy: "on_demand">

so thats working

so why doesn't this

from subnet.rb  - the failing bit

if self.ipam == IPAM::MODES[:external_ipam]       external_ipam_proxy = SmartProxy.with_features('External IPAM').first

      if external_ipamproxy.nil?         self.errors.add :ipam, ('There must be at least one Smart Proxy present with an External IPAM plugin installed and configured')       elsif self.external_ipamproxy.nil?         self.errors.add :ipam, ('A Smart Proxy with an External IPAM feature enabled must be selected in the Proxies tab.')       elsif !subnet_exists_in_externalipam         self.errors.add :network, ('Subnet not found in the configured External IPAM instance')       end     end   end

stephenbcollier commented 4 years ago

Finally found the problem - it was a typo in line 208 on my patch of subnet.rb externalipam instead of external_ipam. Your code is correct - ALL working now - thanks - now finish setting up and demo to appropriate stakeholders Foreman -> phpIPAM -> powerDNS for full stack automation.

grizzthedj commented 4 years ago

Glad to hear you get everything sorted out! Going to close this issue given the instructions are working