Parent directory /etc/nginx/conf.http.d does not exist since recent update

[zedtux]

:ghost: Brief Description

I have a cookbook which uses the nginx_site helper which started to raise the following error since I've updated the nginx cookbook from version 10.3.1:

* template[/etc/nginx/conf.http.d/east-stg-mst-1.myorg.lu.conf] action create[2020-12-08T09:51:26+01:00] INFO: Processing template[/etc/nginx/conf.http.d/east-stg-mst-1.myorg.lu.conf] action create (/var/chef/cache/cookbooks/nginx/resources/site.rb line 68)
* Parent directory /etc/nginx/conf.http.d does not exist.
================================================================================
Error executing action `create` on resource 'template[/etc/nginx/conf.http.d/east-stg-mst-1.myorg.lu.conf]'
================================================================================
Chef::Exceptions::EnclosingDirectoryDoesNotExist
------------------------------------------------
Parent directory /etc/nginx/conf.http.d does not exist.
Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/nginx/resources/site.rb
68: template config_file do
69: cookbook new_resource.cookbook
70: source new_resource.template
71:
72: user new_resource.user
73: group new_resource.group
74: mode '0640'
75:
76: variables(
77: new_resource.variables.merge({ name: new_resource.name })
78: )
79: end
80:
Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/nginx/resources/site.rb:68:in `block in class_from_file'
template("/etc/nginx/conf.http.d/east-stg-mst-1.myorg.lu.conf") do
action [:create]
default_guard_interpreter :default
source "teleport.erb"
declared_type :template
cookbook_name "teleport-le"
owner "www-data"
group "www-data"
mode "0640"
variables {:public_host=>"east-stg-mst-1.myorg.lu", :name=>"east-stg-mst-1.myorg.lu"}
cookbook "teleport-le"
path "/etc/nginx/conf.http.d/east-stg-mst-1.myorg.lu.conf"
verifications []
end
System Info:
------------
chef_version=15.8.23
platform=ubuntu
platform_version=18.04
ruby=ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
program_name=/usr/bin/chef-client
executable=/opt/chef/bin/chef-client

Creating the /etc/nginx/conf.http.d folder prevents from facing this error.

:pancakes: Cookbook version

11.1.0 (latest version of this cookbook)

:woman_cook: Chef-Infra Version

See above

:tophat: Platform details

See above

Steps To Reproduce

Steps to reproduce the behavior:

Not sure yet

:police_car: Expected behavior

Not failing at all.

:heavy_plus_sign: Additional context

The working version (10.3.1) was using the following last lines to build the nginx.conf file which matches existing folders on my server:

  include <%= @nginx_dir %>/conf.d/*.conf;
  include <%= @nginx_dir %>/sites-enabled/*;
}

but now, the newer version uses those lines which points to a non existing folder:

  include <%= @nginx_dir %>/conf.d/*.conf;
  include <%= @nginx_dir %>/conf.http.d/list*.conf;
}

[bmhughes]

Generally, v11 was a breaking change so it's expected that there will be some incompatibility.

If you want to use the old directory then you'll need to set the conf_dir property to relevant path, although I'd have expected the directory to be created here regardless.

Are you using nginx_config to manage the main configuration file? The previous version performed a lot of configuration in the install resource, this was one of the major changes with v11 where it was remvoed to improve the role segmentation of the resources. The same goes for the service management performed by the install resource.

[zedtux]

The nginx_dir is completely fine, it is about the "conf.http.d" folder which doesn't exist and I can't find anything on the internet recommending to use that folder. What works for me is the "sites-enabled" folder (with plenty of results in Google 😉) which has all the configs (well in the case of this cookbook, just one for LetsEncrypt).

You seem to forget to push the tags, so I can't easily find from when this has been changed, but in the latest available tag, 10.6.1, the folder is still right and would work.

The question is now: Why have you changed "sites-enabled" by "conf.http.d"?

[bmhughes]

They've been moved as 'sites' are sub-configurations of the http module, hence moving to http.d. If you wish to use a different module (stream for example) then the previous versions didn't easily enable this, with v11 you can override the directory and template and then include the generated files in the relevant nginx module config declaration. The sites-enabled/available is a historical Debian thing which don't make sense when managing with Chef, let chef do the orchestration.

The nginx_dir is completely fine, it is about the "conf.http.d" folder which doesn't exist and I can't find anything on the internet recommending to use that folder.

I'm not talking about nginx_dir. I'm talking about if you don't create an :nginx_config resource then the main nginx configuration will not be managed by Chef, so the includes won't be correct. This will also create the requried directories, but they should also be created by the nginx_site resource and also when creating the list.conf template. I'd need to see more of the chef-client run to determine any more, on our CI builds with Debian 9 and 10 everything is created and works as expected.

[zedtux]

Okay, thank you for those details.

If I understand you correctly, you'd like me to add the following to my cookbook:

nginx_config 'nginx' do
  action :create
  notifies :reload, 'nginx_service[nginx]', :delayed
end

?

[bmhughes]

Yes start with that. you may need to add extra properties/create your own template depending on what you need from nginx, the included one is reasonable for a basic setup only (although I believe that carried over from v10).

[zedtux]

Well I have now to figure out the config I need to pass to this nginx_config ... Why is not nginx_config knowing the basic config? And why is the doc showing that block as an example if it is incomplet? 😛

[bmhughes]

The test recipe uses a plain resource so it should be fine for a basic setup. Nginx configuration can get very complicated, there's no way to cover everything with a catch-all hence being able to override the template/pass arbitrary variables. What's your basic config isn't someone elses remember, it's best to keep these type of cookbooks as generic/abstract as possible.

[zedtux]

Okay, I understand that. I will see what I will do with this ...

Back to the initial comment, I guess there's still something to fix in order to avoid the issue with this missing folder.

[bmhughes]

Back to the initial comment, I guess there's still something to fix in order to avoid the issue with this missing folder.

I'm struggling to reproduce it tbh, from the ci tests the directory resource get's created 2/3 times with a normal run so I'd have expected it to do the same for you. I'll have to try without an :nginx_config resource to see if that makes any difference when I get chance.

[zedtux]

You can see my code, you know my Ubuntu version ... I can't really give you much I guess.

[zedtux]

Oh? Did you reproduced it successfully that you've closed the issue?

[bmhughes]

Oh? Did you reproduced it successfully that you've closed the issue?

Yep, copy paste error. Fix PR is in, still I'd recommend using nginx_config either way unless you're managing it elsewhere.

[zedtux]

Thank you @bmhughes