Could not prefetch cups_queue provider 'cups' on Ubuntu 18.04

threepistons commented 5 years ago

Given Ubuntu 18.04 LTS with Puppet 6.4.2 and CUPS 2.2.7 and leoarnold/cups 2.2.2.

When I apply the manifest I use roles and profiles. The profiles::printing class follows:

class profiles::printing {

  # for convenience so I don't type the same thing over and over
  $ppdpath  = '/usr/share/cups/model'
  $models   = ['KMC759ux.ppd']
  case $facts['lsbdistid'] {
    'Raspbian': {
      $packages = ['cups-bsd', 'a2ps']
    }
    default: {
      $packages = ['cups-bsd', 'a2ps', 'indicator-printers']
    }
  }

  include ksmb # this is a single-class module that installs the "ksmb" kerberised SMB backend.

  class { 'cups':
    default_queue => 'USB_Building_Printing',
    papersize     => 'A4',
  }

  package { $packages:
    ensure => installed,
  }

  file { $ppdpath: ensure => directory }

  $models.each | $model | {
    file { $model:
      ensure  => file,
      replace => yes,
      source  => "puppet:///modules/profiles/ppds/${model}",
      path    => "${ppdpath}/${model}",
      group   => 'lp',
      mode    => '0644',
    }
  }

  cups_queue { 'USB_Building_Printing':
    ensure      => 'printer',
    uri         => 'ksmb://cs-print/USB-BUILDING-PRINTING',
    ppd         => "${ppdpath}/KMC759ux.ppd",
    shared      => false,
    enabled     => true,
    accepting   => true,
    description => 'All Urban Sciences printers, swipe your campus card at any printer to collect your job.',
    options     => {
      'KMDuplex'      => '2Sided',
      'PageSize'      => 'A4',
      'InputSlot'     => 'Tray4',
      'PaperSources'  => 'PC215',
      'Finisher'      => 'FS533',
      'KOPunch'       => 'PK519-4',
      'SaddleUnit'    => 'None',
      'PrinterHDD'    => 'HDD',
      'SelectColor'   => 'Grayscale',
      'Model'         => 'C458',
      'TextPureBlack' => 'On',
    },
  }

}

iptables-save says -A INPUT -i lo -m comment --comment "002 accept all loopback traffic" -j ACCEPT in respect of traffic to 127.0.0.1.

I don't think it's #6 again because my doctored ppd file passes ipptool:

~/git/PUPPET/puppet5(50-rolesprofiles) » cupstestppd site/profiles/files/ppds/KMC759ux.ppd
site/profiles/files/ppds/KMC759ux.ppd: PASS

In order to manage the pull printer queues.

Then I get the error message:

Debug: /Stage[main]/Cups::Server::Config/File[/etc/cups/lpoptions]: Nothing to manage: no ensure and the resource doesn't exist
Debug: Class[Cups::Server::Config]: The container Stage[main] will propagate my refresh event
Debug: Class[Cups::Server::Config]: The container Class[Cups::Server] will propagate my refresh event
Info: Class[Cups::Server::Config]: Scheduling refresh of Class[Cups::Server::Services]
Info: Class[Cups::Server::Services]: Scheduling refresh of Service[cups]
Debug: Executing: '/bin/systemctl is-active cups'
Debug: Executing: '/bin/systemctl is-enabled cups'
Debug: Executing: '/bin/systemctl is-active cups'
Debug: Executing: '/bin/systemctl show cups --property=NeedDaemonReload'
Debug: Executing: '/bin/systemctl restart cups'
Notice: /Service[cups]: Triggered 'refresh' from 1 event
Debug: /Service[cups]: The container Class[Cups::Server::Services] will propagate my refresh event
Debug: Class[Cups::Server::Services]: The container Stage[main] will propagate my refresh event
Debug: Class[Cups::Server::Services]: The container Class[Cups::Server] will propagate my refresh event
Debug: Class[Cups::Server]: The container Stage[main] will propagate my refresh event
Debug: Class[Cups::Server]: The container Class[Cups] will propagate my refresh event
Debug: Prefetching cups resources for cups_queue
Error: Could not prefetch cups_queue provider 'cups': IPP query 'ipptool -c ipp://localhost/ /dev/stdin' failed.
EXITCODE: 1
STDIN:
          {
            OPERATION CUPS-Get-Classes
            GROUP operation
            ATTR charset attributes-charset utf-8
            ATTR language attributes-natural-language en
            DISPLAY printer-name
            DISPLAY member-names
          }

STDOUT:

STDERR:
ipptool: Unable to connect to localhost on port 631 - Operation now in progress

Debug: Storing state
Debug: Pruned old state cache entries in 0.00 seconds
Debug: Stored state in 0.02 seconds
Error: Failed to apply catalog: IPP query 'ipptool -c ipp://localhost/ /dev/stdin' failed.
EXITCODE: 1
STDIN:
          {
            OPERATION CUPS-Get-Classes
            GROUP operation
            ATTR charset attributes-charset utf-8
            ATTR language attributes-natural-language en
            DISPLAY printer-name
            DISPLAY member-names
          }

STDOUT:

STDERR:
ipptool: Unable to connect to localhost on port 631 - Operation now in progress

Removing the queue with lpadmin -x doesn't solve the problem.

The same manifest on a Raspbian 9.4 Pi running Puppet 4.8.2 gives:

Debug: Exec[cups::papersize](provider=posix): Executing check 'cat /etc/papersize | grep -w A4'
Debug: Executing: 'cat /etc/papersize | grep -w A4'
Debug: Exec[cups::papersize](provider=posix): Executing 'paperconfig -p A4'
Debug: Executing: 'paperconfig -p A4'
Notice: /Stage[main]/Cups::Server::Config/Exec[cups::papersize]/returns: executed successfully
Debug: /Stage[main]/Cups::Server::Config/Exec[cups::papersize]: The container Class[Cups::Server::Config] will propagate my refresh event
Debug: /Stage[main]/Cups::Server::Config/File[/etc/cups/lpoptions]: Nothing to manage: no ensure and the resource doesn't exist
Debug: Class[Cups::Server::Config]: The container Stage[main] will propagate my refresh event
Debug: Class[Cups::Server::Config]: The container Class[Cups::Server] will propagate my refresh event
Info: Class[Cups::Server::Config]: Scheduling refresh of Class[Cups::Server::Services]
Info: Class[Cups::Server::Services]: Scheduling refresh of Service[cups]
Debug: Executing: '/bin/systemctl is-active cups'
Debug: Executing: '/bin/systemctl is-enabled cups'
Debug: Executing: '/bin/systemctl is-active cups'
Debug: Executing: '/bin/systemctl restart cups'
Notice: /Stage[main]/Cups::Server::Services/Service[cups]: Triggered 'refresh' from 1 events
Debug: /Stage[main]/Cups::Server::Services/Service[cups]: The container Class[Cups::Server::Services] will propagate my refresh event
Debug: Class[Cups::Server::Services]: The container Stage[main] will propagate my refresh event
Debug: Class[Cups::Server::Services]: The container Class[Cups::Server] will propagate my refresh event
Debug: Prefetching cups resources for cups_queue
Debug: Storing state
Debug: Stored state in 0.55 seconds
Error: Failed to apply catalog: IPP query 'ipptool -c ipp://localhost/ /dev/stdin' failed.
EXITCODE: 1
STDIN:
          {
            OPERATION CUPS-Get-Classes
            GROUP operation
            ATTR charset attributes-charset utf-8
            ATTR language attributes-natural-language en
            DISPLAY printer-name
            DISPLAY member-names
          }

STDOUT:

STDERR:
ipptool: Unable to connect to localhost on port 631 - Operation now in progress

threepistons commented 5 years ago

Regenerating types on the puppet master seems to make the Pis happy, but the Ubuntu systems are still giving the same error. ETA: nope, the Pis are happy the first run after removing the queue by hand and then make the error message forever after.

Uncommenting the PPD line about the filter and installing the filter Perl script makes no difference. I tend not to use the filter because it's for managing user boxes that we don't make use of because we have PCounter.

leoarnold commented 5 years ago

Have you tried this? https://github.com/leoarnold/puppet-cups#systemd-based-linux-distributions

threepistons commented 5 years ago

Thanks for the suggestion. It fails pdk test unit though. Where does the variable $::systemd in the workaround class come from?

threepistons commented 5 years ago

Ah, I see, it's a custom fact. Unit tests now happy.

threepistons commented 5 years ago

Manifest now saying:

class profiles::printing {

  # for convenience so I don't type the same thing over and over
  $ppdpath    = '/usr/share/cups/model'
  $filterpath = '/usr/lib/cups/filter'
  $filter     = 'KMbeuEmpPS.pl'
  $models     = ['KMC759ux.ppd']
  case $facts['lsbdistid'] {
    'Raspbian': {
      $packages = ['cups-bsd', 'a2ps']
    }
    default: {
      $packages = ['cups-bsd', 'a2ps', 'indicator-printers']
    }
  }

  include ksmb

  class { 'cups':
    default_queue => 'USB_Building_Printing',
    papersize     => 'A4',
  }

  include cups::workarounds::systemd_service_restart

  package { $packages:
    ensure => installed,
  }

  file { $ppdpath: ensure => directory }

  file { $filter:
    ensure  => file,
    replace => yes,
    mode    => '0755',
    path    => "${filterpath}/${filter}",
    source  => "puppet:///modules/profiles/ppds/${filter}"
  }

  $models.each | $model | {
    file { $model:
      ensure  => file,
      replace => yes,
      source  => "puppet:///modules/profiles/ppds/${model}",
      path    => "${ppdpath}/${model}",
      group   => 'lp',
      mode    => '0644',
    }
  }

  cups_queue { 'USB_Building_Printing':
    ensure      => 'printer',
    uri         => 'ksmb://cs-print/USB-BUILDING-PRINTING',
    ppd         => "${ppdpath}/KOC759UX.ppd",
    shared      => false,
    enabled     => true,
    accepting   => true,
    description => 'All Urban Sciences printers, swipe your campus card at any printer to collect your job.',
    options     => {
      'KMDuplex'      => '2Sided',
      'PageSize'      => 'A4',
      'InputSlot'     => 'Tray4',
      'PaperSources'  => 'PC215',
      'Finisher'      => 'FS533',
      'KOPunch'       => 'PK519-4',
      'SaddleUnit'    => 'None',
      'PrinterHDD'    => 'HDD',
      'SelectColor'   => 'Grayscale',
      'Model'         => 'C458',
      'TextPureBlack' => 'On',
    },
  }

}

Acceptance test results:

The Pis are happy.

But the Ubuntu 18.04s are still saying:


Debug: Prefetching cups resources for cups_queue
Error: Could not prefetch cups_queue provider 'cups': IPP query 'ipptool -c ipp://localhost/ /dev/stdin' failed.
EXITCODE: 1
STDIN:
      {
        OPERATION CUPS-Get-Classes
        GROUP operation
        ATTR charset attributes-charset utf-8
        ATTR language attributes-natural-language en
        DISPLAY printer-name
        DISPLAY member-names
      }

STDOUT:

STDERR: ipptool: Unable to connect to localhost on port 631 - Operation now in progress

Debug: Storing state Debug: Pruned old state cache entries in 0.00 seconds Debug: Stored state in 0.04 seconds Error: Failed to apply catalog: IPP query 'ipptool -c ipp://localhost/ /dev/stdin' failed. EXITCODE: 1 STDIN: { OPERATION CUPS-Get-Classes GROUP operation ATTR charset attributes-charset utf-8 ATTR language attributes-natural-language en DISPLAY printer-name DISPLAY member-names }

STDOUT:

STDERR: ipptool: Unable to connect to localhost on port 631 - Operation now in progress


The `Error: Could not prefetch cups_queue provider 'cups'` grabs my eye.

leoarnold commented 5 years ago

At prefetch, Puppet takes a look around to find which print queues are installed

https://github.com/leoarnold/puppet-cups/blob/b7c8299044f7f2aa74c77113abfe1b65b20b9606/lib/puppet/provider/cups_queue/cups.rb#L30-L36

It does so by running a few executions of the ipptool command line utility

https://github.com/leoarnold/puppet-cups/blob/b7c8299044f7f2aa74c77113abfe1b65b20b9606/lib/puppet_x/cups/shell.rb#L23

The problem is Unable to connect to localhost on port 631 - Operation now in progress, i.e. the CUPS service is currently not listening on port 631, probably because it is restarting in order to implement the changed configuration file.

The workaround drops in a config which tells systemd that the CUPS service is not ready until it is listening to that specific port

https://github.com/leoarnold/puppet-cups/blob/b7c8299044f7f2aa74c77113abfe1b65b20b9606/manifests/workarounds/systemd_service_restart.pp#L33-L39

Obviously, systemd is failing us here.

threepistons commented 5 years ago

I am firing up git's magic time machine because this talk of systemd is ringing a bell.

threepistons commented 5 years ago

Summary: running cupsd with -f doesn't help.

Some years ago, a previous author of Puppet code at our site added a "monkey patch" to systemd: /etc/systemd/system/cups.service

[Unit]
Description=CUPS Printing Service
Documentation=man:cupsd(8) man:cupsd.conf(5)
Requires=cups.socket
[Service]
ExecStart=/usr/sbin/cupsd -f
[Install]
Also=cups.socket cups.path
WantedBy=printer.target

I can't figure why this is needed and note that the cupsd manual says to use -l in the systemd unit, so I removed the monkey patch. When this problem started, I suspected the monkey patch to be a fix for it. Reinstating it doesn't resolve this problem though.

threepistons commented 5 years ago

My earlier manifests that don't use roles and profiles but used cups::resources in hiera were not affected. I'm going to try using cups::resources in Hiera within roles and profiles, somehow, but that will be tomorrow.

threepistons commented 5 years ago

I'm not sure why, but forcibly removing the monkey patch with file { '/etc/systemd/system/cups.service': ensure => absent } allows me to use cups_queue in my manifest, whereas not forcibly removing an already-present monkey patch (e.g. as a result of moving nodes from production to test without reinstalling the OS) forces me to use cups::resources hashes in Hiera. Even though cups::resources are turned into cups_queues.

Not complaining, my manifests now work.

threepistons commented 5 years ago

Sorry for taking up your time with something that turned out to be my problem.

leoarnold commented 5 years ago

Here's some food for thought, in case you want to dig deep: A plain

file { '/etc/systemd/system/cups.service':
# ....
}

is bound for trouble. You need something to trigger a systemctl daemon-reload to make systemd aware of the (changed) service file. Then again, I'm unsure about how Ubuntu 18.04 LTS runs CUPS. Maybe there's even a conflict of interest with an old /etc/init.d/cups script.

Furthermore the monkey-patch should be integrated into the Puppet dependency tree, c.f. https://github.com/leoarnold/puppet-cups#automatic-dependencies and https://github.com/leoarnold/puppet-cups/blob/b7c8299044f7f2aa74c77113abfe1b65b20b9606/manifests/init.pp#L96-L102

threepistons commented 5 years ago

You need something to trigger a systemctl daemon-reload to make systemd aware of the (changed) service file.

file { '/etc/.../cups.service': params => list } ~> class { 'cups': params => list } was in the old manifests to do exactly that, and is in the current manifest with ensure => absent to make the cups class reapply after removing the file, which causes at least one service reload based on the reports Foreman shows me.

Then again, I'm unsure about how Ubuntu 18.04 LTS runs CUPS. Maybe there's even a conflict of interest with an old /etc/init.d/cups script.

I will take a look on Monday (I use 18.04 at work) and report back. It's the least I can do.

threepistons commented 5 years ago

There is still cups and cups-browsed in /etc/init.d. The stock service files don't reference them. I removed the symlinks from /etc/rc5.d and rebooted and I still have CUPS running.

I suspect that the init.d files are packaged in CUPS for Systemd refuseniks and aren't relied on at all on Systemd systems.

leoarnold / puppet-cups

Could not prefetch cups_queue provider 'cups' on Ubuntu 18.04 #207