ipp-usb migration path - Githubissues

alexpevzner commented 2 years ago

Hi @tillkamppeter, @zdohnal, @debiantriage, @michaelrsweet and everybody else who may be interested,

I want to create a single discussion point for problems caused by migration from legacy USB drivers to the IPP-USB approach.

The running ipp-usb daemon known to be incompatible with legacy/proprietary printer and scanner drivers.

Although for most users IPP-USB works perfectly well, some people report printing or scanning breakage when system was upgraded, and ipp-usb was automatically installed.

This problem was discussed many times in many places, but we still have not found a good solution.

It was many times proposed, that ipp-usb will claim USB device access only when needed, and release it after use, or will not claim all the IPP-USB interfaces, allowing coexistence with legacy/proprietary drivers. I don't agree with this approach due to the following reasons:

From the ipp-usb perspective, it is hard to guess, when device usage was finished
Some devices don't like to be simultaneously accessed via IPP-USB and via legacy interface. See #26, for example
If ipp-usb will fail to claim USB access to the already announced but temporary released device, the behaviour will be quite confusing for users

My proposal is following:

On a newly installed system, ipp-usb always installed and enabled automatically
During upgrade of existent system, if there are already existent print queues, using the legacy USB access, user needs to switch to the ipp-usb explicitly
Some UI needs to be created to prompt the user to make such a decision
UI needs to be provided to let user to switch between ipp-usb and legacy approach at any time

However, I have no idea what mechanisms we already have to implement this proposal, and what missed parts needs to be created.

Any comments welcome!

tillkamppeter commented 2 years ago

On a newly installed system I fully agree that ipp-usb gets installed and so printers and scanners are used with it and CUPS' "usb" backend will not detect them and users print and scan driverless. HPLIP's "hp" backend would also need facilities to give way to IPP-over-USB and skip printers from being reported as supported if they support 7/1/4, as CUPS' "usb" backend does.

On upgrades I think we should (at least sooner or later) switch users into the driverless realm. For this we should create a script which goes through all locally created CUPS queues and on those which are USB (usb://..., hp:/usb/..., ???) check whether the printer does 7/1/4 and in this case remove its queue or change it to the correct driverless URI and driverless mode (-m everywhere). Classic drivers should be not included in the distro any more then (scenario especially also for CUPS 3.x or CUPS Snap as standard CUPS).

By the IPP-over-USB standard all printers which do 7/1/4 support driverless printing and scanning via IPP-over-USB.

A UI to switch between IPP-over-USB could be in the printer setup tool, at least as we are in the classic architecture (CUPS 2.x or older, CUPS not in a Snap). If we switch to the New Architecture (CUPS 3.x or CUPS Snap, classic drivers and PPDs not supported any more) classic drivers cannot get installed any more but Printer Applications are used instead. If no Printer Application is in use on a system, ipp-usb should take care of all 7/1/4 USB printers so that they just work. If a Printer Application provides classic support for a 7/1/4 printer and this has some useful advantages, the Printer Application ideally should communicate with ipp-usb to make ipp-usb stopping to provide support for this printer as long as the Printer Application is actually running. So we need to create such functionality in a way that it also works out of a Snap or from Snap to Snap (Printer Application Snap to ipp-usb Snap).

alexpevzner commented 2 years ago

On upgrades I think we should (at least sooner or later) switch users into the driverless realm

I don't think this is really needed. If somebody already has working system, it is better to leave it in this working state. And these old systems wil gradually go away naturally, as hardware where they installed does.

I believe, the best thing that we can do here, is to offer user to perform such an upgrade to the driverless printing/scanning, but don't perform this upgrade automatically, without an explicit user confirmation

By the IPP-over-USB standard all printers which do 7/1/4 support driverless printing and scanning via IPP-over-USB.

Not all, unfortunately. At least HP Laser MFP 135a known to offer 7/1/4 support, which actually doesn't work.

If a Printer Application provides classic support for a 7/1/4 printer and this has some useful advantages, the Printer Application ideally should communicate with ipp-usb to make ipp-usb stopping to provide support for this printer as long as the Printer Application is actually running.

It should require explicit user action. If printer application implicitly takes control, it can cause scanning regression. Unfortunately, at the SANE side there is no process to communicate with.

So we need to create such functionality in a way that it also works out of a Snap or from Snap to Snap (Printer Application Snap to ipp-usb Snap).

I don't quite understand how packages from different Snaps can communicate.

We already created mechanism, that prevents CUPS from stealing USB device, served by the ipp-usb, see #28 for details. There we use Avahi to organize communication between CUPS and ipp-usb.

zdohnal commented 2 years ago

Hi @alexpevzner ,

thank you for creating this issue!

It was many times proposed, that ipp-usb will claim USB device access only when needed, and release it after use, or will not claim all the IPP-USB interfaces, allowing coexistence with legacy/proprietary drivers. I don't agree with this approach due to the following reasons:
1. From the `ipp-usb` perspective, it is hard to guess, when device usage was finished

If I would follow IPP communication of IPP backend from CUPS, it sends IPP requests to the printer - so claim the device when the request comes, and release once the response is sent?

2. Some devices don't like to be simultaneously accessed via IPP-USB and via legacy interface. See [Query for "usb-max-interfaces" #26](https://github.com/OpenPrinting/ipp-usb/issues/26), for example

It would be interesting to see whether restarting ipp-usb (which IIUC should reclaim the USB interface as well - the similar way as plugging off and plugging in do...) helps as well - because if it helps, IMHO then claiming and releasing the interface change could help here.

3. If `ipp-usb` will fail to claim USB access to the already announced but temporary released device, the behaviour will be quite confusing for users

Such situation can happen only if someone is already printing to the device via other means (queue with classic driver/printer application/driverless permanent queue), which is not common on desktops (users usually print only to one queue at the time) and in enterprises IMO USB printers aren't used that much (because it would mean to have the print server right next to the device, which seems unlikely). The behavior you've described is exactly the situation which now happens for permanent queues with classic drivers and scanners :( .

My proposal is following:

1. On a **newly** installed system, `ipp-usb` always installed and enabled automatically

2. During upgrade of **existent** system, if there are already existent print queues, using the legacy USB access, user needs to switch to the `ipp-usb` **explicitly**

Ad 1,2 - idea: if the IPP-over-USB device appears on the USB interface, udev rule is triggered and ipp-usb is started, ipp-usb asks for local permanent queues from local cupsd, and does the best effort to check whether the new device already has a queue (f.e. by checking URI of local permanent queues, extract model name from it if possible) - if it does, log it and reject the device in ipp-usb until the permanent queue is removed? This would solve clear installations and upgrades as well.

3. Some UI needs to be created to prompt the user to make such a decision

4. UI needs to be provided to let user to switch between `ipp-usb` and legacy approach at any time

If there is going to be a UI for this, it will need to use:

A. an interface which will provide dBUS API which will be understood by all players - ipp-usb, printer applications in the future, current classic drivers, CUPS, sane-backends - which tells to a specific application which device to ignore, and implement this quirk support in all mentioned players.

or

B. the UI will write the settings into XDG directory and all players will check this file and act accordingly - again the support for this needs to distributed over all players.

However, I have no idea what mechanisms we already have to implement this proposal, and what missed parts needs to be created.

Any comments welcome!

My point here is if ipp-usb cannot be designed for claiming and releasing USB device, we are up to implementing quirks in all printer applications, sane-backends and classic driver packages (for now) for to be able to switch between driver/printer application and ipp-usb :( .

tillkamppeter commented 2 years ago

By the IPP-over-USB standard all printers which do 7/1/4 support driverless printing and scanning via IPP-over-USB.

Not all, unfortunately. At least HP Laser MFP 135a known to offer 7/1/4 support, which actually doesn't work.

It seems that all the "HP Laser" (not "LaserJet") printers are re-brands from HP's acquisition of Samsung's printer division and those are all 7/1/4 but do not fulfill driverless IPP standards. see also this Ubuntu bug report on the HP Laser 107a.

zdohnal commented 2 years ago

Hi @tillkamppeter ,

On a newly installed system I fully agree that ipp-usb gets installed and so printers and scanners are used with it and CUPS' "usb" backend will not detect them and users print and scan driverless. HPLIP's "hp" backend would also need facilities to give way to IPP-over-USB and skip printers from being reported as supported if they support 7/1/4, as CUPS' "usb" backend does.

unfortunately sometimes driverless configuration doesn't provide all options or doesn't work every time, so people tend to switch back to drivers (to printer applications in the future), so IMO the hard removal from listing will cause feature loss.

On upgrades I think we should (at least sooner or later) switch users into the driverless realm.

+1

For this we should create a script which goes through all locally created CUPS queues and on those which are USB (usb://..., hp:/usb/..., ???) check whether the printer does 7/1/4 and in this case remove its queue or change it to the correct driverless URI and driverless mode (-m everywhere). Classic drivers should be not included in the distro any more then (scenario especially also for CUPS 3.x or CUPS Snap as standard CUPS).

Here is the problem - for finding out capabilities of the printer, you have to rely on printer is alive during upgrade - which can be done by udev event (which is problem in snap :( - so there it would need to check whether a service appears on mDNS and I don't know if there is a mechanism which can trigger a service once service appears on mDNS communication...) - so it would be great if the deed was done in as a part of ipp-usb:

start ipp-usb
check quirks
check the current local queues
- if similar queue exists, remove the original one
add the device into ipp-usb

Once this is done once, ipp-usb could leave a file in f.e. /var/cache/ipp-usb, which will indicate the migration was done for the queue, and prevent the multiple running on the same device.

But still we will need to implement blacklists in sane-backends to prevent showing broken devices (because the classic scanners are broken as well because they cannot access USB port taken by ipp-usb).

If a Printer Application provides classic support for a 7/1/4 printer and this has some useful advantages, the Printer Application ideally should communicate with ipp-usb to make ipp-usb stopping to provide support for this printer as long as the Printer Application is actually running. So we need to create such functionality in a way that it also works out of a Snap or from Snap to Snap (Printer Application Snap to ipp-usb Snap).

The same way we need to take care of sane-backends - if user wants to handle scanning functionality via classic driver, it should be possible to tell it to ipp-usb and vice versa, to prevent non-functional scanner objects reported by SANE backends.

zdohnal commented 2 years ago

On upgrades I think we should (at least sooner or later) switch users into the driverless realm

I don't think this is really needed. If somebody already has working system, it is better to leave it in this working state. And these old systems wil gradually go away naturally, as hardware where they installed does.

Unfortunately there are servers which keep their configuration along several OS versions, so an upgrade path is needed for such cases :(

debiantriage commented 2 years ago

@tillkamppeter expresses the view that

On upgrades I think we should (at least sooner or later) switch users into the driverless realm

@alexpevzner countered with

I don't think this is really needed. If somebody already has working system, it is better to leave it in this working state. And these old systems wil gradually go away naturally, as hardware where they installed does.

Distros do all sorts of things on upgrade without offering an alternative to a user. The change from sysvinit to systemd is a classic example. It went from one working system to another working system.

Debian stable (bullseye) installs ipp-usb as a recommended package to cups-daemon. That was decided by a previous maintainer (OdyX). I think the idea was to get driverless printing central, especially when looking ahead to CUPS 3.0. Some pain now, but a payoff in the future.

I believe, the best thing that we can do here, is to offer user to perform such an upgrade to the driverless printing/scanning, but don't perform this upgrade automatically, without an explicit user confirmation

For me - automatic upgrade every day. Get it over and done with. Deal with the fallout, if any. Debian provides extensive advice via its Release Notes. (OK, I know...). We need legacy/classic system thinking to die, not fade away.

I have not seen any undue kickback in Debian bullseye over ipp-usb. Ubuntu 22.04 will be interesting; their users tend to be more vociferous.

zdohnal commented 2 years ago

Debian stable (bullseye) installs ipp-usb as a recommended package to cups-daemon.

That's the same thing I've attempted to do recently in Fedora - adding ipp-usb as a weak dependency, which is brought into the dnf transaction by default. The resulting situation ended as a Fedora 36 final blocker, because (quoting) "All elements of the default panel (or equivalent) configuration in all release-blocking desktops must function correctly in typical use." - because available scanners and installed printers suddenly stopped to work.

That was decided by a previous maintainer (OdyX). I think the idea was to get driverless printing central, especially when looking ahead to CUPS 3.0. Some pain now, but a payoff in the future.

I believe, the best thing that we can do here, is to offer user to perform such an upgrade to the driverless printing/scanning, but don't perform this upgrade automatically, without an explicit user confirmation

For me - automatic upgrade every day. Get it over and done with. Deal with the fallout, if any. Debian provides extensive advice via its Release Notes. (OK, I know...). We need legacy/classic system thinking to die, not fade away.

Unfortunately some users tend to switch back to drivers, because driverless standards don't support some of the features, because the device doesn't advertise it. I don't know of a way how to ask vendors for updating their firmware to make driverless standards a full fledged citizen without some paid subscription for support (which most ordinary users don't have), so common users can get this firmware update and use the options available to a driver via driverless means.

I have not seen any undue kickback in Debian bullseye over ipp-usb. Ubuntu 22.04 will be interesting; their users tend to be more vociferous.

In Fedora there was commotion over this change due the breakages it caused, as I mentioned above, so we will see.

If the ipp-usb design cannot be changed, then my idea how to tackle this:

for already created print queues create a migration path (maybe in ipp-usb itself, since it is started once the USB device is up and running, get local CUPS queues, parse their uri and check whether it is the same device, if so, remove the queue)
for scanners and to prevent discovery of IPP-over-USB device by other backends if ipp-usb is using them implement an attempt to open and close found USB device - in case ipp-usb already claimed the device, the opening the port will fail and the device won't be shown. If ipp-usb doesn't claim the device, the device will be accessible and will be shown. This will have to implemented in CUPS, hplip, gutenprint, sane-backends.

WDYT?

alexpevzner commented 2 years ago

@zdohnal,

Dynamic attach/detach to the USB device, in theory should work. But practically, due to firmware bugs, it will not work always. For some people it will work smoothly, some others will report sporadic errors like "printing stopped working after a scan attempt", or visa versa.

Please note, that migration to driverless approach also, in theory, should work smoothly, but practically, as now we know, there are exceptions.

I expect that group of the users, affected by the problems, caused by dynamic attach/detach to the USB device will be comparable in size, if not greater, that the group of the users, affected by the problems, caused by driverless transitions, though it will be different groups.

Please note, as we can't test on a big variety of hardware, here we only can speculate.

This is why I'm thinking quite sceptical about dynamic USB device attachment at least as a method to minimize the amount of frustrated users. May be, it should be added as a configurable feature, disabled by default, for advanced users (at least, most of them know what are they doing).

I like the idea that ipp-usb should skip devices, if CUPS queue already exists. It should prevent automatic upgrade for people, who already have working installation. However, it would be even better, if these users will be explicitly asked for the need of upgrade (and as ipp-usb doesn't have user interface, it should be implemented in somewhere around the desktop). But even without UI, it will probably will be better, that implicit automatic upgrade.

BTW, do we have some mechanism, that allows ask user a question during or after the system upgrade?

Somebody can point me into the right direction, how can I obtain a list of CUPS queues programmatically? I'm not very familiar with the CUPS API. Also, what do I have for the already configured devices, only model name, or some part of their USB identity (bus/port number, device ID) is also known?

For scanners, I guess most drivers don't attempt to communicate with the device before actual scan attempt. I.e., device added to the list based on scanning the USB device, and when device is attempted to be actually used, error occurs. The same situation often happens and reported, when user has not enough permissions to access the USB device.

Changing something at SANE side is hopeless: too many backends, many of them unmaintained, many of them are proprietary and out of our control.

alexpevzner commented 2 years ago

For this we should create a script which goes through all locally created CUPS queues and on those which are USB (usb://..., hp:/usb/..., ???) check whether the printer does 7/1/4 and in this case remove its queue or change it to the correct driverless URI and driverless mode (-m everywhere). Classic drivers should be not included in the distro any more then (scenario especially also for CUPS 3.x or CUPS Snap as standard CUPS).

Here is the problem - for finding out capabilities of the printer, you have to rely on printer is alive during upgrade - which can be done by udev event (which is problem in snap :( - so there it would need to check whether a service appears on mDNS and I don't know if there is a mechanism which can trigger a service once service appears on mDNS communication...) - so it would be great if the deed was done in as a part of ipp-usb:

ipp-usb doesn't have UI, so it can't communicate with the human users.

What about the following idea:

If installer, during the system upgrade, sees a permanent USB print queue, it offers user something like: "This device may support driverless printing. Should it be upgraded to the driverless printing, if later we will see that this is possible?".
If user chooses upgrade, the queue is marked somehow, so ipp-usb can later read this flag
When printer is actually attached, if queue exists and flagged for upgrade, ipp-usb removes the queue and serves printer by itself. Otherwise, if queue exists but not flagged, ipp-usb ignores the device.

zdohnal commented 2 years ago

@alexpevzner

IMHO we can only do the best effort regarding migrations and co-existing with drivers/printer applications - like it is not in our power to be able to parse correctly every uri which exists and can be generated by special backends like hp/usb/gutenprintXY+usb/scanner backends or other 3rd party drivers. There will be always some devices with broken USB implementations :( , but there should be an effort to make the process easy for some of them and prevent possible problems (like installations of non-functional printer applications or showing non-functional scanners).

I expect that group of the users, affected by the problems, caused by dynamic attach/detach to the USB device will be comparable in size, if not greater, that the group of the users, affected by the problems, caused by driverless transitions, though it will be different groups.

You're right - although there are two things to consider:

printer driver -> driverless/printer application are going to happen, so somewhere along the road there will have to be a migration path for better UIX and it would help a great deal if we knew that certain amount of driverless USB printers cannot work via driverless and need to have a printer application for its usage.
in case there are problem with driverless features, IMHO it would be great if we caught it early and could create a quirks for the affected models, so other users will know whether they can use ipp-usb, or use a classic driver (or printer application in the future)

IMHO there is a bigger benefit in tackling this some way than letting it be due possible USB firmware bugs :(

Please note, as we can't test on a big variety of hardware, here we only can speculate.

As it is always, in the land of printers and scanners :) - I can offer testing on one IPP-over-USB device I have at work :D .

This is why I'm thinking quite sceptical about dynamic USB device attachment at least as a method to minimize the amount of frustrated users. May be, it should be added as a configurable feature, disabled by default, for advanced users (at least, most of them know what are they doing).

Unfortunately the common users are affected by this the most - so any non-default option can be difficult - we're making printing just work :)

I like the idea that ipp-usb should skip devices, if CUPS queue already exists. It should prevent automatic upgrade for people, who already have working installation.

That could be done, however it would be great if we tried to migrate as soon as we can - and gather data which devices are broken with ipp-over-usb.

However, it would be even better, if these users will be explicitly asked for the need of upgrade (and as ipp-usb doesn't have user interface, it should be implemented in somewhere around the desktop).

Maybe configuration option or directive in ipp-usb.conf which would turn on migrations for non-UI solution?

BTW, do we have some mechanism, that allows ask user a question during or after the system upgrade?

I'm not sure about other distros, but we in Red Hat started to use Leapp system, which uses Leapp actors from leapp-repository for RHEL/CentOS upgrades - this system will scan your machine and will make report what needs to be changed and what will be changed automatically - so here you can inform user about something.

Somebody can point me into the right direction, how can I obtain a list of CUPS queues programmatically? I'm not very familiar with the CUPS API. Also, what do I have for the already configured devices, only model name, or some part of their USB identity (bus/port number, device ID) is also known?

cupsEnumDests() for getting all local queues and cupsGetNamedDest() for getting a specified destination should do the trick - see the manual. Or you can use IPP API in CUPS, since you care only about permanent queues (cupsEnumDests() does mDNS discovery as well, so it returns temp queues as well). Regarding what is known - in most cases it is model name, interface number and serial number - all those are included in device-uri IPP attribute, which is parsed by USB backend to find out the correct USB port they're connected to.

For scanners, I guess most drivers don't attempt to communicate with the device before actual scan attempt. I.e., device added to the list based on scanning the USB device, and when device is attempted to be actually used, error occurs. The same situation often happens and reported, when user has not enough permissions to access the USB device.

Changing something at SANE side is hopeless: too many backends, many of them unmaintained, many of them are proprietary and out of our control.

Unfortunately if we aren't going to change ipp-usb design, we can just mitigate scanner breakage for such device - either by trying to open and then close the port (and listing the device if USB port accessible), or implementing quirks mechanism for sane-backends, hplip, gutenprint (and possible some UI above them to add those quirks...). IMO in the scanner territory the most affected are pixma from sane-backends and hpaio from hplip, so we can implement something for them first, and then see what other backends conflicts with ipp-usb.

For this we should create a script which goes through all locally created CUPS queues and on those which are USB (usb://..., hp:/usb/..., ???) check whether the printer does 7/1/4 and in this case remove its queue or change it to the correct driverless URI and driverless mode (-m everywhere). Classic drivers should be not included in the distro any more then (scenario especially also for CUPS 3.x or CUPS Snap as standard CUPS).

Here is the problem - for finding out capabilities of the printer, you have to rely on printer is alive during upgrade - which can be done by udev event (which is problem in snap :( - so there it would need to check whether a service appears on mDNS and I don't know if there is a mechanism which can trigger a service once service appears on mDNS communication...) - so it would be great if the deed was done in as a part of ipp-usb:

ipp-usb doesn't have UI, so it can't communicate with the human users.

As I mentioned above - if the migration option was settable in ipp-usb.conf, we can do this without UI for now. Then the migration process would start once the device is turned on and plugged in.

AFAIU the problem is in SNAP that udev rules don't work - in that case someone else, not ipp-usb, will have to listen on mDNS traffic and check whether some device appeared on localhost, get its info and start parsing it with current local permanent queues...

What about the following idea:

1. If installer, during the system upgrade, sees a permanent USB print queue, it offers user something like: "This device may support driverless printing. Should it be upgraded to the driverless printing, if later we will see that this is possible?".

AFAIK you cannot be sure that the device connected to the permanent queue is capable of using driverless, unless it is turned on or you have a database of driverless USB printers...

alexpevzner commented 2 years ago

Hi!

as a first step to simplify migration, I've released ipp-usb version 0.9.22 with the following important changes:

If device initialization fails due to the protocol-level errors, ipp-usb detaches from the device
added ipp-usb status command which prints current per-device status to console (output looks quite similar to the ipp-usb check, but check generates its output based purely on scanning the USB bus, while status displays the actual status of the running daemon)

OpenPrinting / ipp-usb

ipp-usb migration path #50