search

alexpevzner / sane-airscan

Scanner Access Now Easy - universal driver for eSCL (Apple AirScan) and WSD
Other
278 stars 39 forks source link

Device returns 300dpi image when airscan asks for 600dpi image #57

Closed tangyanli closed 4 years ago

tangyanli commented 4 years ago

Hi, Alexander:

I encountered a device firmware bug. When airscan asks for 600dpi image, the device actually returns 300dpi image. So when I use simple-scan and set the DPI=600/1200/2400, the acquired image is decreased to 25%. I just ask for a help. Could airscan correct the DPI according to the returned image?

simple-scan-Canon-iR2625-2630.log

alexpevzner commented 4 years ago

Hi Tess,

does it correctly handle other resolutions (100/200/300/400)? If in dоubt, you may guess by returned image size in pixels, which is about 3507x4961 in 300 DPI mode.

Is this problem reproduced in a Grayscale mode too?

tangyanli commented 4 years ago

Hi, Alexander

I tested the 100/300 DPI, the device handle them correctly. I haven't tested the Grayscale yet. I thought the device actually doesn’t support 600DPI, so the problem should be reproduced in a Grayscale mode too. I will test the Grayscale later.

tangyanli commented 4 years ago

Hi, Alexander

I did more test. And the test result are summarized as below: Device can handle 100/200/300 dpi correctly in both color and grayscale mode. But the device returns 300dpi image when airsan asks for 400/600dpi image in both color and grayscale mode.

In the short, the MaxOpticalResolution of the device should be 300dpi.

 <scan:MaxOpticalXResolution>600</scan:MaxOpticalXResolution>
 <scan:MaxOpticalYResolution>600</scan:MaxOpticalYResolution>

Is it possible to correct the DPI at airscan side? And do you need more log files?

alexpevzner commented 4 years ago

Hi Tess,

I've requested to perform these tests to be 100% sure that it is not caused by sane-airscan bug.

The simplest solution could be to enforce upper limit, based on a device model name. Are you sure 300 DPI is a physical limitation, and will not be fixed by a later Canon firmware? Their own official scanning tool, is it able to scan at 600 DPI?

alexpevzner commented 4 years ago

BTW, does it have the same limitation when using WSD?

To enable WSD, add the following lines to the /etc/sane.d/airscan.conf:

[options]
protocol = manual

The scanner should appear twice in the device selection dialog: one entry for eSCL and second for WSD.

In simple-scan this dialog is well-hidden behind its menus, so I suggest to use xsane for testing

tangyanli commented 4 years ago

Hi, Alexander

The “Scan and Send” settings on the device panel shows it supports 600 DPI (even 1200DPI). I haven’t test the Canon official scanning tool, so I am not sure 300 DPI is a physical limitation.

I also tested the WSD, the maximum DPI in the log file is 300 DPI, so this issue can’t be reproduced on WSD mode.

Maybe you are right, I should report the bug to Canon device department..., this would be a difficult way. Or maybe a new version firmware already exists.

Can I update my local source to enforce upper limit based on the device model name? If can, I need your help again to tell me how to update the code.

tangyanli commented 4 years ago

I prefer escl because the WSD mode can’t be used when the device is connected by USB.

alexpevzner commented 4 years ago

Hi Tess,

I think, complaining to Canon will make sense in any case. You can try to reproduce this issue, using Mopria scan for Android (it also uses eSCL). At this case, it will be difficult for Canon to tell that the problem is in our open source (read "unsupported") driver.

Let me think a little bit how to work around this problem in more-or-less generic way. If Canon will eventually fix their firmware, I don't want to prohibit high-resolution scan for users of such an upgraded firmware.

Probably, I'll add some option to the configuration file

tangyanli commented 4 years ago

Hi, Alexander

Yes, you are right! I didn't consider it well. I was too anxious to find a workaround :-)

tangyanli commented 4 years ago

Hi, Alexander

I don't have Android system, so I am not able to test the Mopria. Just now I tested the Canon official scanning tool (ScanGear) on Windows, the maximum DPI that can be set on the tool is 300 DPI.

It will be grateful if any workaround is supplied by airscan. If not, it doesn't matter I will work living with this issue.

Let me think a little bit how to work around this problem in more-or-less generic way.

tangyanli commented 4 years ago

Hi, Alexander:

I closed this issue. Thank you anyway.

BTW, I tested airscan (eSCL mode) on Canon iR-ADV C5550/5560 that supports real 600 DPI resolution. All things are OK.

alexpevzner commented 4 years ago

Hi Tess,

I'm still thinking about this issue, so do not rush to give up.

What confuses me, is the fact that all Canon iR2625/iR2530 specifications that I was able to find consistently claim that device can scan up to 600 DPI, while in all your experiments it was able to scan only up to 300.

Can it be so that scanning at 600 DPI needs to be explicitly enabled in the device's web console or on a front panel? I've tried to guess myself, but documentation is only on web, split into small pages, very hard to search, so I could miss something. May be, you'll look to the device's web console by yourself?

tangyanli commented 4 years ago

Hi, Alexander

Thanks for you reminder. I checked the user manual, it really records that the device supports 600 DPI scanning. So it will be easy to report this issue to Canon now.

But I might find another issue... :-( For the WSD mode, Canon say,

The available scanning size is A4(LTR). Note that even if you place an original larger than A4(LTR), only A4(LTR) size is scanned.

Refer URI: https://oip.manual.canon/USRMA-4831-zz-CS-357-enUS/contents/devu-scan-rscan-wsd-wsdscan.html

On windows, I cannot select Paper Size when using the WSD to scan. On Linux, when I chose the WSD mode and set the PageSize=A4/A5/A6, the simple-scan crashes. But it's OK if set the PageSize=A3/Letter/Legal.

simple-scan-Canon-MF745C-746-(cf:9d:6c).log

Canon MF745C/746 (cf:9d:6c): device created
Canon MF745C/746 (cf:9d:6c): DEVICE_STM_OPENED->DEVICE_STM_PROBING
Canon MF745C/746 (cf:9d:6c): using protocol "WSD"
Canon MF745C/746 (cf:9d:6c): HTTP POST http://172.16.9.44/wsd/scan
Canon MF745C/746 (cf:9d:6c): HTTP POST http://172.16.9.44/wsd/scan: 200 OK
Canon MF745C/746 (cf:9d:6c): new decoder: jpeg
Canon MF745C/746 (cf:9d:6c): DEVICE_STM_PROBING->DEVICE_STM_IDLE
Canon MF745C/746 (cf:9d:6c): sane_open("w6:Canon MF745C/746 (cf:9d:6c)"): Success
Canon MF745C/746 (cf:9d:6c): sane_start()
Canon MF745C/746 (cf:9d:6c): DEVICE_STM_IDLE->DEVICE_STM_SCANNING
Canon MF745C/746 (cf:9d:6c): PROTO_OP_SCAN: submitting: attempt=0
Canon MF745C/746 (cf:9d:6c): HTTP POST http://172.16.9.44/wsd/scan
Canon MF745C/746 (cf:9d:6c): HTTP POST http://172.16.9.44/wsd/scan: 200 OK
Canon MF745C/746 (cf:9d:6c): PROTO_OP_SCAN: decoding
Canon MF745C/746 (cf:9d:6c): PROTO_OP_SCAN: decoded: status="Success" next=PROTO_OP_LOAD delay=0
Canon MF745C/746 (cf:9d:6c): PROTO_OP_LOAD: submitting: attempt=0
Canon MF745C/746 (cf:9d:6c): HTTP POST http://172.16.9.44/wsd/scan
Canon MF745C/746 (cf:9d:6c): HTTP POST http://172.16.9.44/wsd/scan: 200 OK
Canon MF745C/746 (cf:9d:6c): PROTO_OP_LOAD: decoding
Canon MF745C/746 (cf:9d:6c): PROTO_OP_LOAD: decoded: status="Success" next=PROTO_OP_FINISH delay=0
Canon MF745C/746 (cf:9d:6c): DEVICE_STM_SCANNING->DEVICE_STM_DONE
malloc(): corrupted top size
Aborted (core dumped)
alexpevzner commented 4 years ago

Hi Tess,

may I ask you to collect a crash stack?

To do so, you need to rebuild sane-airscan by yourself (binary packages come without debug info). Then, I usually don't bother with install, but simply replace installed /usr/lib/x86_64-linux-gnu/sane/libsane-airscan.so.1 (the actual path depends on your distro and arch) with the symbolic link to the libsane-airscan.so.1 in my build directory.

Then please reproduce a crash, and you will get core file. Then, please call gdb with two parameters: path to the crashed program and path to the core file. Then issue bt command and send me, please, its output.

tangyanli commented 4 years ago

All right, I will send you the crash stack tomorrow.

alexpevzner commented 4 years ago

OK, I'm waiting

alexpevzner commented 4 years ago

To save one round-trip, I would also ask you to retest with https://github.com/alexpevzner/sane-airscan-unstable

It looks like buffer overflow problem. In the sane-airscan-unstable memory management was significantly reworked, so there is a chance that this bug is fixed there.

But because there is also a chance, that this bug is not fixed, but only hidden in the sane-airscan-unstable, if problem is not reproduced in the sane-airscan-unstable, I still want to look to the sane-airscan crash stack.

In another words, if problem exists in sane-airscan, but doesn't exist in sane-airscan-unstable, I want to see sane-airscan stack. If problem exists in both versions, I'd prefer to debug sane-airscan-unstable.

tangyanli commented 4 years ago

Hi, Alexander

This issue can be reproduced in the sane-airscan-unstable for both Ubuntu19.10 & Debian10. But the phenomenon is different with the sane-airscan: The simple-scan crashes only when I set the PaperSize = A6; it is OK when I set the PageSize=A3/A4/A5/Letter/Legal.

I debuged the sane-airscan-unstable , and the call stack is as below. It seems we need to debug further , but I am sorry I don't know how to debug further.

Thread 9 "scan-thread" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffd620e700 (LWP 22374)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff6a783eb in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff6a57899 in __GI_abort () at abort.c:79
#2  0x00007ffff6ac238e in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff6beb3a5 "%s\n")
    at ../sysdeps/posix/libc_fatal.c:181
#3  0x00007ffff6aca4dc in malloc_printerr (str=str@entry=0x7ffff6bed480 "malloc(): invalid size (unsorted)")
    at malloc.c:5332
#4  0x00007ffff6acd284 in _int_malloc (av=av@entry=0x7fffd0000020, bytes=bytes@entry=7441) at malloc.c:3736
#5  0x00007ffff6ad0d45 in __libc_calloc (n=<optimised out>, elem_size=<optimised out>) at malloc.c:3428
#6  0x00007ffff7d955b1 in g_malloc0 () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#7  0x000055555558c301 in  ()
#8  0x00007ffff7db9181 in  () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#9  0x00007ffff69a4669 in start_thread (arg=<optimised out>) at pthread_create.c:479
#10 0x00007ffff6b542b3 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb)
alexpevzner commented 4 years ago

Hi Tess,

now I can reproduce, thank a lot! Will fix tomorrow.

alexpevzner commented 4 years ago

P.S. The bug is in image clipping. With eSCL clipping is done in hardware, but with WSD at leas on my scanner image clipping is broken, so I do it in the driver: request a full scan and then clip to requested region. And I've somewhere mistaken in this code.

tangyanli commented 4 years ago

So great to hear that you have reproduced the issue. In fact, I am now trying to find the code caused the crash by using a stupid method: output the log information... Now, I can stop :-)

alexpevzner commented 4 years ago

Believe or not, I'm always debugging using logs, and only use gdb to inspect core dumps

alexpevzner commented 4 years ago

Hi Tess,

fixed in -unstable, you may want to retest

tangyanli commented 4 years ago

Hi, Alexander

Perfect! I have retested, it's OK for all page size. Thanks!

About the iR2625/iR2530, I will let you know if I get some feedback from Canon. But I don't know how long I have to wait.

tangyanli commented 4 years ago

Hi, Alexander

I read the user guide again. I noticed there is a tip under the Scanner capability section: The 400x400dpi, 600x600dpi are available only under Black and Write Mode.

So I retested the Canon official scanning tool (ScanGear) on Windows: Color Mode has four selections: "Black and White", "Grayscale" , "Color" and "Auto (Color/B&W)". The tip is right, the 400dpi and 600dpi display when I select the Color Mode as "Black and Write". On other cases, the 400dpi and 600dpi are hidden.

Below is my assumption.

For sane-airscan eSCL mode, the device returns three supported color mode. The 400dpi and 600dpi is available only for the BlackAndWhite1 color mode. But I cannot select the BlackAndWhite1 by using xsane and simple-scan, so I am unable to confirm this assumption.

<scan:ColorModes>
     <scan:ColorMode>BlackAndWhite1</scan:ColorMode>
     <scan:ColorMode>Grayscale8</scan:ColorMode>
     <scan:ColorMode>RGB24</scan:ColorMode>
</scan:ColorModes>

For sane-airscan wsd mode, the device only returns one supported color mode: RGB24. So the device doesn't return 600dpi resolution.

<wscn:PlatenColor>
      <wscn:ColorEntry>RGB24</wscn:ColorEntry>
</wscn:PlatenColor>
<wscn:PlatenResolutions>
   <wscn:Widths>
         <wscn:Width>300</wscn:Width>
   </wscn:Widths>
   <wscn:Heights>
          <wscn:Height>300</wscn:Height>
   </wscn:Heights>
</wscn:PlatenResolutions>

So now I think this is device specification / limitation...

alexpevzner commented 4 years ago

Hi Tess,

I think, I'll add DPI limit based on device model name. Could you please direct me to the relevant place in the manual, I prefer to keep things like this more or less documented?

tangyanli commented 4 years ago

Hi, Alexander

The guide is written in Chinese, is it OK? URI is as below, at Page 1035 https://gdlp01.c-wss.com/gds/3/0300037423/02/iR2645_UsersGuide_SC_1.pdf

In Chinese, it is written as

*1 仅在选择<黑白二值>时可用。

I translated it in English: They are available only under Black and Write Mode.

tangyanli commented 4 years ago

I found the English version user guide: https://oip.manual.canon/USRMA-4209-zz-CSL-2600-enUV/contents/devu-apdx-sys_spec-send.html?search=600

tangyanli commented 4 years ago

Hi, Alexander

Here is another case. I am now using xsane to scan from Canon MB5480 ADF (by eSCL mode). At one moment, I forget to put the paper on ADF, then the xsane crashed. And the call stack is as below (I am glad I learned the gdb some days ago...), and at this time the device panel displayed error message which reminded me that "The ADF is empty, please click OK and try again after you putting on paper".

Thread 1 "xsane" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff6f93535 in __GI_abort () at abort.c:79
#2  0x00007ffff6fea508 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff70f528d "%s\n")
    at ../sysdeps/posix/libc_fatal.c:181
#3  0x00007ffff6ff0c1a in malloc_printerr (str=str@entry=0x7ffff70f343b "free(): invalid pointer") at malloc.c:5341
#4  0x00007ffff6ff242c in _int_free (av=<optimized out>, p=<optimized out>, have_lock=<optimized out>) at malloc.c:4165
#5  0x00007fffee34d711 in http_data_unref (data=0x7fffee36f5b0 <http_query_no_data>) at airscan-http.c:1626
#6  0x00007fffee34b593 in device_read_next (dev=0x55555591f608) at airscan-device.c:1618
#7  device_read (dev=dev@entry=0x55555591f608, data=data@entry=0x7ffffffed9a0 "\360&tUUU", max_len=max_len@entry=65536, 
    len_out=len_out@entry=0x7ffffffdcd9c) at airscan-device.c:1755
#8  0x00007fffee341f9c in sane_read (handle=0x55555591f608, data=0x7ffffffed9a0 "\360&tUUU", max_len=65536, len=0x7ffffffdcd9c)
    at airscan.c:304
#9  0x00005555555d38e5 in ?? ()
#10 0x00007ffff788e1ae in ?? () from /lib/x86_64-linux-gnu/libgdk-x11-2.0.so.0
#11 0x00007ffff7740dd8 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#12 0x00007ffff77411c8 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#13 0x00007ffff77414c2 in g_main_loop_run () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#14 0x00007ffff7a508e7 in gtk_main () from /lib/x86_64-linux-gnu/libgtk-x11-2.0.so.0
#15 0x00005555555e4c32 in ?? ()
#16 0x0000555555575fa5 in main ()
(gdb)
alexpevzner commented 4 years ago

Hi Tess,

do you have a protocol trace from this crash case?

tangyanli commented 4 years ago

Hi, Alexander:

I just send you the protocol trace by e-mail.

tangyanli commented 4 years ago

Hi, Alexander

It's weekend again. About the iR2625 and the crash case, if you fix it so quickly and don't hurry to get the test result, I would test it on next Monday; but if you hurry to get the test result, I also can test on it tomorrow.

I think, I'll add DPI limit based on device model name.

tangyanli commented 4 years ago

Hi, Alexander

I tested the new code, the crash case is fixed. Thank you so much!💐

alexpevzner commented 4 years ago

Hi Tess,

please test resolutions issue

tangyanli commented 4 years ago

Hi, Alexander

Thanks so much for the workaround. The printer is not around today, I will reply you on tomorrow morning.

alexpevzner commented 4 years ago

Hi Tess,

No problem, I can wait. You don't need to interrupt your weekend.

tangyanli commented 4 years ago

Hi, Alexander

I have tested the resolution issue. The issue is fixed. Thanks so much for the workaround. And the devices mentioned in this issue are summarized as below:

iR 2625/2630: Both eSCL mode and WSD mode are OK. MB5480: eSCL mode is OK, and I cann't find the WSD setting on device panel & web console. iR-ADV C5550/5560: eSCL mode is OK. But I was unable to test the WSD mode due to below error.

00:00:09.976: Canon iR-ADV C5550/556_9_21: device_start_wait: Device busy 00:00:09.977: Canon iR-ADV C5550/556_9_21: sane_start(): Device busy 00:00:09.977: Canon iR-ADV C5550/556_9_21: sane_close()

alexpevzner commented 4 years ago

Hi Tess,

thank a lot for update. Now I can release 0.99.14

MB5480 requires WSD to be enabled in the web console: https://ugp01.c-ij.com/ij/webmanual/Manual/All/MB5400%20series/EN/UG/ug_changing_settings0300.html

And iR-ADV C5550/5560 too: https://oip.manual.canon/USRMA-4409-zz-CS-5500-enGB/contents/devu-scan-rscan-wsd.html

For now I will not mark these devices as supporting WSD, but if you will test them in this mode, I will update the README.md

tangyanli commented 4 years ago

Hi, Alexander

Sorry for that I am still unable to use the WSD mode on MB5480 & iR-ADV C5550/5560. For MB5480, after enabling the "WSD scan" on web console, I configure the "Canon MB5400_wsd" in the airscan.conf as below:

"Canon MB5400_wsd" = http://172.16.4.106:80/wsd/scan, WSD

And the airscan outputs below error message:

Canon MB5400_wsd: HTTP POST http://172.16.4.106:80/wsd/scan: got response headers (404) Canon MB5400_wsd: HTTP POST http://172.16.4.106:80/wsd/scan: 404 Not Found Canon MB5400_wsd: HTTP POST http://172.16.4.106:80/wsd/scan: Not Found Canon MB5400_wsd: scanner capabilities query: Not Found

For iR-ADV C5550/5560, the "WSD scan" is always enable. Even on windows, the WSD fails to scan. It seems my device has some problem.

tangyanli commented 4 years ago

It's my fault.

After updating the airscan.conf as below, MB5480 wsd mode works.

http://172.16.4.106:80/wsd/scanservice.cgi, WSD

alexpevzner commented 4 years ago

Does it work with automatic discovery?

P.S. WSD URLs are very hard to guess. This is why I wrote a separate discovery utility. I have no hope that I will be able to explain to "normal" user by e-mail how to guess WSD endpoints.

tangyanli commented 4 years ago

The automatic discovery works. In order to reduce the discover time, I always configure the device in the airscan.conf.

alexpevzner commented 4 years ago

Regarding C5550/5560, manual says that you need to press some buttons on a front panel for each scanning document:

https://oip.manual.canon/USRMA-4409-zz-CS-5500-enGB/contents/devu-scan-rscan-wsd-wsdscan.html#417_h1_01

My Kyocera works the similar way, very inconvenient and usable only for testing. eSCL works without all this inconvenience.

tangyanli commented 4 years ago

The C5550/5560 is familiar with iR 2625/2630, they all need to press [WSD scan] - [Allow Remote Scan] before scanning. But the C5550/5560 WSD mode still fails. I don't know the reason...

alexpevzner commented 4 years ago

There are also some words about access management system and DepartmentID Authentication, that I don't understand:

https://oip.manual.canon/USRMA-4409-zz-CS-5500-enGB/contents/devu-scan-rscan-wsd.html

Have you already checked that?

tangyanli commented 4 years ago

No, I haven’t checked the Authentication. I can investigate it, but it may need some time. Any result, I will let you know. 😊

alexpevzner commented 4 years ago

OK. There is no any urgency here.

From a practical point of view, it doesn't improve support of C5550/5560, because it works well in the eSCL mode. But it increased testing coverage of the WSD, which may be useful to improve support of other devices, that don't implement eSCL.

Thank a lot for you valuable effort in testing this software!

tangyanli commented 4 years ago

Hi, Alexander

Both of below settings are disable, but I am still unable to test the WSD mode on iR-ADV C5550/5560. It seems the firmware has some bug.

ACCESS MANAGEMENT SYSTEM DepartmentID Authentication

And you are always welcome, I should thank you for the great tools (airscan & ipp-usb) and various help.

Thank a lot for you valuable effort in testing this software!