Canon Pixma TS5300 is unsupported: malformed request

nonnullish commented 1 year ago

Hello! First of all - thank you for developing this app :) I tried to use it on my Canon Pixma TS5353, but sadly got errors:

I saw the discussion in the issue with the HP scanner, so I went to /eSCL/ScannerCapabilities, and got this:

2.63Canon TS5300 seriesCanon00188b0e8fc000000000-0000-1000-8000-00188b0e8fc0http://7884F9000000.local./index.html?page=PAGE_AAPhttp://7884F9000000.local./icon/printer_icon.pngmopria-certified-scan1.212550135081Grayscale8RGB24PhotoTextTextAndPhotoHalftoneThruAutoimage/jpegapplication/pdfimage/jpegapplication/pdf757510010015015020020030030060060012001200sRGBNTSCDocumentPhotoPreviewTextAndGraphicAuto1200120000000111BackgroundReduction0101

I’m assuming Canon doesn’t pass this data as XML like other manufacturers (?).

I also checked the links in the string and the first one allows for changing the AirPrint settings and the second one is an image.

I guess the manufacturers try to make these interfaces as locked as possible so I don’t know if it’s even possible to support this printer, but let me know if more information is needed.

LeoKlaus commented 1 year ago

Hi, thank you for the bug report and all the information. The eSCL spec doesn't explicitly mention ScannerCapabilities has to return an XML, but the interface description is rather specific:

8.2 Interface

Scan Capabilities /{root}/ScannerCapabilities GET Description List the scanner capabilities: color entries, scan modes, ADF,… Payload OUT XML: /scan:ScannerCapabilities Status Code 200 OK – Success 500 Internal Server Error - Unknown internal error. 503 Service Unavailable – Server busy. Retry later

I'm pretty sure this response is out of spec and I have no idea how one would be able to reliably parse this. Splitting it up, I'm fairly certain it's supposed to look like this:

2.63Canon TS5300 series Canon 00188b0e8fc000000000-0000-1000-8000-00188b0e8fc0 http://7884F9000000.local./index.html?page=PAGE_AAP http://7884F9000000.local./icon/printer_icon.png mopria-certified-scan 1.212550135081

Grayscale8 RGB24

Photo Text TextAndPhoto

Halftone Thru Auto

image/jpeg application/pdf image/jpeg application/pdf

75 75 100 100 150 150 200 200 300 300 600 600 1200 1200

sRGB NTSC Document PhotoPreview TextAndGraphicAuto1200120000000111 BackgroundReduction 01 01

But even then these attributes are not even in the order they are in the example given in the spec, color mode for example precedes the resolution (I don't think the order is part of the spec, realistically it doesn't matter with XML).

Can you try the following command (on Mac and Linux, curl should already be installed on your system):

curl http://7884F9000000.local./eSCL/ScannerCapabilities \
-H "Accept: application/xml"

and see whether that returns an XML?

If you have a Mac, can you scan with this scanner using the default system dialog (Preferences->Printers&Scanners->Your Scanner->Open Scanner...)?

nonnullish commented 1 year ago

H, thanks for the response! Scanning with the default system dialog on a Mac does work, as well as requesting XML:

<?xml version="1.0" encoding="UTF-8"?><!---->
<scan:ScannerCapabilities
    xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 eSCL.xsd"
    xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03"
    xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <pwg:Version>2.63</pwg:Version>
    <pwg:MakeAndModel>Canon TS5300 series</pwg:MakeAndModel>
    <scan:Manufacturer>Canon</scan:Manufacturer>
    <pwg:SerialNumber>00188b0e8fc0</pwg:SerialNumber>
    <scan:UUID>00000000-0000-1000-8000-00188b0e8fc0</scan:UUID>
    <scan:AdminURI>http://7884F9000000.local./index.html?page=PAGE_AAP</scan:AdminURI>
    <scan:IconURI>http://7884F9000000.local./icon/printer_icon.png</scan:IconURI>
    <scan:Certifications>
        <scan:Certification>
            <scan:Name>mopria-certified-scan</scan:Name>
            <scan:Version>1.2</scan:Version>
        </scan:Certification>
    </scan:Certifications>
    <scan:Platen><!-- FB start -->
        <scan:PlatenInputCaps>
            <scan:MinWidth>1</scan:MinWidth>
            <scan:MaxWidth>2550</scan:MaxWidth>
            <scan:MinHeight>1</scan:MinHeight>
            <scan:MaxHeight>3508</scan:MaxHeight>
            <scan:MaxScanRegions>1</scan:MaxScanRegions>
            <scan:SettingProfiles>
                <scan:SettingProfile>
                    <scan:ColorModes>
                        <scan:ColorMode>Grayscale8</scan:ColorMode>
                        <scan:ColorMode scan:default="true">RGB24</scan:ColorMode>
                    </scan:ColorModes>
                    <scan:ContentTypes>
                        <pwg:ContentType>Photo</pwg:ContentType>
                        <pwg:ContentType>Text</pwg:ContentType>
                        <pwg:ContentType>TextAndPhoto</pwg:ContentType>
                        <pwg:ContentType>Halftone</pwg:ContentType>
                        <pwg:ContentType>Thru</pwg:ContentType>
                        <pwg:ContentType>Auto</pwg:ContentType>
                    </scan:ContentTypes>
                    <scan:DocumentFormats>
                        <pwg:DocumentFormat>image/jpeg</pwg:DocumentFormat>
                        <pwg:DocumentFormat>application/pdf</pwg:DocumentFormat>
                        <scan:DocumentFormatExt scan:default="true">image/jpeg</scan:DocumentFormatExt>
                        <scan:DocumentFormatExt>application/pdf</scan:DocumentFormatExt>
                    </scan:DocumentFormats>
                    <scan:SupportedResolutions>
                        <scan:DiscreteResolutions>
                            <scan:DiscreteResolution>
                                <scan:XResolution>75</scan:XResolution>
                                <scan:YResolution>75</scan:YResolution>
                            </scan:DiscreteResolution>
                            <scan:DiscreteResolution>
                                <scan:XResolution>100</scan:XResolution>
                                <scan:YResolution>100</scan:YResolution>
                            </scan:DiscreteResolution>
                            <scan:DiscreteResolution>
                                <scan:XResolution>150</scan:XResolution>
                                <scan:YResolution>150</scan:YResolution>
                            </scan:DiscreteResolution>
                            <scan:DiscreteResolution>
                                <scan:XResolution>200</scan:XResolution>
                                <scan:YResolution>200</scan:YResolution>
                            </scan:DiscreteResolution>
                            <scan:DiscreteResolution>
                                <scan:XResolution scan:default="true">300</scan:XResolution>
                                <scan:YResolution scan:default="true">300</scan:YResolution>
                            </scan:DiscreteResolution>
                            <scan:DiscreteResolution>
                                <scan:XResolution>600</scan:XResolution>
                                <scan:YResolution>600</scan:YResolution>
                            </scan:DiscreteResolution>
                            <scan:DiscreteResolution>
                                <scan:XResolution>1200</scan:XResolution>
                                <scan:YResolution>1200</scan:YResolution>
                            </scan:DiscreteResolution>
                        </scan:DiscreteResolutions>
                    </scan:SupportedResolutions>
                    <scan:ColorSpaces>
                        <scan:ColorSpace>sRGB</scan:ColorSpace>
                    </scan:ColorSpaces>
                    <scan:CcdChannels>
                        <scan:CcdChannel>NTSC</scan:CcdChannel>
                    </scan:CcdChannels>
                </scan:SettingProfile>
            </scan:SettingProfiles>
            <scan:SupportedIntents>
                <scan:Intent>Document</scan:Intent>
                <scan:Intent>Photo</scan:Intent>
                <scan:Intent>Preview</scan:Intent>
                <scan:Intent>TextAndGraphic</scan:Intent>
                <scan:Intent>Auto</scan:Intent>
            </scan:SupportedIntents>
            <scan:MaxOpticalXResolution>1200</scan:MaxOpticalXResolution>
            <scan:MaxOpticalYResolution>1200</scan:MaxOpticalYResolution>
            <scan:RiskyLeftMargin>0</scan:RiskyLeftMargin>
            <scan:RiskyRightMargin>0</scan:RiskyRightMargin>
            <scan:RiskyTopMargin>0</scan:RiskyTopMargin>
            <scan:RiskyBottomMargin>0</scan:RiskyBottomMargin>
        </scan:PlatenInputCaps>
    </scan:Platen><!--
    FB end -->
    <scan:CompressionFactorSupport>
        <scan:Min>0</scan:Min>
        <scan:Max>1</scan:Max>
        <scan:Normal>1</scan:Normal>
        <scan:Step>1</scan:Step>
    </scan:CompressionFactorSupport>
    <scan:SupportedMediaTypes>
        <scan:MediaType>BackgroundReduction</scan:MediaType>
    </scan:SupportedMediaTypes>
    <scan:SharpenSupport>
        <scan:Min>0</scan:Min>
        <scan:Max>1</scan:Max>
        <scan:Normal>0</scan:Normal>
        <scan:Step>1</scan:Step>
    </scan:SharpenSupport>
</scan:ScannerCapabilities>

LeoKlaus commented 1 year ago

That looks much better. I’ll add the header in the protocol and update the app. I’ll report back once I’ve submitted the update with Apple.

nonnullish commented 1 year ago

Superb! Thank you! :)

LeoKlaus commented 1 year ago

I've now submitted the update and am waiting for the review team. Once it's approved, I'll let you know and send you a link for the TestFlight beta.

LeoKlaus commented 1 year ago

The update is live now, the app store review was actually quicker than the TestFlight one. You should be able to update the app through the App Store.

nonnullish commented 1 year ago

Thanks, I've just checked with the updated app, but sadly I am still getting the error "Scan failed: Malformed request! You likely tried to mix parameters that can't be mixed". ☹️ I tried to use different settings but it didn't help. I think I'll try debugging the request body with Postman.

nonnullish commented 1 year ago

OK, so here are my findings:

This script works & so I tried to replicate what it does with Postman.

I sent a POST request at http://7884F9000000.local./eSCL/ScanJobs with this example request & got response code 201:

<?xml version="1.0" encoding="UTF-8"?>
<scan:ScanSettings xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm" xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03">
<pwg:Version>2.63</pwg:Version>
<pwg:ScanRegions>
    <pwg:ScanRegion>
        <pwg:XOffset>0</pwg:XOffset>
        <pwg:YOffset>0</pwg:YOffset>
        <pwg:Width>2550</pwg:Width>
        <pwg:Height>3508</pwg:Height>
        <pwg:ContentRegionUnits>escl:ThreeHundredthsOfInches</pwg:ContentRegionUnits>
    </pwg:ScanRegion>
</pwg:ScanRegions>
<pwg:InputSource>Platen</pwg:InputSource>
<pwg:DocumentFormat>image/jpeg</pwg:DocumentFormat>
<scan:ColorMode>RGB24</scan:ColorMode>
<scan:XResolution>75</scan:XResolution>
<scan:YResolution>75</scan:YResolution>
</scan:ScanSettings>

Then I sent a GET request at http://7884F9000000.local./eSCL/ScannerStatus and got this:

<?xml version="1.0" encoding="UTF-8"?>
<scan:ScannerStatus xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 eSCL.xsd" xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03" xmlns:httpdest="http://schemas.hp.com/imaging/httpdestination/2011/10/13" xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<pwg:Version>2.63</pwg:Version>
<pwg:State>Processing</pwg:State>
<scan:Jobs>
    <scan:JobInfo>
        <pwg:JobUri>/eSCL/ScanJobs/315221ae-027b-1000-adae-00188b0e8fc0</pwg:JobUri>
        <pwg:JobUuid>315221ae-027b-1000-adae-00188b0e8fc0</pwg:JobUuid>
        <scan:Age>4068</scan:Age>
        <pwg:ImagesCompleted>0</pwg:ImagesCompleted>
        <pwg:ImagesToTransfer>0</pwg:ImagesToTransfer>
        <pwg:JobState>Processing</pwg:JobState>
        <pwg:JobStateReasons>
            <pwg:JobStateReason>JobScanning</pwg:JobStateReason>
        </pwg:JobStateReasons>
    </scan:JobInfo>
    <scan:JobInfo>
        <pwg:JobUri>/eSCL/ScanJobs/3151cb35-027b-1000-9735-00188b0e8fc0</pwg:JobUri>
        <pwg:JobUuid>3151cb35-027b-1000-9735-00188b0e8fc0</pwg:JobUuid>
        <scan:Age>10749</scan:Age>
        <pwg:ImagesCompleted>0</pwg:ImagesCompleted>
        <pwg:ImagesToTransfer>0</pwg:ImagesToTransfer>
        <pwg:JobState>Canceled</pwg:JobState>
        <pwg:JobStateReasons>
            <pwg:JobStateReason>JobCanceledAtDevice</pwg:JobStateReason>
        </pwg:JobStateReasons>
    </scan:JobInfo>
    <scan:JobInfo>
        <pwg:JobUri>/eSCL/ScanJobs/314fad53-027b-1000-b953-00188b0e8fc0</pwg:JobUri>
        <pwg:JobUuid>314fad53-027b-1000-b953-00188b0e8fc0</pwg:JobUuid>
        <scan:Age>164927</scan:Age>
        <pwg:ImagesCompleted>0</pwg:ImagesCompleted>
        <pwg:ImagesToTransfer>0</pwg:ImagesToTransfer>
        <pwg:JobState>Aborted</pwg:JobState>
        <pwg:JobStateReasons>
            <pwg:JobStateReason>JobCanceledAtDevice</pwg:JobStateReason>
        </pwg:JobStateReasons>
    </scan:JobInfo>
</scan:Jobs>
</scan:ScannerStatus>

I copied the first JobUri & sent a GET request at http://7884F9000000.local./eSCL/ScanJobs/315221ae-027b-1000-adae-00188b0e8fc0/NextDocument, got 200 OK and an image

I don't have a Mac so I can't really dig into the code itself, but I hope this information will be useful to you!

LeoKlaus commented 1 year ago

Hey, thanks for the reply and further information. This week is pretty full for me so I didn't have the time to look at this yet.

Do the options selectable in the App look correct to you or are settings available that shouldn't be? I'm guessing that your printer requires more specific requests in other areas as well.

Can you give me one example of settings you used for a failed request (within the app)?

The error message indicates that your printer returned a 409 which is weird if the parameters match.

I think it's best if I create some sort of debugging capability within the app so you can just send me logs for the request. I hope I'll get this done this weekend. Sorry for the inconvenience and again thank you for all the infromation!

LeoKlaus commented 1 year ago

I've just submitted a new version with debug logging to Apple, I'll let you know once it's approved.

nonnullish commented 1 year ago

Hi, thanks so much for taking such care in this. Don't stress too much about it though - I know how much work there is in this kind of projects - but I do appreciate it. I didn't get to analyze the options too well yet, but they seemed fine. I tried different combinations too, so I'll see with the debug logging on what goes wrong exactly.

LeoKlaus commented 1 year ago

https://testflight.apple.com/join/CelMOo9W

Here you can join the TestFlight beta and download the build that includes debug logging. Logging is enabled by default and all you have to do is press the new button in the top right corner. Collecting logs may take a few seconds. When the logs appear, you can copy them to your clipboard by tapping the "doc" icon in the top right corner.

I really appreciate the time you're taking for this, it helps me immensely!

nonnullish commented 1 year ago

Hi, thanks for the TestFlight link! This time I used HTTPS and the IP of the printer in my Postman testing because I guess that is what the app is also using.

I sent a GET request at https://192.168.0.17/eSCL/ScannerStatus, with no custom headers and no body. The response showed two scanning jobs that I did before.
I sent a "Custom Scan" so that I could see the options, but got the error. There were still only two scanning jobs at /ScannerStatus.

I looked into the logs:

[22/7/2023, 10:37] [Browser] Browser: browser.stateUpdateHandler <private>
[22/7/2023, 10:37] [esclScanner] Capabilities: Querying capabilities for device ...
[22/7/2023, 10:37] [esclScanner] Capabilities: Success. Capabilities for device  parsed!
[22/7/2023, 10:37] [esclScanner] Capabilities: Querying capabilities for device ...
[22/7/2023, 10:37] [esclScanner] Capabilities: Success. Capabilities for device  parsed!
[22/7/2023, 10:37] [esclScanner] Capabilities: Querying capabilities for device ...
[22/7/2023, 10:37] [esclScanner] Capabilities: Success. Capabilities for device  parsed!
[22/7/2023, 10:37] [esclScanner] Status: Querying status for device Canon TS5300 series...
[22/7/2023, 10:38] [esclScanner] Status: Device is idle
[22/7/2023, 10:38] [esclScanner] PostRequest: created body:
<scan:ScanSettings xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm" xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03">
<pwg:Version>2.63</pwg:Version>
<pwg:ScanRegions>
<pwg:ScanRegion>
<pwg:Width>2480</pwg:Width>
<pwg:Height>3508</pwg:Height>
<pwg:XOffset>0</pwg:XOffset>
<pwg:YOffset>0</pwg:YOffset>
<pwg:ContentRegionUnits>escl:ThreeHundredthsOfInches</pwg:ContentRegionUnits>
</pwg:ScanRegion>
</pwg:ScanRegions>
<pwg:InputSource>Platen</pwg:InputSource>
<pwg:DocumentFormat>image/jpeg</pwg:DocumentFormat>
<scan:ColorMode>Grayscale8</scan:ColorMode>
<scan:XResolution>75</scan:XResolution>
<scan:YResolution>75</scan:YResolution>
</scan:ScanSettings>
[22/7/2023, 10:38] [esclScanner] PostRequest: Sending request to Canon TS5300 series...
[22/7/2023, 10:38] [esclScanner] PostRequest: Received invalid response from device: 409
[22/7/2023, 10:38] [esclScanner] ScanDocumentAndSave: Scanner didn't accept the job. 409
[22/7/2023, 10:38] [esclScanner] Capabilities: Querying capabilities for device ...

I copied the request body and sent a POST request at https://192.168.0.17/eSCL/ScanJobs. And now this is something: without the <?xml version="1.0" encoding="UTF-8"?> header in the body I did get the 409 error (right bottom corner of the screenshot). Even though the request did have the Content Type: application/xml header:
With the <?xml version="1.0" encoding="UTF-8"?> header in the body I got 201 Created.
/ScannerStatus now showed a new job with a status JobScanning.

I don't know if that missing header in the request body is the reason of the error, but it seems plausible?

LeoKlaus commented 1 year ago

without the <?xml version="1.0" encoding="UTF-8"?> header in the body I did get the 409 error

I'm pretty sure that's it! I've just pushed an update to TestFlight to include the header. Sorry for the delay, XCode Cloud has some severe issues with the XCode 15 Beta...

nonnullish commented 1 year ago

Just updated - it works, thank you so much!

LeoKlaus commented 1 year ago

Great, thank you! I'll push this release to the Store then!

LeoKlaus commented 1 year ago

1.4.0 just got approved by Apple in case you want to leave TestFlight :)

nonnullish commented 1 year ago

Great, I’ll switch to the AppStore app then :)

LeoKlaus / OpenAirScan

Canon Pixma TS5300 is unsupported: malformed request #4

8.2 Interface