search

psf / requests

A simple, yet elegant, HTTP library.
https://requests.readthedocs.io/en/latest/
Apache License 2.0
52.19k stars 9.33k forks source link

XML gets shortened when submitting a post request #6804

Closed danster99 closed 3 weeks ago

danster99 commented 1 month ago

I am trying to send an xml in a post request to a SOAP endpoint of a TMS server. The code is the following:

response = requests.post(
        url,
        data=xml_output,
        headers={"Content-Type": "text/xml"},
        timeout=60,
        cert=(cert_file, decrypted_key_file),
    )

The server returns a response saying that it got an unexpected end of input in lixe x, char y. The error is dependent on the size of the xml, but is reproducible.

I am attaching 2 XMLs as examples:

EX 1:

<?xml version="1.0" ?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tms="urn:CDM/tmsIntegrationService/" xmlns:sh="http://www.unece.org/cefact/namespaces/StandardBusinessDocumentHeader">
   <soapenv:Header>
      <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
         <wsse:UsernameToken>
            <wsse:Username>xxx_xxx</wsse:Username>
            <wsse:Password>xxxxxxxxxxxxxxxxxxxx</wsse:Password>
         </wsse:UsernameToken>
      </wsse:Security>
   </soapenv:Header>
   <soapenv:Body>
      <tms:transportInstructionMessage>
         <sh:StandardBusinessDocumentHeader>
            <sh:HeaderVersion>1</sh:HeaderVersion>
            <sh:Sender>
               <sh:Identifier>PAWEŁ ZABŁOCKI ZABŁOCKI I PARTNERZY</sh:Identifier>
            </sh:Sender>
            <sh:Receiver>
               <sh:Identifier>Test Identifier</sh:Identifier>
            </sh:Receiver>
            <sh:DocumentIdentification>
               <sh:Standard>GS1</sh:Standard>
               <sh:TypeVersion>3.2</sh:TypeVersion>
               <sh:InstanceIdentifier>100002</sh:InstanceIdentifier>
               <sh:Type>Transport Instruction</sh:Type>
               <sh:CreationDateAndTime>2024-10-01T07:58:43Z</sh:CreationDateAndTime>
            </sh:DocumentIdentification>
            <sh:BusinessScope>
               <sh:Scope>
                  <sh:Type>EDIcustomerNumber</sh:Type>
                  <sh:InstanceIdentifier>90000050</sh:InstanceIdentifier>
               </sh:Scope>
               <sh:Scope>
                  <sh:Type>fileType</sh:Type>
                  <sh:InstanceIdentifier>IF</sh:InstanceIdentifier>
               </sh:Scope>
               <sh:Scope>
                  <sh:Type>department</sh:Type>
                  <sh:InstanceIdentifier>62</sh:InstanceIdentifier>
               </sh:Scope>
               <sh:Scope>
                  <sh:Type>application</sh:Type>
                  <sh:InstanceIdentifier>LOGI</sh:InstanceIdentifier>
               </sh:Scope>
            </sh:BusinessScope>
         </sh:StandardBusinessDocumentHeader>
         <transportInstruction>
            <creationDateTime>2024-10-01T07:58:43Z</creationDateTime>
            <documentStatusCode>ORIGINAL</documentStatusCode>
            <documentActionCode>ADD</documentActionCode>
            <transportInstructionIdentification>
               <entityIdentification>12345_test/012934535</entityIdentification>
            </transportInstructionIdentification>
            <transportInstructionFunction>SHIPMENT</transportInstructionFunction>
            <logisticServicesSeller/>
            <logisticServicesBuyer>
               <additionalPartyIdentification additionalPartyIdentificationTypeCode="searchname">PAWEŁ ZABŁOCKI ZABŁOCKI I PARTNERZY</additionalPartyIdentification>
            </logisticServicesBuyer>
            <transportInstructionShipment>
               <additionalShipmentIdentification additionalShipmentIdentificationTypeCode="refopd">LC48</additionalShipmentIdentification>
               <note languageCode="EN" noteTypeCode="INF"/>
               <note languageCode="EN" noteTypeCode="INF">Fragile goods !</note>
               <note languageCode="EN" noteTypeCode="INF">Towar niepiętrowalny</note>
               <note languageCode="EN" noteTypeCode="INF"/>
               <note languageCode="EN" noteTypeCode="INF">Fragile goods !</note>
               <note languageCode="EN" noteTypeCode="INF">Towar niepiętrowalny</note>
               <receiver>
                  <additionalPartyIdentification additionalPartyIdentificationTypeCode="searchname">3b045ae1-edec-485d-9</additionalPartyIdentification>
                  <address>
                     <city>Üllő</city>
                     <countryCode>HU</countryCode>
                     <name>AUCHAN MAGYARORSZÁG</name>
                     <postalCode>2225</postalCode>
                     <streetAddressOne>Zsaróka út 8</streetAddressOne>
                  </address>
               </receiver>
               <shipper>
                  <additionalPartyIdentification additionalPartyIdentificationTypeCode="searchname">ad11a7fc-0442-4b03-8</additionalPartyIdentification>
                  <address>
                     <city>WIRY</city>
                     <countryCode>PL</countryCode>
                     <name>PAWEŁ ZABŁOCKI ZABŁOCKI I PARTNERZY</name>
                     <postalCode>62-051</postalCode>
                     <streetAddressOne>KASZTANOWA 12</streetAddressOne>
                  </address>
               </shipper>
               <shipTo>
                  <additionalPartyIdentification additionalPartyIdentificationTypeCode="searchname">3b045ae1-edec-485d-9</additionalPartyIdentification>
                  <address>
                     <city>Üllő</city>
                     <countryCode>HU</countryCode>
                     <name>AUCHAN MAGYARORSZÁG</name>
                     <postalCode>2225</postalCode>
                     <streetAddressOne>Zsaróka út 8</streetAddressOne>
                  </address>
                  <contact>
                     <contactTypeCode>BJ</contactTypeCode>
                     <personName>John Doe</personName>
                     <communicationChannel>
                        <communicationChannelCode>EMAIL</communicationChannelCode>
                        <communicationValue>john.doe@mail.com</communicationValue>
                     </communicationChannel>
                     <communicationChannel>
                        <communicationChannelCode>TELEPHONE</communicationChannelCode>
                        <communicationValue>+391234123490</communicationValue>
                     </communicationChannel>
                  </contact>
               </shipTo>
               <shipFrom>
                  <additionalPartyIdentification additionalPartyIdentificationTypeCode="searchname">ad11a7fc-0442-4b03-8</additionalPartyIdentification>
                  <address>
                     <city>WIRY</city>
                     <countryCode>PL</countryCode>
                     <name>PAWEŁ ZABŁOCKI ZABŁOCKI I PARTNERZY</name>
                     <postalCode>62-051</postalCode>
                     <streetAddressOne>KASZTANOWA 12</streetAddressOne>
                  </address>
               </shipFrom>
               <transportInstructionTerms>
                  <transportServiceCategoryType>30</transportServiceCategoryType>
                  <logisticService>
                     <logisticServiceRequirementCode logisticServiceTypeCode="parameters">ForPlanning</logisticServiceRequirementCode>
                  </logisticService>
                  <logisticService>
                     <logisticServiceRequirementCode logisticServiceTypeCode="productType">PROD01</logisticServiceRequirementCode>
                  </logisticService>
                  <logisticService>
                     <logisticServiceRequirementCode logisticServiceTypeCode="parameters">ROD</logisticServiceRequirementCode>
                  </logisticService>
                  <logisticService>
                     <logisticServiceRequirementCode logisticServiceTypeCode="parameters">ROP</logisticServiceRequirementCode>
                  </logisticService>
               </transportInstructionTerms>
               <plannedDespatch>
                  <logisticEventPeriod>
                     <beginDate>2024-10-27</beginDate>
                     <beginTime>01:10:00</beginTime>
                     <endDate>2024-10-27</endDate>
                     <endTime>10:00:00</endTime>
                  </logisticEventPeriod>
               </plannedDespatch>
               <transportReference>
                  <entityIdentification>12345_test/012934535</entityIdentification>
                  <transportReferenceTypeCode>customerRef</transportReferenceTypeCode>
               </transportReference>
               <transportInstructionShipmentItem>
                  <lineItemNumber>1</lineItemNumber>
                  <transportCargoCharacteristics>
                     <cargoTypeCode>neam</cargoTypeCode>
                     <cargoTypeDescription languageCode="PL">Smycze</cargoTypeDescription>
                     <totalGrossVolume measurementUnitCode="MTQ">7.5</totalGrossVolume>
                     <totalGrossWeight measurementUnitCode="KGM">5.000</totalGrossWeight>
                     <totalLoadingLength measurementUnitCode="PP">2.8000000000000003</totalLoadingLength>
                     <totalPackageQuantity measurementUnitCode="Euro pallet 120x80">7</totalPackageQuantity>
                     <totalItemQuantity measurementUnitCode="Euro pallet 120x80">7</totalItemQuantity>
                  </transportCargoCharacteristics>
               </transportInstructionShipmentItem>
            </transportInstructionShipment>
         </transportInstruction>
      </tms:transportInstructionMessage>
   </soapenv:Body>
</soapenv:Envelope>

the error here is unexpected end of input block at row 161, col 15

EX 2:

<?xml version="1.0" ?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tms="urn:CDM/tmsIntegrationService/" xmlns:sh="http://www.unece.org/cefact/namespaces/StandardBusinessDocumentHeader">
   <soapenv:Header>
      <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
         <wsse:UsernameToken>
            <wsse:Username>xxx_xxx</wsse:Username>
            <wsse:Password>xxxxxxxxxxxxxxxxxxxx</wsse:Password>
         </wsse:UsernameToken>
      </wsse:Security>
   </soapenv:Header>
   <soapenv:Body>
      <tms:transportInstructionMessage>
         <sh:StandardBusinessDocumentHeader>
            <sh:HeaderVersion>1</sh:HeaderVersion>
            <sh:Sender>
               <sh:Identifier>PAWEŁ ZABŁOCKI ZABŁOCKI I PARTNERZY</sh:Identifier>
            </sh:Sender>
            <sh:Receiver>
               <sh:Identifier>Test Identifier</sh:Identifier>
            </sh:Receiver>
            <sh:DocumentIdentification>
               <sh:Standard>GS1</sh:Standard>
               <sh:TypeVersion>3.2</sh:TypeVersion>
               <sh:InstanceIdentifier>100002</sh:InstanceIdentifier>
               <sh:Type>Transport Instruction</sh:Type>
               <sh:CreationDateAndTime>2024-10-01T08:06:49Z</sh:CreationDateAndTime>
            </sh:DocumentIdentification>
            <sh:BusinessScope>
               <sh:Scope>
                  <sh:Type>EDIcustomerNumber</sh:Type>
                  <sh:InstanceIdentifier>90000050</sh:InstanceIdentifier>
               </sh:Scope>
               <sh:Scope>
                  <sh:Type>fileType</sh:Type>
                  <sh:InstanceIdentifier>IF</sh:InstanceIdentifier>
               </sh:Scope>
               <sh:Scope>
                  <sh:Type>department</sh:Type>
                  <sh:InstanceIdentifier>62</sh:InstanceIdentifier>
               </sh:Scope>
               <sh:Scope>
                  <sh:Type>application</sh:Type>
                  <sh:InstanceIdentifier>LOGI</sh:InstanceIdentifier>
               </sh:Scope>
            </sh:BusinessScope>
         </sh:StandardBusinessDocumentHeader>
         <transportInstruction>
            <creationDateTime>2024-10-01T08:06:49Z</creationDateTime>
            <documentStatusCode>ORIGINAL</documentStatusCode>
            <documentActionCode>ADD</documentActionCode>
            <transportInstructionIdentification>
               <entityIdentification>GV-RD-0099703/1</entityIdentification>
            </transportInstructionIdentification>
            <transportInstructionFunction>SHIPMENT</transportInstructionFunction>
            <logisticServicesSeller/>
            <logisticServicesBuyer>
               <additionalPartyIdentification additionalPartyIdentificationTypeCode="searchname">PAWEŁ ZABŁOCKI ZABŁOCKI I PARTNERZY</additionalPartyIdentification>
            </logisticServicesBuyer>
            <transportInstructionShipment>
               <additionalShipmentIdentification additionalShipmentIdentificationTypeCode="refopd">LC49</additionalShipmentIdentification>
               <note languageCode="EN" noteTypeCode="INF"/>
               <note languageCode="EN" noteTypeCode="INF">Fragile goods !</note>
               <note languageCode="EN" noteTypeCode="INF">Towar niepiętrowalny</note>
               <note languageCode="EN" noteTypeCode="INF"/>
               <note languageCode="EN" noteTypeCode="INF">Fragile goods !</note>
               <note languageCode="EN" noteTypeCode="INF">Towar niepiętrowalny</note>
               <receiver>
                  <additionalPartyIdentification additionalPartyIdentificationTypeCode="searchname">669ff966-c2ca-4745-8</additionalPartyIdentification>
                  <address>
                     <city>Hrádek</city>
                     <countryCode>CZ</countryCode>
                     <name>Borgers Hradek</name>
                     <postalCode>33842</postalCode>
                     <streetAddressOne>Rokycanska ulice 223/II</streetAddressOne>
                  </address>
               </receiver>
               <shipper>
                  <additionalPartyIdentification additionalPartyIdentificationTypeCode="searchname">ad11a7fc-0442-4b03-8</additionalPartyIdentification>
                  <address>
                     <city>WIRY</city>
                     <countryCode>PL</countryCode>
                     <name>PAWEŁ ZABŁOCKI ZABŁOCKI I PARTNERZY</name>
                     <postalCode>62-051</postalCode>
                     <streetAddressOne>KASZTANOWA 12</streetAddressOne>
                  </address>
               </shipper>
               <shipTo>
                  <additionalPartyIdentification additionalPartyIdentificationTypeCode="searchname">669ff966-c2ca-4745-8</additionalPartyIdentification>
                  <address>
                     <city>Hrádek</city>
                     <countryCode>CZ</countryCode>
                     <name>Borgers Hradek</name>
                     <postalCode>33842</postalCode>
                     <streetAddressOne>Rokycanska ulice 223/II</streetAddressOne>
                  </address>
                  <contact>
                     <contactTypeCode>BJ</contactTypeCode>
                     <personName>John Doe</personName>
                     <communicationChannel>
                        <communicationChannelCode>EMAIL</communicationChannelCode>
                        <communicationValue>john.doe@mail.com</communicationValue>
                     </communicationChannel>
                     <communicationChannel>
                        <communicationChannelCode>TELEPHONE</communicationChannelCode>
                        <communicationValue>+391231212340</communicationValue>
                     </communicationChannel>
                  </contact>
               </shipTo>
               <shipFrom>
                  <additionalPartyIdentification additionalPartyIdentificationTypeCode="searchname">ad11a7fc-0442-4b03-8</additionalPartyIdentification>
                  <address>
                     <city>WIRY</city>
                     <countryCode>PL</countryCode>
                     <name>PAWEŁ ZABŁOCKI ZABŁOCKI I PARTNERZY</name>
                     <postalCode>62-051</postalCode>
                     <streetAddressOne>KASZTANOWA 12</streetAddressOne>
                  </address>
               </shipFrom>
               <transportInstructionTerms>
                  <transportServiceCategoryType>30</transportServiceCategoryType>
                  <logisticService>
                     <logisticServiceRequirementCode logisticServiceTypeCode="parameters">ForPlanning</logisticServiceRequirementCode>
                  </logisticService>
                  <logisticService>
                     <logisticServiceRequirementCode logisticServiceTypeCode="productType">PROD01</logisticServiceRequirementCode>
                  </logisticService>
                  <logisticService>
                     <logisticServiceRequirementCode logisticServiceTypeCode="parameters">ROD</logisticServiceRequirementCode>
                  </logisticService>
                  <logisticService>
                     <logisticServiceRequirementCode logisticServiceTypeCode="parameters">ROP</logisticServiceRequirementCode>
                  </logisticService>
               </transportInstructionTerms>
               <plannedDespatch>
                  <logisticEventPeriod>
                     <beginDate>2024-10-10</beginDate>
                     <beginTime>08:00:00</beginTime>
                     <endDate>2024-10-10</endDate>
                     <endTime>22:00:00</endTime>
                  </logisticEventPeriod>
               </plannedDespatch>
               <transportReference>
                  <entityIdentification>GV-RD-0099703/1</entityIdentification>
                  <transportReferenceTypeCode>customerRef</transportReferenceTypeCode>
               </transportReference>
               <transportInstructionShipmentItem>
                  <lineItemNumber>1</lineItemNumber>
                  <transportCargoCharacteristics>
                     <cargoTypeCode>neam</cargoTypeCode>
                     <cargoTypeDescription languageCode="PL">Smycze</cargoTypeDescription>
                     <totalGrossVolume measurementUnitCode="MTQ">0.00</totalGrossVolume>
                     <totalGrossWeight measurementUnitCode="KGM">24.500</totalGrossWeight>
                     <totalLoadingLength measurementUnitCode="PP">2.8000000000000003</totalLoadingLength>
                     <totalPackageQuantity measurementUnitCode="Euro pallet 120x80">7</totalPackageQuantity>
                     <totalItemQuantity measurementUnitCode="Euro pallet 120x80">7</totalItemQuantity>
                  </transportCargoCharacteristics>
               </transportInstructionShipmentItem>
            </transportInstructionShipment>
         </transportInstruction>
      </tms:transportInstructionMessage>
   </soapenv:Body>
</soapenv:Envelope>

here the row is the same but the column is different

vaidyasaurabh08 commented 3 weeks ago

Tried to replicate this using a simple flask server setup using the 2 xmls as above. The server did not complain and was able to print out the entire xml that was sent to it. That makes me think that there should be no payload limit on the requests client while it sends out the http requests. Do you think there is some chunking happening over your network due to firewall etc? or at your server?

Test client code:

import requests
FLASK_LOCAL_URL = "http://127.0.01:5000/"

xml_output = """<?xml version="1.0" ?>.... 
"""

response = requests.post(
        FLASK_LOCAL_URL,
        data=xml_output,
        headers={"Content-Type": "text/xml"},
        timeout=60
    )

print(response)

Output <Response [200]>

Can you help with instructions on how to reproduce this using a test server?