ValueError: Insufficient data - ... bytes requested but only 18358 available on AWS Lambda/Amazon Linux

[robincsamuel]

I'm trying to build an AS2 gateway with AWS Lambda (Amazon Linux, x86_64), using chalice. I got a base version working fine on my local system.

However, I'm facing several issues with different configurations and the major one is the decryption/decompression issue with asn1crypto library.

Below is the stack trace,

Failed to parse AS2 message: 
Traceback (most recent call last):
  File "/var/task/pyas2lib/as2.py", line 591, in parse
    self.enc_alg, decrypted_content = decrypt_message(
  File "/var/task/pyas2lib/cms.py", line 193, in decrypt_message
    cms_content = cms.ContentInfo.load(encrypted_data)
  File "/var/task/asn1crypto/core.py", line 230, in load
    value, _ = _parse_build(encoded_data, spec=spec, spec_params=kwargs, strict=strict)
  File "/var/task/asn1crypto/core.py", line 5668, in _parse_build
    info, new_pointer = _parse(encoded_data, encoded_len, pointer)
  File "/var/task/asn1crypto/parser.py", line 225, in _parse
    raise ValueError(_INSUFFICIENT_DATA_MESSAGE % (contents_end, data_len))
ValueError: Insufficient data - 1545633062365291768294061961431016176374627348951649321802178575314893183602912632065505301327437350584106059005008117218689175730975503838612273058029938831908239103060349018132643124869528847276954337830257591395451447158305993254068662164490556331379749698778432190 bytes requested but only 18358 available

Failed to parse AS2 message: 
Traceback (most recent call last):
  File "/var/task/pyas2lib/cms.py", line 58, in decompress_message
    cms_content = cms.ContentInfo.load(compressed_data)
  File "/var/task/asn1crypto/core.py", line 230, in load
    value, _ = _parse_build(encoded_data, spec=spec, spec_params=kwargs, strict=strict)
  File "/var/task/asn1crypto/core.py", line 5668, in _parse_build
    info, new_pointer = _parse(encoded_data, encoded_len, pointer)
  File "/var/task/asn1crypto/parser.py", line 225, in _parse
    raise ValueError(_INSUFFICIENT_DATA_MESSAGE % (contents_end, data_len))
ValueError: Insufficient data - 1545631240861543359501279624088988123554064671567487520281914362577698843550672848852978282426209400061697586633380480950953001104403175629083821146808112112539624518748043246611948084041583273941180809691449207723938876550369994869041800112067195332656411633136753191 bytes requested but only 11670 available

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/task/pyas2lib/as2.py", line 603, in parse
    self.compressed, self.payload = self._decompress_data(self.payload)
  File "/var/task/pyas2lib/as2.py", line 511, in _decompress_data
    decompressed_data = decompress_message(compressed_data)
  File "/var/task/pyas2lib/cms.py", line 64, in decompress_message
    raise DecompressionError("Decompression failed with cause: 
{}
".format(e))
pyas2lib.exceptions.DecompressionError: Decompression failed with cause: Insufficient data - 1545631240861543359501279624088988123554064671567487520281914362577698843550672848852978282426209400061697586633380480950953001104403175629083821146808112112539624518748043246611948084041583273941180809691449207723938876550369994869041800112067195332656411633136753191 bytes requested but only 11670 available

Here's is a similar issue, https://github.com/wbond/asn1crypto/issues/85

Any hints would be helpful, and thanks for the package :)

[abhishek-ram]

@robincsamuel what is the server you are communicating with?

[robincsamuel]

@abhishek-ram it’s a django-pyas2 project running on an AWS EC2 (ubuntu)

[robincsamuel]

@abhishek-ram I did some testing and it works fine on my local system. I connected my local application to the remote database (the DB lambda use) so that the keys are the same (they are loaded from the database) and it worked fine with the same payload, send from the same server I used with the lambda.

I've also logged the keys & payload from the lambda and that too is fine.

[abhishek-ram]

Then it is most likely an issue with the build of the server where its running

[robincsamuel]

yeah, that's my guess too. It's a standard AWS Lambda platform. I've posted the issue on asn1crypto repo as well.

[robincsamuel]

I’m closing this assuming it’s asn1crypto issue.