search

arunavabasucom / radis-app

A web app for high-resolution infrared molecular spectra using RADIS
https://radis.app
GNU Lesser General Public License v3.0
11 stars 15 forks source link

CO calculations unexpected slow / stuck #486

Closed erwanp closed 2 years ago

erwanp commented 2 years ago

Running the default calculation (CO, 300 K, etc.) with range increased to 1900 - 10,000 cm-1 times out.

However, the same spectrum on Radis-Lab is computed in less than 1s, with < 100 MB memory usage, so it shouldn't be a AWS hardware limitation on Radis-App. 

# Radis-Lab code : 
%matplotlib inline
from radis import calc_spectrum
s = calc_spectrum(1900, 10000,         # cm-1
                  molecule='CO',
                  isotope='1',
                  pressure=1.01325,   # bar
                  Tgas=300,           # K
                  mole_fraction=0.1,
                  path_length=1,      # cm
                  databank='hitran', 
                  verbose=False
                  )
s.plot()    # plot the radiance by default

What's the best way to investigate the error ? Run the Radis-app backend locally ?

suzil commented 2 years ago

I just tried the default settings but increasing up to 10k cm-1 upper range and it returns a 502 status code. Here is what I can see from the logs:

START RequestId: 828d9091-1a49-482f-b7fa-e22806574d31 Version: $LATEST
--
Payload {'species': [{'molecule': 'CO', 'mole_fraction': 0.1}], 'min_wavenumber_range': 1900, 'max_wavenumber_range': 10000, 'tgas': 300, 'tvib': None, 'trot': None, 'pressure': 1.01325, 'path_length': 1, 'simulate_slit': False, 'mode': 'absorbance'}
HAPI version: 1.2.2.0
To get the most up-to-date version please check http://hitran.org/hapi
ATTENTION: Python versions of partition sums from TIPS-2021 are now available in HAPI code
MIT license: Copyright 2021 HITRAN team, see more at http://hitran.org.
If you use HAPI in your research or software development,
please cite it using the following reference:
R.V. Kochanov, I.E. Gordon, L.S. Rothman, P. Wcislo, C. Hill, J.S. Wilzewski,
HITRAN Application Programming Interface (HAPI): A comprehensive approach
to working with spectroscopic data, J. Quant. Spectrosc. Radiat. Transfer 177, 15-30 (2016)
DOI: 10.1016/j.jqsrt.2016.03.005
ATTENTION: This is the core version of the HITRAN Application Programming Interface.
For more efficient implementation of the absorption coefficient routine,
as well as for new profiles, parameters and other functional,
please consider using HAPI2 extension library.
HAPI2 package is available at http://github.com/hitranonline/hapi2
Using /tmp/.radisdb/hitran/downloads__can_be_deleted/CO
Data is fetched from http://hitran.org
BEGIN DOWNLOAD: CO_1
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_1.data
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_1.data
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_1.data
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_1.data
Header written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_1.header
END DOWNLOAD
Lines parsed: 1344
PROCESSED
Data is fetched from http://hitran.org
BEGIN DOWNLOAD: CO_2
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_2.data
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_2.data
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_2.data
Header written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_2.header
END DOWNLOAD
Lines parsed: 1042
PROCESSED
Data is fetched from http://hitran.org
BEGIN DOWNLOAD: CO_3
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_3.data
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_3.data
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_3.data
Header written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_3.header
END DOWNLOAD
Lines parsed: 920
PROCESSED
Data is fetched from http://hitran.org
BEGIN DOWNLOAD: CO_4
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_4.data
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_4.data
Header written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_4.header
END DOWNLOAD
Lines parsed: 800
PROCESSED
Data is fetched from http://hitran.org
BEGIN DOWNLOAD: CO_5
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_5.data
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_5.data
Header written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_5.header
END DOWNLOAD
Lines parsed: 674
PROCESSED
Data is fetched from http://hitran.org
BEGIN DOWNLOAD: CO_6
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_6.data
65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_6.data
Header written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_6.header
END DOWNLOAD
Lines parsed: 601
PROCESSED
Added HITRAN-CO database in /tmp/radis.json
Calculating Equilibrium Spectrum
Physical Conditions
----------------------------------------
Tgas                 300 K
Trot                 300 K
Tvib                 300 K
isotope              1
mole_fraction        0.1
molecule             CO
overpopulation       None
path_length          1 cm
pressure_mbar        1013.25 mbar
rot_distribution     boltzmann
self_absorption      True
state                X
vib_distribution     boltzmann
wavenum_max          10000.0000 cm-1
wavenum_min          1900.0000 cm-1
Computation Parameters
----------------------------------------
Tref                 296 K
add_at_used
broadening_method    voigt
cutoff               1e-27 cm-1/(#.cm-2)
dbformat             hitran
dbpath               /tmp/.radisdb/hitran/CO.hdf5
dlm_log_pG           0.1375350788016573
dlm_log_pL           0.20180288881201608
export_rovib_fraction  True
folding_thresh       1e-06
include_neighbouring_lines  True
levelsfmt            None
memory_mapping_engine  auto
neighbour_lines      0 cm-1
optimization         min-RMS
parfuncfmt           hapi
parfuncpath          None
parsum_mode          full summation
pseudo_continuum_threshold  0
sparse_dlm           auto
truncation           50 cm-1
wavenum_max_calc     10000.0000 cm-1
wavenum_min_calc     1900.0000 cm-1
waveunit             cm-1
wstep                0.01 cm-1
zero_padding         -1
Information
----------------------------------------
chunksize            None
db_use_cached        True
export_lines         False
export_populations   None
load_energies        False
lvl_use_cached       True
total_lines          847
warning_broadening_threshold  0.01
warning_linestrength_cutoff  0.01
----------------------------------------
0.57s - Spectrum calculated
[ERROR] [1638236826889] LAMBDA_RUNTIME Failed to post handler success response. Http response code: 413.
Traceback (most recent call last):
File "/var/runtime/bootstrap.py", line 480, in <module>
main()
File "/var/runtime/bootstrap.py", line 468, in main
handle_event_request(lambda_runtime_client,
File "/var/runtime/bootstrap.py", line 148, in handle_event_request
lambda_runtime_client.post_invocation_result(invoke_id, result, result_content_type)
File "/var/runtime/lambda_runtime_client.py", line 62, in post_invocation_result
rapid_client.post_invocation_result(invoke_id, result_data if isinstance(result_data, bytes) else result_data.encode('utf-8'), content_type)
RuntimeError: Failed to post invocation response
END RequestId: 828d9091-1a49-482f-b7fa-e22806574d31
REPORT RequestId: 828d9091-1a49-482f-b7fa-e22806574d31  Duration: 6858.35 ms    Billed Duration: 6859 ms    Memory Size: 10240 MB   Max Memory Used: 598 MB XRAY TraceId: 1-61a58294-7fa64f9367988f64389346c6   SegmentId: 0a9e7a2403352ee9 Sampled: true
suzil commented 2 years ago

Lambda's max response size is 6MB and I guess we're hitting that here.

suzil commented 2 years ago

We can work around this by uploading the response to S3 and then returning the bucket and key name as the response and having the client download it from S3.

Would this make sense? Or do you think it's too much data in general and maybe there's a way to shrink it so that clients aren't downloading so much?

erwanp commented 2 years ago

I see two things here. First that the data is still downloaded, confirms that #476 didn't work.

Regarding the payload limit : we could return only a resampled, lower resolution spectrum to the user. It would ensure it is less than 6 MB.

From afar we won't notice any difference. However, as the user zooms in, we should make sure higher resolution elements are retrieved.

I've seen similar things on Vaex examples where they would visualize data from maps with billion points. You would zoom in, and at some point the map would be re-updated with higher resolution data.

It may not be easy, as I understand it the Lambda doesn't retain the calculated objects in memory once the payload has been returned.

erwanp commented 2 years ago

Resampling done ; the "auto-increase-the-resolution-when-zooming" could be added later ; for the moment users can still re-compute a spectrum on the higher-resolution range.

Maybe we could return a small pop-up info for the users ? "Spectrum was large and was resampled for rendering." in #490

erwanp commented 2 years ago

I tried after #490 (and your fix) and it's still stuck for 1900 - 10,000 cm-1. I can investigate : if I run the backend locally, how do I get the log ?

suzil commented 2 years ago

I'm not sure whether or not it replicates locally. I've had issues with a recent RADIS version running the backend locally which I haven't spent the time to fix yet. If you want to try, you can follow the instructions on the README and then check the output on the shell (the Docker logs are reproduced there with the start-api command).

I just checked the logs from production and it looks to be the same error code (HTTP 413) meaning the return payload is too large:

2021-11-30T12:41:26.489-05:00 | START RequestId: 411a41ca-b560-4e63-8337-c912efd8b8e2 Version: $LATEST
-- | --
  | 2021-11-30T12:41:26.492-05:00 | Payload {'species': [{'molecule': 'CO', 'mole_fraction': 0.1}], 'min_wavenumber_range': 1900, 'max_wavenumber_range': 2300, 'tgas': 300, 'tvib': None, 'trot': None, 'pressure': 1.01325, 'path_length': 1, 'simulate_slit': False, 'mode': 'absorbance'}
  | 2021-11-30T12:41:26.540-05:00 | HAPI version: 1.2.2.0
  | 2021-11-30T12:41:26.540-05:00 | To get the most up-to-date version please check http://hitran.org/hapi
  | 2021-11-30T12:41:26.540-05:00 | ATTENTION: Python versions of partition sums from TIPS-2021 are now available in HAPI code
  | 2021-11-30T12:41:26.540-05:00 | MIT license: Copyright 2021 HITRAN team, see more at http://hitran.org.
  | 2021-11-30T12:41:26.540-05:00 | If you use HAPI in your research or software development,
  | 2021-11-30T12:41:26.540-05:00 | please cite it using the following reference:
  | 2021-11-30T12:41:26.540-05:00 | R.V. Kochanov, I.E. Gordon, L.S. Rothman, P. Wcislo, C. Hill, J.S. Wilzewski,
  | 2021-11-30T12:41:26.540-05:00 | HITRAN Application Programming Interface (HAPI): A comprehensive approach
  | 2021-11-30T12:41:26.540-05:00 | to working with spectroscopic data, J. Quant. Spectrosc. Radiat. Transfer 177, 15-30 (2016)
  | 2021-11-30T12:41:26.540-05:00 | DOI: 10.1016/j.jqsrt.2016.03.005
  | 2021-11-30T12:41:26.540-05:00 | ATTENTION: This is the core version of the HITRAN Application Programming Interface.
  | 2021-11-30T12:41:26.540-05:00 | For more efficient implementation of the absorption coefficient routine,
  | 2021-11-30T12:41:26.540-05:00 | as well as for new profiles, parameters and other functional,
  | 2021-11-30T12:41:26.540-05:00 | please consider using HAPI2 extension library.
  | 2021-11-30T12:41:26.540-05:00 | HAPI2 package is available at http://github.com/hitranonline/hapi2
  | 2021-11-30T12:41:26.552-05:00 | Using /tmp/.radisdb/hitran/downloads__can_be_deleted/CO
  | 2021-11-30T12:41:26.552-05:00 | Data is fetched from http://hitran.org
  | 2021-11-30T12:41:26.788-05:00 | BEGIN DOWNLOAD: CO_1
  | 2021-11-30T12:41:26.859-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_1.data
  | 2021-11-30T12:41:26.883-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_1.data
  | 2021-11-30T12:41:26.885-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_1.data
  | 2021-11-30T12:41:26.886-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_1.data
  | 2021-11-30T12:41:26.887-05:00 | Header written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_1.header
  | 2021-11-30T12:41:26.887-05:00 | END DOWNLOAD
  | 2021-11-30T12:41:26.904-05:00 | Lines parsed: 1344
  | 2021-11-30T12:41:26.904-05:00 | PROCESSED
  | 2021-11-30T12:41:26.905-05:00 | Data is fetched from http://hitran.org
  | 2021-11-30T12:41:27.098-05:00 | BEGIN DOWNLOAD: CO_2
  | 2021-11-30T12:41:27.176-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_2.data
  | 2021-11-30T12:41:27.199-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_2.data
  | 2021-11-30T12:41:27.200-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_2.data
  | 2021-11-30T12:41:27.201-05:00 | Header written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_2.header
  | 2021-11-30T12:41:27.201-05:00 | END DOWNLOAD
  | 2021-11-30T12:41:27.214-05:00 | Lines parsed: 1042
  | 2021-11-30T12:41:27.215-05:00 | PROCESSED
  | 2021-11-30T12:41:27.215-05:00 | Data is fetched from http://hitran.org
  | 2021-11-30T12:41:27.406-05:00 | BEGIN DOWNLOAD: CO_3
  | 2021-11-30T12:41:27.454-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_3.data
  | 2021-11-30T12:41:27.502-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_3.data
  | 2021-11-30T12:41:27.502-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_3.data
  | 2021-11-30T12:41:27.504-05:00 | Header written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_3.header
  | 2021-11-30T12:41:27.504-05:00 | END DOWNLOAD
  | 2021-11-30T12:41:27.515-05:00 | Lines parsed: 920
  | 2021-11-30T12:41:27.516-05:00 | PROCESSED
  | 2021-11-30T12:41:27.516-05:00 | Data is fetched from http://hitran.org
  | 2021-11-30T12:41:27.711-05:00 | BEGIN DOWNLOAD: CO_4
  | 2021-11-30T12:41:27.759-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_4.data
  | 2021-11-30T12:41:27.783-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_4.data
  | 2021-11-30T12:41:27.785-05:00 | Header written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_4.header
  | 2021-11-30T12:41:27.785-05:00 | END DOWNLOAD
  | 2021-11-30T12:41:27.795-05:00 | Lines parsed: 800
  | 2021-11-30T12:41:27.795-05:00 | PROCESSED
  | 2021-11-30T12:41:27.796-05:00 | Data is fetched from http://hitran.org
  | 2021-11-30T12:41:28.024-05:00 | BEGIN DOWNLOAD: CO_5
  | 2021-11-30T12:41:28.113-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_5.data
  | 2021-11-30T12:41:28.119-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_5.data
  | 2021-11-30T12:41:28.119-05:00 | Header written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_5.header
  | 2021-11-30T12:41:28.119-05:00 | END DOWNLOAD
  | 2021-11-30T12:41:28.136-05:00 | Lines parsed: 674
  | 2021-11-30T12:41:28.138-05:00 | PROCESSED
  | 2021-11-30T12:41:28.138-05:00 | Data is fetched from http://hitran.org
  | 2021-11-30T12:41:28.323-05:00 | BEGIN DOWNLOAD: CO_6
  | 2021-11-30T12:41:28.372-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_6.data
  | 2021-11-30T12:41:28.372-05:00 | 65536 bytes written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_6.data
  | 2021-11-30T12:41:28.373-05:00 | Header written to /tmp/.radisdb/hitran/downloads__can_be_deleted/CO/CO_6.header
  | 2021-11-30T12:41:28.373-05:00 | END DOWNLOAD
  | 2021-11-30T12:41:28.381-05:00 | Lines parsed: 601
  | 2021-11-30T12:41:28.381-05:00 | PROCESSED
  | 2021-11-30T12:41:31.157-05:00 | Added HITRAN-CO database in /tmp/radis.json
  | 2021-11-30T12:41:31.210-05:00 | Calculating Equilibrium Spectrum
  | 2021-11-30T12:41:31.210-05:00 | Physical Conditions
  | 2021-11-30T12:41:31.210-05:00 | ----------------------------------------
  | 2021-11-30T12:41:31.210-05:00 | Tgas 300 K
  | 2021-11-30T12:41:31.210-05:00 | Trot 300 K
  | 2021-11-30T12:41:31.210-05:00 | Tvib 300 K
  | 2021-11-30T12:41:31.210-05:00 | isotope 1,2,3
  | 2021-11-30T12:41:31.210-05:00 | mole_fraction 0.1
  | 2021-11-30T12:41:31.210-05:00 | molecule CO
  | 2021-11-30T12:41:31.210-05:00 | overpopulation None
  | 2021-11-30T12:41:31.210-05:00 | path_length 1 cm
  | 2021-11-30T12:41:31.210-05:00 | pressure_mbar 1013.25 mbar
  | 2021-11-30T12:41:31.210-05:00 | rot_distribution boltzmann
  | 2021-11-30T12:41:31.211-05:00 | self_absorption True
  | 2021-11-30T12:41:31.211-05:00 | state X
  | 2021-11-30T12:41:31.211-05:00 | vib_distribution boltzmann
  | 2021-11-30T12:41:31.211-05:00 | wavenum_max 2300.0000 cm-1
  | 2021-11-30T12:41:31.211-05:00 | wavenum_min 1900.0000 cm-1
  | 2021-11-30T12:41:31.211-05:00 | Computation Parameters
  | 2021-11-30T12:41:31.211-05:00 | ----------------------------------------
  | 2021-11-30T12:41:31.211-05:00 | Tref 296 K
  | 2021-11-30T12:41:31.211-05:00 | add_at_used
  | 2021-11-30T12:41:31.211-05:00 | broadening_method voigt
  | 2021-11-30T12:41:31.211-05:00 | cutoff 1e-27 cm-1/(#.cm-2)
  | 2021-11-30T12:41:31.211-05:00 | dbformat hitran
  | 2021-11-30T12:41:31.211-05:00 | dbpath /tmp/.radisdb/hitran/CO.hdf5
  | 2021-11-30T12:41:31.211-05:00 | dlm_log_pG 0.1375350788016573
  | 2021-11-30T12:41:31.211-05:00 | dlm_log_pL 0.20180288881201608
  | 2021-11-30T12:41:31.211-05:00 | export_rovib_fraction True
  | 2021-11-30T12:41:31.211-05:00 | folding_thresh 1e-06
  | 2021-11-30T12:41:31.211-05:00 | include_neighbouring_lines True
  | 2021-11-30T12:41:31.211-05:00 | levelsfmt None
  | 2021-11-30T12:41:31.211-05:00 | memory_mapping_engine auto
  | 2021-11-30T12:41:31.211-05:00 | neighbour_lines 0 cm-1
  | 2021-11-30T12:41:31.211-05:00 | optimization min-RMS
  | 2021-11-30T12:41:31.211-05:00 | parfuncfmt hapi
  | 2021-11-30T12:41:31.211-05:00 | parfuncpath None
  | 2021-11-30T12:41:31.211-05:00 | parsum_mode full summation
  | 2021-11-30T12:41:31.211-05:00 | pseudo_continuum_threshold 0
  | 2021-11-30T12:41:31.211-05:00 | sparse_dlm auto
  | 2021-11-30T12:41:31.211-05:00 | truncation 50 cm-1
  | 2021-11-30T12:41:31.211-05:00 | wavenum_max_calc 2300.0000 cm-1
  | 2021-11-30T12:41:31.211-05:00 | wavenum_min_calc 1900.0000 cm-1
  | 2021-11-30T12:41:31.211-05:00 | waveunit cm-1
  | 2021-11-30T12:41:31.211-05:00 | wstep 0.01 cm-1
  | 2021-11-30T12:41:31.211-05:00 | zero_padding -1
  | 2021-11-30T12:41:31.211-05:00 | Information
  | 2021-11-30T12:41:31.211-05:00 | ----------------------------------------
  | 2021-11-30T12:41:31.211-05:00 | chunksize None
  | 2021-11-30T12:41:31.211-05:00 | db_use_cached True
  | 2021-11-30T12:41:31.211-05:00 | export_lines False
  | 2021-11-30T12:41:31.211-05:00 | export_populations None
  | 2021-11-30T12:41:31.211-05:00 | load_energies False
  | 2021-11-30T12:41:31.211-05:00 | lvl_use_cached True
  | 2021-11-30T12:41:31.211-05:00 | total_lines 742
  | 2021-11-30T12:41:31.211-05:00 | warning_broadening_threshold 0.01
  | 2021-11-30T12:41:31.211-05:00 | warning_linestrength_cutoff 0.01
  | 2021-11-30T12:41:31.211-05:00 | ----------------------------------------
  | 2021-11-30T12:41:31.307-05:00 | 0.10s - Spectrum calculated
  | 2021-11-30T12:41:31.418-05:00 | END RequestId: 411a41ca-b560-4e63-8337-c912efd8b8e2
  | 2021-11-30T12:41:31.418-05:00 | REPORT RequestId: 411a41ca-b560-4e63-8337-c912efd8b8e2 Duration: 4926.19 ms Billed Duration: 4927 ms Memory Size: 10240 MB Max Memory Used: 453 MB XRAY TraceId: 1-61a66246-4cab89b679d2f9a1368d972a SegmentId: 10781ebc6e440020 Sampled: true
  | 2021-11-30T12:41:35.361-05:00 | START RequestId: 44a675ec-0b2b-4c0e-9028-2fda61f6dd69 Version: $LATEST
  | 2021-11-30T12:41:35.364-05:00 | Payload {'species': [{'molecule': 'CO', 'mole_fraction': 0.1}], 'min_wavenumber_range': 1900, 'max_wavenumber_range': 10000, 'tgas': 300, 'tvib': None, 'trot': None, 'pressure': 1.01325, 'path_length': 1, 'simulate_slit': False, 'mode': 'absorbance'}
  | 2021-11-30T12:41:35.410-05:00 | Calculating Equilibrium Spectrum
  | 2021-11-30T12:41:35.410-05:00 | Physical Conditions
  | 2021-11-30T12:41:35.410-05:00 | ----------------------------------------
  | 2021-11-30T12:41:35.411-05:00 | Tgas 300 K
  | 2021-11-30T12:41:35.411-05:00 | Trot 300 K
  | 2021-11-30T12:41:35.411-05:00 | Tvib 300 K
  | 2021-11-30T12:41:35.411-05:00 | isotope 1,2,3
  | 2021-11-30T12:41:35.411-05:00 | mole_fraction 0.1
  | 2021-11-30T12:41:35.411-05:00 | molecule CO
  | 2021-11-30T12:41:35.411-05:00 | overpopulation None
  | 2021-11-30T12:41:35.411-05:00 | path_length 1 cm
  | 2021-11-30T12:41:35.411-05:00 | pressure_mbar 1013.25 mbar
  | 2021-11-30T12:41:35.411-05:00 | rot_distribution boltzmann
  | 2021-11-30T12:41:35.411-05:00 | self_absorption True
  | 2021-11-30T12:41:35.411-05:00 | state X
  | 2021-11-30T12:41:35.411-05:00 | vib_distribution boltzmann
  | 2021-11-30T12:41:35.411-05:00 | wavenum_max 10000.0000 cm-1
  | 2021-11-30T12:41:35.411-05:00 | wavenum_min 1900.0000 cm-1
  | 2021-11-30T12:41:35.411-05:00 | Computation Parameters
  | 2021-11-30T12:41:35.411-05:00 | ----------------------------------------
  | 2021-11-30T12:41:35.411-05:00 | Tref 296 K
  | 2021-11-30T12:41:35.411-05:00 | add_at_used
  | 2021-11-30T12:41:35.411-05:00 | broadening_method voigt
  | 2021-11-30T12:41:35.411-05:00 | cutoff 1e-27 cm-1/(#.cm-2)
  | 2021-11-30T12:41:35.411-05:00 | dbformat hitran
  | 2021-11-30T12:41:35.411-05:00 | dbpath /tmp/.radisdb/hitran/CO.hdf5
  | 2021-11-30T12:41:35.411-05:00 | dlm_log_pG 0.1375350788016573
  | 2021-11-30T12:41:35.411-05:00 | dlm_log_pL 0.20180288881201608
  | 2021-11-30T12:41:35.411-05:00 | export_rovib_fraction True
  | 2021-11-30T12:41:35.411-05:00 | folding_thresh 1e-06
  | 2021-11-30T12:41:35.411-05:00 | include_neighbouring_lines True
  | 2021-11-30T12:41:35.411-05:00 | levelsfmt None
  | 2021-11-30T12:41:35.411-05:00 | memory_mapping_engine auto
  | 2021-11-30T12:41:35.411-05:00 | neighbour_lines 0 cm-1
  | 2021-11-30T12:41:35.411-05:00 | optimization min-RMS
  | 2021-11-30T12:41:35.411-05:00 | parfuncfmt hapi
  | 2021-11-30T12:41:35.411-05:00 | parfuncpath None
  | 2021-11-30T12:41:35.411-05:00 | parsum_mode full summation
  | 2021-11-30T12:41:35.411-05:00 | pseudo_continuum_threshold 0
  | 2021-11-30T12:41:35.411-05:00 | sparse_dlm auto
  | 2021-11-30T12:41:35.411-05:00 | truncation 50 cm-1
  | 2021-11-30T12:41:35.411-05:00 | wavenum_max_calc 10000.0000 cm-1
  | 2021-11-30T12:41:35.411-05:00 | wavenum_min_calc 1900.0000 cm-1
  | 2021-11-30T12:41:35.411-05:00 | waveunit cm-1
  | 2021-11-30T12:41:35.411-05:00 | wstep 0.01 cm-1
  | 2021-11-30T12:41:35.411-05:00 | zero_padding -1
  | 2021-11-30T12:41:35.411-05:00 | Information
  | 2021-11-30T12:41:35.411-05:00 | ----------------------------------------
  | 2021-11-30T12:41:35.411-05:00 | chunksize None
  | 2021-11-30T12:41:35.411-05:00 | db_use_cached True
  | 2021-11-30T12:41:35.411-05:00 | export_lines False
  | 2021-11-30T12:41:35.411-05:00 | export_populations None
  | 2021-11-30T12:41:35.411-05:00 | load_energies False
  | 2021-11-30T12:41:35.411-05:00 | lvl_use_cached True
  | 2021-11-30T12:41:35.411-05:00 | total_lines 2100
  | 2021-11-30T12:41:35.411-05:00 | warning_broadening_threshold 0.01
  | 2021-11-30T12:41:35.411-05:00 | warning_linestrength_cutoff 0.01
  | 2021-11-30T12:41:35.411-05:00 | ----------------------------------------
  | 2021-11-30T12:41:36.059-05:00 | 0.65s - Spectrum calculated
  | 2021-11-30T12:41:36.552-05:00 | [ERROR] [1638294096552] LAMBDA_RUNTIME Failed to post handler success response. Http response code: 413.
  | 2021-11-30T12:41:36.553-05:00 | Traceback (most recent call last):
  | 2021-11-30T12:41:36.553-05:00 | File "/var/runtime/bootstrap.py", line 480, in <module>
  | 2021-11-30T12:41:36.553-05:00 | main()
  | 2021-11-30T12:41:36.553-05:00 | File "/var/runtime/bootstrap.py", line 468, in main
  | 2021-11-30T12:41:36.554-05:00 | handle_event_request(lambda_runtime_client,
  | 2021-11-30T12:41:36.554-05:00 | File "/var/runtime/bootstrap.py", line 148, in handle_event_request
  | 2021-11-30T12:41:36.554-05:00 | lambda_runtime_client.post_invocation_result(invoke_id, result, result_content_type)
  | 2021-11-30T12:41:36.554-05:00 | File "/var/runtime/lambda_runtime_client.py", line 62, in post_invocation_result
  | 2021-11-30T12:41:36.554-05:00 | rapid_client.post_invocation_result(invoke_id, result_data if isinstance(result_data, bytes) else result_data.encode('utf-8'), content_type)
  | 2021-11-30T12:41:36.554-05:00 | RuntimeError: Failed to post invocation response
  | 2021-11-30T12:41:37.111-05:00 | END RequestId: 44a675ec-0b2b-4c0e-9028-2fda61f6dd69
  | 2021-11-30T12:41:37.111-05:00 | REPORT RequestId: 44a675ec-0b2b-4c0e-9028-2fda61f6dd69 Duration: 1747.99 ms Billed Duration: 1748 ms Memory Size: 10240 MB Max Memory Used: 551 MB XRAY TraceId: 1-61a6624f-443de33e063377543db4716e SegmentId: 177817de5d68d56c Sampled: true
  | 2021-11-30T12:41:37.111-05:00 | RequestId: 44a675ec-0b2b-4c0e-9028-2fda61f6dd69 Error: Runtime exited with error: exit status 1 Runtime.ExitError
  | 2021-11-30T12:41:37.430-05:00 | OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
  | 2021-11-30T12:41:37.625-05:00 | OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
  | 2021-11-30T12:41:38.191-05:00 | /var/task/joblib/_multiprocessing_helpers.py:45: UserWarning: [Errno 38] Function not implemented. joblib will operate in serial mode
  | 2021-11-30T12:41:38.191-05:00 | warnings.warn('%s. joblib will operate in serial mode' % (e,))
erwanp commented 2 years ago

Are we sure that we are in the synchronous and not asynchronous mode? Else the limit is 250 kb

suzil commented 2 years ago

It's definitely synchronous. If it wasn't, you wouldn't ever get an error returned back to the frontend from the backend.

I added a print message here to show the size with UTF-8 encoding: https://github.com/suzil/radis-app/blob/master/lib/radis-lambda/app.py#L93

When I increase the upper range to the max value, I get:

Size of result 7338209

which is 7.34 MB which is over the 6 MB limit.

erwanp commented 2 years ago

Worked !

image