search

PASTAplus / ezEML

A web front-end for the metapype-eml client
Apache License 2.0
7 stars 8 forks source link

Prevent out-of-memory failures in Check Data Tables #149

Closed jon-ide closed 10 months ago

jon-ide commented 11 months ago

Running Check Data Tables with a very large table can take down the ezEML service. E.g., high_frequency_temperature_profiles.csv submitted by user Isabella Oleksy.

jon-ide commented 11 months ago

For now, we will limit checking to the first 1 million rows and display a message to that effect to the user. We will consider using a Task Queue to run very large tables in chunks, in the background, with a callback to the client.

jon-ide commented 10 months ago

Closed in release 2024.01.03.