search

datajoint / datajoint-python

Relational data pipelines for the science lab
https://datajoint.com/docs
GNU Lesser General Public License v2.1
169 stars 84 forks source link

LostConnectionError during a long-duration make #1195

Open arturoptophys opened 4 days ago

arturoptophys commented 4 days ago

Bug Report

Description

I"m running some longer (several hours) calculations (spikesorting) in the make of a table, occasionally I get LostConnectionError: Connection was lost during a transaction, although everything ran correctly. Before I also get [WARNING]: MySQL server has gone away. Reconnecting to the server.

In the make I have only a single insert call. I assume the transaction is started when call to make is made? Solution at the moment is to call populate again and load the processed files instead of generating them.

Reproducibility

Difficult to reliably reproduce as it doesn't always occur and seem to depend on a re connection happening during make call.

Expected Behavior

Make should handle long calculations and potential re connection during this time.

Additional Research and Context

not sure if this is related issue https://github.com/datajoint/datajoint-python/issues/896

Best, Artur

dimitri-yatsenko commented 4 days ago

This is solved by #1171 @ethho