search

markddrake / YADAMU---Yet-Another-DAta-Migration-Utility

A utility for Migrating Data between Oracle, Postgres, MySQL MariaDB, Snowflake. Stage Data from supported database to Amazon S3 and Azure Blob Storage in JSON and CSV Formats
MIT License
17 stars 9 forks source link

Node reports "Warning: too many listeners - possible memory leak" during export operations #210

Closed markddrake closed 2 years ago

markddrake commented 2 years ago

Node rep

markddrake commented 2 years ago

Node generates warning message during export operations if the schema contains more that 4 tables.

The warning is that too many listeners have been attached to the writeStream object, and that this could cause a memory leak.

markddrake commented 2 years ago

This appears to be coming from dbReaderFile.js. It appears that using the 'end: false' option with the streams-based pipeline causes the some of the 'error', 'closes' and finish' event handlers added by the pipeline operation not to be removed when the pipeline completed. A workaround has been added to track the set of event handlers that are present before the pipeline operation commences and remove any handlers that were not present after the pipeline operation has completed.