OOM from JavaScript

[gabrielecaruso]

Hello, first of all, thanks you for this software! Is amazing and is just what i was searching for!

I just got an error...i'm importing like 800GB of media, all went good for 3-4 days but today i have this error:

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory [2024-01-08 08:05:55.326]: cli.spawn Cli extract --index /data/config/Privati.idx --journal 0108-GPnI exited by signal SIGABRT [2024-01-08 08:05:55.502]: cli.task.import error Failed to import: [object Object] [2024-01-08 08:05:55.517]: cli.task.import Stop watching files due error [2024-01-08 08:06:16.086]: cli.run error Error: [object Object]

Is there a way to increase the max javascript memory?

[xemle]

Hi @gabrielecaruso

thank you very much and I am glad that you found HomeGallery. Welcome

I just got an error...i'm importing like 800GB of media, all went good for 3-4 days but today i have this error:

First of all the 800 GB depends how many files you have. 800 GB can be 8 huge videos or hundreds of thousands of images. Depending on the file count the file indexer seems to reach a file limit. This error happens after some time since I assume that you are bootstrapping your gallery and the bootstrapping is done iteratively (files are added not all at once and the file index grows over time). So how many files to you have in total? Do you have also some further context before the error - as best in trace log level?

I know this limit for the database creation and there is an option for fixing it in the example configuration - but not for the index step :-(

According to stack overflow, you can fix it by an node environment variable:

NODE_OPTIONS=--max-old-space-size=2048

Hope that helps to fix you issue.

[xemle]

The new v1.16.0 has a stream based database processing and should handle the database creation with less memory. Maybe it worth trying it.

Further, I am closing this issue due inactivity.

Please reopen it, if it becomes active again.