search

Jacpowerz / python_link_crawler

0 stars 0 forks source link

Program gets killed after too much RAM usage #1

Open Jacpowerz opened 4 months ago

Jacpowerz commented 4 months ago

If too much RAM is used it will exit the program immediately without getting caught by an exception. Leads to data loss

Jacpowerz commented 2 months ago

Added a whole Overkill Manager thing to try and move the memory usage onto the drive and basically run the tasks using CPU instead but it doesn't;t work. I need more precise control of the RAM for that or some way to work with the garbage collector. I may need to use a different language, but it's a lot of work to rewrite the entire application. Might be able to write a smaller feature in a different language and integrate it, but that idea needs more research.