marella / chatdocs

Chat with your documents offline using AI.
MIT License
685 stars 98 forks source link

Crash on low end PC #3

Open PotatoDestroyer opened 1 year ago

PotatoDestroyer commented 1 year ago

crashing after adding a lot of PDFs on low end PC

# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 258032 bytes for Chunk::new
# Possible reasons:
#   The system is out of physical RAM or swap space
#   The process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
#   JVM is running with Unscaled Compressed Oops mode in which the Java heap is
#     placed in the first 4GB address space. The Java Heap base address is the
#     maximum limit for the native heap growth. Please use -XX:HeapBaseMinAddress
#     to set the Java Heap base and to place the Java Heap above 4GB virtual address.
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (arena.cpp:189), pid=13844, tid=23816
#
# JRE version: OpenJDK Runtime Environment JBR-17.0.6+10-829.9-jcef (17.0.6+10) (build 17.0.6+10-b829.9)
# Java VM: OpenJDK 64-Bit Server VM JBR-17.0.6+10-829.9-jcef (17.0.6+10-b829.9, mixed mode, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
---------------  S U M M A R Y ------------

Command Line: exit -XX:ErrorFile=C:\Users\support2\\java_error_in_pycharm64_%p.log -XX:HeapDumpPath=C:\Users\support2\\java_error_in_pycharm64.hprof -Xms128m -Xmx750m -XX:ReservedCodeCacheSize=512m -XX:+UseG1GC -XX:SoftRefLRUPolicyMSPerMB=50 -XX:CICompilerCount=2 -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -XX:+IgnoreUnrecognizedVMOptions -XX:CompileCommand=exclude,com/intellij/openapi/vfs/impl/FilePartNodeRoot,trieDescend -ea -Dsun.io.useCanonCaches=false -Dsun.java2d.metal=true -Djbr.catch.SIGABRT=true -Djdk.http.auth.tunneling.disabledSchemes="" -Djdk.attach.allowAttachSelf=true -Djdk.module.illegalAccess.silent=true -Dkotlinx.coroutines.debug=off -Xmx987m -Djb.vmOptionsFile=C:\Users\support2\AppData\Roaming\\JetBrains\\PyCharmCE2023.1\pycharm64.exe.vmoptions -Djava.system.class.loader=com.intellij.util.lang.PathClassLoader -Didea.vendor.name=JetBrains -Didea.paths.selector=PyCharmCE2023.1 -Djna.boot.library.path=C:\Program Files\JetBrains\PyCharm Community Edition 2023.1.2/lib/jna/amd64 -Dpty4j.preferred.native.folder=C:\Program Files\JetBrains\PyCharm Community Edition 2023.1.2/lib/pty4j -Djna.nosys=true -Djna.noclasspath=true -Didea.platform.prefix=PyCharmCore -Dsplash=true --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.ref=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED --add-opens=java.base/java.time=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/jdk.internal.vm=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.fs=ALL-UNNAMED --add-opens=java.base/sun.security.ssl=ALL-UNNAMED --add-opens=java.base/sun.security.util=ALL-UNNAMED --add-opens=java.base/sun.net.dns=ALL-UNNAMED --add-opens=java.desktop/java.awt=ALL-UNNAMED --add-opens=java.desktop/java.awt.dnd.peer=ALL-UNNAMED --add-opens=java.desktop/java.awt.event=ALL-UNNAMED --add-opens=java.desktop/java.awt.image=ALL-UNNAMED --add-opens=java.desktop/java.awt.peer=ALL-UNNAMED --add-opens=java.desktop/java.awt.font=ALL-UNNAMED --add-opens=java.desktop/javax.swing=ALL-UNNAMED --add-opens=java.desktop/javax.swing.plaf.basic=ALL-UNNAMED --add-opens=java.desktop/javax.swing.text.html=ALL-UNNAMED --add-opens=java.desktop/sun.awt.datatransfer=ALL-UNNAMED --add-opens=java.desktop/sun.awt.image=ALL-UNNAMED --add-opens=java.desktop/sun.awt.windows=ALL-UNNAMED --add-opens=java.desktop/sun.awt=ALL-UNNAMED --add-opens=java.desktop/sun.font=ALL-UNNAMED --add-opens=java.desktop/sun.java2d=ALL-UNNAMED --add-opens=java.desktop/sun.swing=ALL-UNNAMED --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-opens=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED --add-opens=jdk.jdi/com.sun.tools.jdi=ALL-UNNAMED -Dide.native.launcher=true -Djcef.sandbox.ptr=0000020E90174FA0 

Host: 12th Gen Intel(R) Core(TM) i7-12700, 20 cores, 7G,  Windows 10 , 64 bit Build 19041 (10.0.19041.2913)
Time: Wed Jun  7 12:36:17 2023 Arabian Standard Time elapsed time: 32.528581 seconds (0d 0h 0m 32s)

---------------  T H R E A D  ---------------

Current thread (0x0000020ebc83f590):  JavaThread "C2 CompilerThread0" daemon [_thread_in_native, id=23816, stack(0x000000f0b5800000,0x000000f0b5900000)]

Current CompileTask:
C2:  32528 25800   !   4       com.intellij.ide.IdeEventQueue::dispatchByCustomDispatchers (110 bytes)

Stack: [0x000000f0b5800000,0x000000f0b5900000]
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [jvm.dll+0x683c5a]
V  [jvm.dll+0x842764]
V  [jvm.dll+0x843f5e]
V  [jvm.dll+0x8445c3]
V  [jvm.dll+0x249b75]
V  [jvm.dll+0xabcac]
V  [jvm.dll+0xac27c]
V  [jvm.dll+0x368857]
V  [jvm.dll+0x1bd0b8]
V  [jvm.dll+0x21c359]
V  [jvm.dll+0x21b621]
V  [jvm.dll+0x1a4fdd]
V  [jvm.dll+0x22b098]
V  [jvm.dll+0x229159]
V  [jvm.dll+0x7f81ac]
V  [jvm.dll+0x7f270a]
V  [jvm.dll+0x682a95]
C  [ucrtbase.dll+0x21bb2]
C  [KERNEL32.DLL+0x17614]
C  [ntdll.dll+0x526a1]

---------------  S Y S T E M  ---------------

OS:
 Windows 10 , 64 bit Build 19041 (10.0.19041.2913)
OS uptime: 7 days 4:23 hours
Hyper-V role detected

CPU: total 20 (initial active 20) (10 cores per cpu, 2 threads per core) family 6 model 151 stepping 2 microcode 0x1f, cx8, cmov, fxsr, ht, mmx, 3dnowpref, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, lzcnt, tsc, tscinvbit, avx, avx2, aes, erms, clmul, bmi1, bmi2, adx, sha, fma, vzeroupper, clflush, clflushopt, clwb, hv

Memory: 4k page, system-wide physical 7897M (408M free)
TotalPageFile size 32473M (AvailPageFile size 0M)
current process WorkingSet (physical memory assigned to process): 1128M, peak: 1133M
current process commit charge ("private bytes"): 1004M, peak: 1008M

vm_info: OpenJDK 64-Bit Server VM (17.0.6+10-b829.9) for windows-amd64 JRE (17.0.6+10-b829.9), built on 2023-04-09 by "builduser" with MS VC++ 16.10 / 16.11 (VS2019)

END.

>
94bb494nd41f commented 1 year ago

What kind of model were you trying to load? I guess 8 GB is not that much but should work fine with smaller modells like https://huggingface.co/TheBloke/wizardLM-7B-GGML although even the smalles modell need 6.3 Gb at least.