search

developersu / ns-usbloader

Awoo Installer and GoldLeaf uploader of the NSPs (and other files), RCM payload injector, application for split/merge files.
GNU General Public License v3.0
1.81k stars 123 forks source link

Linux Out Of Memory bug #40

Open ghost opened 4 years ago

ghost commented 4 years ago

My System: Linux user 5.4.2-1-MANJARO #1 SMP PREEMPT Thu Dec 5 09:55:57 UTC 2019 x86_64 GNU/Linux

i tried with java openjdk 13 and openjdk 8

Everytime when i try to install something over usb, my system becomes unresponsive, i tried different distributions as i thought my manjaro system had a problem but with different live-isos its still the same.

could only get some logs on pop OS so thats from there

[ 764.484796] pop-upgrade invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0 [ 764.484798] CPU: 0 PID: 1825 Comm: pop-upgrade Tainted: G OE 5.3.0-7625-generic #27~1576337002~19.10~bc3488b-Ubuntu [ 764.484799] Hardware name: Gigabyte Technology Co., Ltd. B450 AORUS M/B450 AORUS M, BIOS F50 11/27/2019 [ 764.484799] Call Trace: [ 764.484804] dump_stack+0x6d/0x9a [ 764.484806] dump_header+0x4f/0x1eb [ 764.484807] oom_kill_process.cold+0xb/0x10 [ 764.484808] out_of_memory.part.0+0x1df/0x3d0 [ 764.484809] out_of_memory+0x6d/0xd0 [ 764.484810] alloc_pages_slowpath+0xd3d/0xe30 [ 764.484812] ? page_cache_release+0x3d/0x210 [ 764.484814] alloc_pages_nodemask+0x2d0/0x320 [ 764.484815] alloc_pages_current+0x87/0xe0 [ 764.484817] page_cache_alloc+0x72/0x90 [ 764.484818] pagecache_get_page+0xb0/0x2f0 [ 764.484819] filemap_fault+0x6e3/0xad0 [ 764.484821] ? alloc_set_pte+0x118/0x610 [ 764.484822] ? xas_find+0x16c/0x1b0 [ 764.484823] ? filemap_map_pages+0x1b3/0x360 [ 764.484824] do_fault+0x3c/0x130 [ 764.484825] do_fault+0x248/0x640 [ 764.484826] ? switch_to_asm+0x40/0x70 [ 764.484827] __handle_mm_fault+0x4c5/0x7a0 [ 764.484828] handle_mm_fault+0xca/0x1f0 [ 764.484830] do_user_addr_fault+0x1f9/0x450 [ 764.484831] __do_page_fault+0x58/0x90 [ 764.484832] do_page_fault+0x2c/0xe0 [ 764.484833] page_fault+0x34/0x40 [ 764.484834] RIP: 0033:0x7f88f447653a [ 764.484837] Code: Bad RIP value. [ 764.484837] RSP: 002b:00007ffcf84587b0 EFLAGS: 00010246 [ 764.484838] RAX: 0000000000000000 RBX: 000056551ddb95f0 RCX: 0000000000000000 [ 764.484839] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 000056551ddb9730 [ 764.484839] RBP: 0000000000000001 R08: 0000000000000000 R09: 000056551ddb9c20 [ 764.484840] R10: 00007ffcf8457b04 R11: 0000000000000000 R12: 00000000000003e8 [ 764.484840] R13: 000056551bb69440 R14: 000056551ddbb830 R15: 00000000000003e8 [ 764.484841] Mem-Info: [ 764.484844] active_anon:268893 inactive_anon:3063779 isolated_anon:0 active_file:399 inactive_file:451 isolated_file:2 unevictable:0 dirty:0 writeback:0 unstable:0 slab_reclaimable:35319 slab_unreclaimable:76908 mapped:20145 shmem:3108565 pagetables:3526 bounce:0 free:33210 free_pcp:153 free_cma:0 [ 764.484845] Node 0 active_anon:1075572kB inactive_anon:12255116kB active_file:1596kB inactive_file:1804kB unevictable:0kB isolated(anon):0kB isolated(file):8kB mapped:80580kB dirty:0kB writeback:0kB shmem:12434260kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [ 764.484846] Node 0 DMA free:15860kB min:64kB low:80kB high:96kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15996kB managed:15860kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [ 764.484848] lowmem_reserve[]: 0 3356 15856 15856 15856 [ 764.484849] Node 0 DMA32 free:63916kB min:14292kB low:17864kB high:21436kB active_anon:132kB inactive_anon:1938648kB active_file:560kB inactive_file:1044kB unevictable:0kB writepending:0kB present:3617024kB managed:3592364kB mlocked:0kB kernel_stack:0kB pagetables:28kB bounce:0kB free_pcp:556kB local_pcp:28kB free_cma:0kB [ 764.484851] lowmem_reserve[]: 0 0 12500 12500 12500 [ 764.484852] Node 0 Normal free:53064kB min:53224kB low:66528kB high:79832kB active_anon:1075440kB inactive_anon:10316468kB active_file:888kB inactive_file:1064kB unevictable:0kB writepending:0kB present:13094400kB managed:12800500kB mlocked:0kB kernel_stack:11344kB pagetables:14076kB bounce:0kB free_pcp:56kB local_pcp:8kB free_cma:0kB [ 764.484853] lowmem_reserve[]: 0 0 0 0 0 [ 764.484854] Node 0 DMA: 14kB (U) 28kB (U) 216kB (U) 032kB 364kB (U) 0128kB 1256kB (U) 0512kB 11024kB (U) 12048kB (M) 34096kB (M) = 15860kB [ 764.484858] Node 0 DMA32: 784kB (ME) 378kB (UME) 2416kB (ME) 832kB (UME) 1264kB (UME) 6128kB (UME) 0256kB 3512kB (UM) 31024kB (UME) 22048kB (UE) 134096kB (UM) = 64736kB [ 764.484862] Node 0 Normal: 1954kB (UME) 2608kB (ME) 22716kB (ME) 24832kB (ME) 7464kB (ME) 7128kB (ME) 7256kB (M) 5512kB (M) 51024kB (M) 122048kB (U) 0*4096kB = 54108kB [ 764.484866] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB [ 764.484867] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [ 764.484867] 3109661 total pagecache pages [ 764.484868] 0 pages in swap cache [ 764.484868] Swap cache stats: add 0, delete 0, find 0/0 [ 764.484868] Free swap = 0kB [ 764.484869] Total swap = 0kB [ 764.484869] 4181855 pages RAM [ 764.484869] 0 pages HighMem/MovableOnly [ 764.484869] 79674 pages reserved [ 764.484870] 0 pages cma reserved [ 764.484870] 0 pages hwpoisoned [ 764.484870] Tasks state (memory values in pages): [ 764.484870] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [ 764.484875] [ 1387] 0 1387 6706 1167 98304 0 0 systemd-journal [ 764.484876] [ 1436] 0 1436 5457 486 69632 0 -1000 systemd-udevd [ 764.484877] [ 1572] 100 1572 23083 224 81920 0 0 systemd-timesyn [ 764.484878] [ 1573] 102 1573 5197 316 77824 0 0 systemd-resolve [ 764.484880] [ 1786] 104 1786 56090 201 77824 0 0 rsyslogd [ 764.484881] [ 1788] 0 1788 5213 68 57344 0 0 cron [ 764.484881] [ 1789] 0 1789 2009 83 57344 0 0 system76-power [ 764.484882] [ 1803] 114 1803 2151 131 57344 0 0 avahi-daemon [ 764.484883] [ 1820] 0 1820 62386 215 94208 0 0 accounts-daemon [ 764.484884] [ 1825] 0 1825 174585 809 163840 0 0 pop-upgrade [ 764.484885] [ 1826] 0 1826 30488 1197 98304 0 0 cupsd [ 764.484886] [ 1827] 0 1827 20473 96 53248 0 0 irqbalance [ 764.484886] [ 1839] 0 1839 4099 248 61440 0 0 systemd-logind [ 764.484887] [ 1842] 0 1842 98637 957 135168 0 0 udisksd [ 764.484888] [ 1850] 0 1850 59932 411 110592 0 0 ModemManager [ 764.484889] [ 1856] 103 1856 2398 558 53248 0 -900 dbus-daemon [ 764.484890] [ 1866] 0 1866 3404 158 65536 0 0 wpa_supplicant [ 764.484891] [ 1874] 0 1874 632 44 45056 0 0 acpid [ 764.484891] [ 1878] 0 1878 12509 2081 106496 0 0 networkd-dispat [ 764.484892] [ 1880] 0 1880 87760 704 163840 0 0 NetworkManager [ 764.484893] [ 1910] 114 1910 2087 80 57344 0 0 avahi-daemon [ 764.484894] [ 1925] 0 1925 46314 1279 135168 0 0 cups-browsed [ 764.484895] [ 2214] 0 2214 63049 815 106496 0 0 polkitd [ 764.484895] [ 2248] 0 2248 62712 243 98304 0 0 gdm3 [ 764.484896] [ 2264] 0 2264 46032 328 94208 0 0 gdm-session-wor [ 764.484897] [ 2270] 999 2270 4699 515 73728 0 0 systemd [ 764.484898] [ 2271] 999 2271 43582 749 110592 0 0 (sd-pam) [ 764.484899] [ 2296] 0 2296 96832 849 172032 0 0 fwupd [ 764.484899] [ 2298] 999 2298 62855 238 102400 0 0 gnome-keyring-d [ 764.484900] [ 2303] 999 2303 43849 159 77824 0 0 gdm-x-session [ 764.484901] [ 2305] 999 2305 875025 15649 790528 0 0 Xorg [ 764.484902] [ 2341] 0 2341 65922 395 110592 0 0 upowerd [ 764.484903] [ 2364] 999 2364 2036 368 53248 0 0 dbus-daemon [ 764.484904] [ 2368] 999 2368 50250 416 126976 0 0 gnome-session-b [ 764.484904] [ 2456] 999 2456 1498 118 49152 0 0 ssh-agent [ 764.484905] [ 2518] 999 2518 62774 252 94208 0 0 gvfsd [ 764.484906] [ 2524] 999 2524 94566 166 98304 0 0 gvfsd-fuse [ 764.484907] [ 2536] 0 2536 62451 200 98304 0 0 boltd [ 764.484908] [ 2610] 999 2610 76327 181 90112 0 0 at-spi-bus-laun [ 764.484908] [ 2615] 999 2615 1801 142 57344 0 0 dbus-daemon [ 764.484909] [ 2624] 999 2624 40670 193 81920 0 0 at-spi2-registr [ 764.484910] [ 2627] 999 2627 153893 445 139264 0 0 xdg-desktop-por [ 764.484911] [ 2654] 999 2654 117208 150 110592 0 0 xdg-document-po [ 764.484911] [ 2658] 999 2658 61761 113 90112 0 0 xdg-permission- [ 764.484912] [ 2672] 999 2672 25359 105 73728 0 0 gnome-session-c [ 764.484913] [ 2675] 999 2675 334848 1244 229376 0 0 pulseaudio [ 764.484914] [ 2688] 999 2688 124233 697 163840 0 0 gnome-session-b [ 764.484915] [ 2695] 106 2695 38218 64 57344 0 0 rtkit-daemon [ 764.484915] [ 2705] 999 2705 1022739 27027 970752 0 0 gnome-shell [ 764.484916] [ 2713] 999 2713 39049 154 69632 0 0 dconf-service [ 764.484917] [ 2830] 999 2830 129555 785 192512 0 0 gnome-shell-cal [ 764.484918] [ 2834] 999 2834 103570 1128 221184 0 0 evolution-sourc [ 764.484919] [ 2877] 999 2877 137055 1423 249856 0 0 goa-daemon [ 764.484919] [ 2885] 999 2885 99766 511 114688 0 0 gvfs-udisks2-vo [ 764.484920] [ 2890] 999 2890 61812 193 81920 0 0 gvfs-mtp-volume [ 764.484921] [ 2894] 999 2894 62412 241 86016 0 0 gvfs-gphoto2-vo [ 764.484922] [ 2898] 999 2898 61832 128 86016 0 0 gvfs-goa-volume [ 764.484923] [ 2907] 999 2907 81542 258 110592 0 0 goa-identity-se [ 764.484924] [ 2912] 999 2912 82013 249 114688 0 0 gvfs-afc-volume [ 764.484925] [ 2918] 0 2918 92496 4982 184320 0 0 packagekitd [ 764.484926] [ 2923] 999 2923 81092 925 94208 0 0 ibus-daemon [ 764.484927] [ 2930] 999 2930 61992 160 81920 0 0 ibus-dconf [ 764.484928] [ 2931] 999 2931 71839 3160 167936 0 0 ibus-extension- [ 764.484929] [ 2933] 999 2933 52592 2318 147456 0 0 ibus-x11 [ 764.484929] [ 2936] 999 2936 61989 117 90112 0 0 ibus-portal [ 764.484930] [ 2949] 999 2949 215609 1184 290816 0 0 evolution-calen [ 764.484931] [ 2959] 999 2959 89689 2481 172032 0 0 gsd-xsettings [ 764.484932] [ 2960] 999 2960 61740 145 81920 0 0 gsd-screensaver [ 764.484932] [ 2961] 999 2961 108150 2466 184320 0 0 gsd-power [ 764.484933] [ 2962] 999 2962 89586 2409 180224 0 0 gsd-keyboard [ 764.484934] [ 2964] 999 2964 80782 238 102400 0 0 gsd-housekeepin [ 764.484935] [ 2965] 999 2965 97999 2188 184320 0 0 gsd-datetime [ 764.484936] [ 2967] 999 2967 117137 152 106496 0 0 gsd-rfkill [ 764.484936] [ 2968] 999 2968 228261 2563 225280 0 0 gsd-media-keys [ 764.484937] [ 2969] 999 2969 119002 378 131072 0 0 gsd-sharing [ 764.484938] [ 2970] 999 2970 81599 229 102400 0 0 gsd-smartcard [ 764.484939] [ 2971] 999 2971 64860 352 110592 0 0 gsd-print-notif [ 764.484940] [ 2972] 999 2972 82640 279 110592 0 0 gsd-sound [ 764.484940] [ 2974] 999 2974 71019 2362 163840 0 0 gsd-wacom [ 764.484941] [ 2975] 999 2975 81341 247 110592 0 0 gsd-wwan [ 764.484942] [ 2978] 999 2978 163980 2432 225280 0 0 gsd-color [ 764.484943] [ 2979] 999 2979 80268 124 94208 0 0 gsd-a11y-settin [ 764.484944] [ 2980] 999 2980 67431 2147 122880 0 0 hidpi-notificat [ 764.484945] [ 2983] 999 2983 200550 2183 225280 0 0 tracker-miner-f [ 764.484946] [ 3025] 999 3025 190796 996 245760 0 0 evolution-addre [ 764.484947] [ 3041] 999 3041 43436 174 90112 0 0 gvfsd-metadata [ 764.484948] [ 3048] 999 3048 57934 339 77824 0 0 gsd-disk-utilit [ 764.484948] [ 3051] 999 3051 110558 4027 200704 0 0 hidpi-daemon [ 764.484949] [ 3056] 999 3056 181595 3815 356352 0 0 evolution-alarm [ 764.484950] [ 3061] 999 3061 43537 197 77824 0 0 ibus-engine-sim [ 764.484951] [ 3122] 116 3122 64421 1332 110592 0 0 colord [ 764.484951] [ 3243] 999 3243 88670 468 159744 0 0 gsd-printer [ 764.484952] [ 3295] 999 3295 184448 8229 344064 0 0 nautilus [ 764.484953] [ 3304] 999 3304 99735 251 118784 0 0 gvfsd-trash [ 764.484954] [ 3351] 7 3351 3776 195 65536 0 0 dbus [ 764.484954] [ 3353] 0 3353 85072 13138 356352 0 0 io.elementary.i [ 764.484955] [ 3364] 0 3364 1776 112 49152 0 0 dbus-launch [ 764.484956] [ 3365] 0 3365 1799 119 49152 0 0 dbus-daemon [ 764.484957] [ 3368] 0 3368 101320 459 122880 0 0 xdg-desktop-por [ 764.484957] [ 3373] 0 3373 117208 125 126976 0 0 xdg-document-po [ 764.484958] [ 3377] 0 3377 61761 112 86016 0 0 xdg-permission- [ 764.484959] [ 3398] 999 3398 99792 270 118784 0 0 gvfsd-network [ 764.484960] [ 3411] 999 3411 81567 273 102400 0 0 gvfsd-dnssd [ 764.484961] [ 3477] 999 3477 650 17 45056 0 0 sh [ 764.484961] [ 3478] 0 3478 118627 533 122880 0 0 gvfsd-admin [ 764.484962] [ 3590] 999 3590 650 17 36864 0 0 repoman.pkexec [ 764.484963] [ 3592] 999 3592 102652 38692 503808 0 0 repoman [ 764.484964] [ 3597] 0 3597 32458 19988 270336 0 -900 python3 [ 764.484964] [ 3629] 999 3629 158036 3400 253952 0 0 gnome-terminal- [ 764.484965] [ 3639] 999 3639 5627 453 49152 0 0 bash [ 764.484966] [ 6262] 0 6262 39017 140 69632 0 0 dconf-service [ 764.484967] [ 6314] 0 6314 7254 154 73728 0 0 sudo [ 764.484968] [ 6315] 0 6315 2702277 62833 1564672 0 0 java [ 764.484969] [ 6434] 0 6434 5213 68 53248 0 0 cron [ 764.484970] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-999.slice/user@999.service,task=java,pid=6315,uid=0 [ 764.485046] Out of memory: Killed process 6315 (java) total-vm:10809108kB, anon-rss:225488kB, file-rss:0kB, shmem-rss:25844kB [ 764.493021] oom_reaper: reaped process 6315 (java), now anon-rss:0kB, file-rss:0kB, shmem-rss:104kB

developersu commented 4 years ago

Using Tinfoil 0.2.1 USB installations?

developersu commented 4 years ago

Tested on Manjaro with JDK 12.0.1 (Java 12) and JDK 1.8 (Java 8). It works for me. Please provide more information regarding what home-brew app did you use. Also it looks like you're out of memory (RAM) so I'm not sure that it's a problem I can fix. Anyway, let me know what do you use first. Thanks!

FichteFoll commented 4 years ago

Just FYI, I also had problems with my Arch freezing completely back in November and haven't tried since, but I intend to make a full report with updated versions soon.

It only happened when I launched as root, but I couldn't get it to work with the udev rule either.

developersu commented 4 years ago

Ok. Try to use 'wheel' group instead of 'pugdev' for example. I did this trick it Manjaro and it works.

jmshrv commented 4 years ago

I'm pretty sure there's a memory leak happening on Manjaro. When I start installing a title via USB, my RAM immediately maxes out and kills performance. I have 16GB RAM.

developersu commented 4 years ago

Try this: java -Xmx512m - jar app.jar

FichteFoll commented 4 years ago

I just made a recording of my RAM overflowing with nothing but ns-usbloader running, even with the options from above. You should be able to see all of the necessary details in the video, but please do ask.

Note that the video stutters when the RAM overflows and pages begin to be swapped. This also happens on my display and not just in the recording.

https://x0.at/Uk1.mp4

OS: Arch Linux 5.5.3 Java: 

openjdk 13.0.2 2020-01-14
OpenJDK Runtime Environment (build 13.0.2+8)
OpenJDK 64-Bit Server VM (build 13.0.2+8, mixed mode)

Log:

Files to upload:
  /data/Games/_Switch/Test.nsp
[ INFO ]    Start chain
[ PASS ] libusb initialization
[ PASS ] Get device list
[ PASS ] Read file descriptors for USB devices
[ PASS ] NS in connected USB devices found
[ PASS ] Open NS USB device
[ INFO ] Free device list
[ PASS ] Handle kernel driver attach & detach
[ PASS ] Set active configuration to device.
[ PASS ] Claim interface
[ INFO ] TransferModule: Validating split files ...
[ INFO ] TransferModule: Validation complete.
[ INFO ] ============= GoldLeaf v0.8 =============
    VIRT:/ equals files added into the application
    HOME:/ equals /home/fichte
[ INFO ] GL Execution interrupted
[ PASS ] Release interface
[ INFO ] Requested handler close
[ INFO ] Requested context close
[ INFO ]    End chain

Since there appears to be no process to actually hold the RAM, I suspect it's a leak/bug in the driver or whatever handles communication with the NS.

developersu commented 4 years ago

@FichteFoll thank you for detailed report. Please let me know if you have any thoughts. I can build a test-versions of the app, but I still have NO idea what could be wrong in code and even how to reproduce it.

developersu commented 4 years ago

Oh, BTW, please let me know if any previous version has been working for you.

FichteFoll commented 4 years ago

FYI, the versions I used in November were whatever Goldleaf's version was at that point and ns-usbloader 0.9.

I'm just as clueless about the source as you are, but at least (or unfortunately, rather) I can reproduce it. If I were to debug this, could you give me any hints about the components this is likely to occur in? Or a very brief rundown of how data transmission works? From a brief look at the code, it appears my starting point would be nsusbloader.COM.USB.GoldLeaf, or specifically its while loop from line 116. Is there anything specific I need to consider when running/building the code locally?

The annoying part of this issue is that the memory is never freed until I reboot, which makes the testing and debugging quite long.

developersu commented 4 years ago

@FichteFoll Ok, if you're fine with building it, it's perfect!

First of all, set in pom.xml <version>1.3.0</version> <!-- Must be 1.2.0 for macOS lower than Mojave --> instead of 1.2.0. Not critical, but it is what we have in JAR with 'no postfixes'.

And yes, you're correct. In nsusbloader.COM.USB.GoldLeaf you may have to change loop of reading commands to something like this (line 989):

private byte[] readGL(){
        ByteBuffer readBuffer;
        readBuffer = ByteBuffer.allocateDirect(4096);

        IntBuffer readBufTransferred = IntBuffer.allocate(1);

        int result;
            result = LibUsb.bulkTransfer(handlerNS, (byte) 0x81, readBuffer, readBufTransferred, 0); // <- INIFINITE. Blocking function, you won't be able to interrupt the process, but you can check if it's the problem.

            if (result == LibUsb.SUCCESS) {
                    int trans = readBufTransferred.get();
                    byte[] receivedBytes = new byte[trans];
                    readBuffer.get(receivedBytes);
                    return receivedBytes;
             }
             logPrinter.print("GL Data transfer issue [read]\n         Returned: " +
                        UsbErrorCodes.getErrCode(result) +
                        "\n         GL Execution stopped", EMsgType.FAIL);
             return null;
    }

*this is what actually turns for the loop at line 116.

Second part is nsusbloader.COM.USB.UsbConnect Remove this:

if (LibUsb.setAutoDetachKernelDriver(handlerNS, true) == LibUsb.SUCCESS)
            logPrinter.print("Handle kernel driver attach & detach", EMsgType.PASS);
        else
            logPrinter.print("Skip kernel driver attach & detach", EMsgType.INFO);

And also you can remove this:

if ( ! initForRCM){
            // Set configuration (soft reset if needed)
            result = LibUsb.setConfiguration(handlerNS, 1);     // 1 - configuration all we need
            if (result != LibUsb.SUCCESS){
                logPrinter.print("Set active configuration to device\n         Returned: "+UsbErrorCodes.getErrCode(result), EMsgType.FAIL);
                close();
                return;
            }
            logPrinter.print("Set active configuration to device.", EMsgType.PASS);
        }

Soo.. this is what I have for now.

One last thing to be clear 'cuz I'm not sure that I understood you correctly: I assume when you tested v0.9 in November you also have had exactly the same issue. Is it correct?

Thanks!

developersu commented 4 years ago

Oh, and regarding how it works. Briefly:

In the early beginning app prepares connection in UsbConnect (and closes it in there as well in the end). In that class you'll find most of the things for interacting with usb4java (libusb). As a result we get NS handler. And also we defining in each of GoldLeaf/Tinfoil classes our own variants of how we would read/write to console (using handler as a mandatory entity). For example specific buffer sizes etc.

Anyway, in GoldLeaf we have that 'main' loop that you saw in line 116. It just waiting for GoldLeaf to send something. Nothing sent before something comes from NS side (! unlike TF), so that's why you could click 'Send to NS' button once you open GoldLeaf application (GL immediately creates appropriate VID/PID pair of the device that is used for connection initiation). Everything else is just calling the functions. The most interesting in the end: it's how read/write things are going. Everything else is handling files read/write and forming the buffers. Since the issue appears when NOTHING actually happens I guess you can just skip reading a lot of things.. Anyway, it's for your consideration.

FichteFoll commented 4 years ago

One last thing to be clear 'cuz I'm not sure that I understood you correctly: I assume when you tested v0.9 in November you also have had exactly the same issue. Is it correct?

I didn't specifically check whether RAM is overflowing, but I experienced the same symptoms where music playback enters a 1s loop (probably internal buffer) and the entire system becomes unresponsive, so I'm confident it's the same issue. It also occured as soon as I hit the button to upload without any interaction on GL's side.

I can't tell exactly when I'll get around to debug this, but I am certainly interested and hope to do it in the next few weeks rather than months. Thanks for the pointers.

Nonononoki commented 4 years ago

Same problem with Fedora 31 with 8GB of RAM. Weirdly enough, the Witcher 3 3.6 update (NSP) installs fine while Pokemon Shield NSZ base game causes a system freeze even though they are roughly the same size. openjdk-8

carcuevas commented 4 years ago

Hi guys same, problem with Debian 10 kernel Linux 5.4.21-20

 java -version
openjdk version "11.0.6" 2020-01-14
OpenJDK Runtime Environment (build 11.0.6+10-post-Debian-1)
OpenJDK 64-Bit Server VM (build 11.0.6+10-post-Debian-1, mixed mode, sharing)

20GB in less than 30 seconds

FichteFoll commented 4 years ago

Just reporting in that I have since found a working solution in GoldtreePy (or more precisely a fork of it). It's also in Python, which I know a lot better than Java and thus would have an easier time fixing should it break in the future. (It also comes without a UI and is only ~480 lines of code.)

developersu commented 3 years ago

Maybe it's was an issue in OpenJFX. Please check latest version where dependencies have been updated. Also it you have the same issue don't hesitate to leave a comment. There are no activity on this thread for a while so maybe it's fixed already.