new UsbFileWrapper in javafs cost too much time

[1162064779]

2023-11-23 14:28:02.334 10589-11529 xptest:UsbFileWrapper xyz.skybox.player.ovr D listFiles5:3 2023-11-23 14:28:02.334 10589-11529 xptest:UsbFileWrapper xyz.skybox.player.ovr D listFiles6:3 2023-11-23 14:28:29.115 10589-11529 xptest:UsbFileWrapper xyz.skybox.player.ovr D listFiles5:4 2023-11-23 14:28:29.115 10589-11529 xptest:UsbFileWrapper xyz.skybox.player.ovr D listFiles6:4 2023-11-23 14:30:27.459 10589-11529 xptest:UsbFileWrapper xyz.skybox.player.ovr D listFiles7

this is my code: @Override public UsbFile[] listFiles() throws IOException { Log.d(TAG, "listFiles1"); if(dir == null) { throw new UnsupportedOperationException("This is a file!"); } Log.d(TAG, "listFiles2"); List list = new ArrayList<>(); Log.d(TAG, "listFiles3"); Iterator<? extends FSEntry> iterator = dir.iterator(); Log.d(TAG, "listFiles4"); int i = 1; while(iterator.hasNext()) { Log.d(TAG, "listFiles5:"+i); FSEntry entry = iterator.next(); Log.d(TAG, "listFiles6:"+i); list.add(new UsbFileWrapper(entry)); i++; } Log.d(TAG, "listFiles7"); UsbFile[] array = new UsbFile[list.size()]; array = list.toArray(array); Log.d(TAG, "listFiles8"); return array; }

When handling a USB drive with the exFAT file system, if a folder contains 1,000 files, this line of code takes about 30 seconds to execute. For over 2,000 files, it takes about 2 minutes: list.add(new UsbFileWrapper(entry)); The listFiles method is frequently used in the project, and such long execution times obviously cause serious issues. It would be beneficial to find ways to optimize this time consumption, or to add some form of caching to improve performance.

[1162064779]

@magnusja

[magnusja]

Yeah the javafs is highly experimental. It wraps around the file system implementations of jnode. And they do not play super well with the buffers in libaums. This can be improved, question is is it better to implement sth like exFAT specifically for libaums. JNODE is under LGPL, which is also unfortunate. This probably takes a couple of weeks to implement. But I dont have any time.

[1162064779]

Alright, thanks for getting back to me. I think the main problem is some unnecessary clashes between jnode and libaums. I'm gonna try and tackle it myself first, like saving the previously created usbfilewrapper to avoid creating it multiple times. In my tests, whipping up a usbfile on exfat takes like 60ms, which is quite a bit. I'll try my best to cut down on that. Really appreciate your time and response again. Hope you have an awesome day! @magnusja

[1162064779]

Ultimately, I discovered that in the exFAT file system, the parseFile function mainly consumes time in the hashName function, specifically during the process of converting the string to uppercase (hashName(name) != nameHash). This part seems particularly time-consuming, and I believe there's significant room for optimization here. However, I'm also wondering about the necessity of this filename validation step. It appears redundant, especially since we've already performed a similar comparison with referenceChecksum != actualChecksum earlier. I hope these insights prove helpful for your project. Thank you! @magnusja