Support loading entire directory contents rather than truncating at 250

[Mechite]

Is your feature request related to a problem? Please describe. Back on the 0.7.x panel, we were able to see full file structure through the web file manager. Now, on 1.x.x, there is a 250 file/directory limit on viewing.

Describe the solution you'd like I would like there to be a way of disabling this limit in the panel configuration, or upping it, as there has never been an issue for my hosting provider having thousands of files queried, so why limit it to 250 when the panel can easily handle viewing more.

Describe alternatives you've considered I've also considered possibly having a 50 file limit, with pages to view that directory.

Additional context

[DaneEveritt]

Just want to clarify that this is not a trivial change, and there is a very real reason why it does that. Your browser is going to start lagging drastically and be struggling if you render that many DOM nodes at once. The proper fix for this is to setup a virtualized listing, but that is not a simple little change to make.

I will not be adding the ability to configure the limits, that is just asking for people to crash their browser.

In the mean time you can just use a SFTP client to browse through these large directories.

[Mechite]

Alright, but isn't 250 a bit low for some applications? If this has been tested well I'll leave that to you.

Thanks for the fast response too! Would be good to close this issue as you'll be getting to improving it far into the future of 1.x, and SFTP works wonders for most people anyway.

[DaneEveritt]

It can stay open since people can search and find it to see what the progress on it is.

[TheFlash787]

To add onto this, I notice that when files are omitted from the file manager due to the folder size, the omitted files don’t seem to be taken from the end of the file stream but instead the beginning (after any folders that are there)

So instead of having shown:

Folders Files A-X (X-Z being omitted due to cap)

We instead receive:

Folders Files C-Z (with the beginning of the files omitted, not the end).

Is this something that was intended for a certain reason or more just an unchangeable side-effect of using this limit system?

-

On a more related note for the issue, I found that whilst using 0.7 and needing to open a folder of 1000+ files (such as a region folder let’s say, or playerdata), the panel would only take less than five seconds to generate the results and all would be displayed.

Is that the wait which you were trying to avoid or is it due to new internals that this wasn’t implemented/hasn’t been implemented yet (or is that in fact the virtualisation mentioned)

Thanks!

[Fireztonez]

I really hope there will be a way to change that limit at least in the .env file soon, because this is pretty anoying to not be able to see all the files on the folder, when you need to. Sure we can open by the SFTP, but a web interface like you have is very practical :D.

I can understand with a really high number of files in a folder, the web interface will start lagging a lot. I think, adding a button to see the next file of the list can probably fix that lags issue, and give the possibility to see all the files on that folder.

[tinyoverflow]

I absolutely do understand why the limit is there, especially for clients using slower computers. But I definitely agree with the previous posters that this limit is way to low. In my opinion there are multiple ways to fix this:

• Keep the listing in memory and only display them on scroll, though this has a catch: It could consume a lot of memory and CPU when the directory is huge. Therefore the issue would've just moved to another location
• A pagination:
  • By autoloading when scrolling OR
  • by having a real pagination. While this works very well, I'm not a big fan of paginating file listings

Not a real fix for this issue, but atleast a worthy addition which can make this semi-obsolete:

• A search which filters through all files. This can be done in frontend only.

[Mechite]

I absolutely do understand why the limit is there, especially for clients using slower computers. But I definitely agree with the previous posters that this limit is way to low. In my opinion there are multiple ways to fix this:

• Keep the listing in memory and only display them on scroll, though this has a catch: It could consume a lot of memory and CPU when the directory is huge. Therefore the issue would've just moved to another location

• A pagination:

  • By autoloading when scrolling OR
  • by having a real pagination. While this works very well, I'm not a big fan of paginating file listings

Not a real fix for this issue, but atleast a worthy addition which can make this semi-obsolete:

• A search which filters through all files. This can be done in frontend only.

I honestly feel like there should literally just be a hard limit on how many files are loaded, and then (as mentioned before) use react-virtualized to load new elements on scroll, and unload elements which are out of frame. Displaying an element is the issue here, not necessarily memory since it really is a trivial fix to truncate the backend response to an acceptable amount (definitely not 250).

Pagination is cool but honestly very tedious to use as you say. The buttons for performing actions inside of the directory listing should be locked into place when scrolling too, as this will make it much easier to use a listing that unloads elements.

Even then, I do still believe we can increase this limit. I really don't think people using a server hosting panel have the worst computers, and even if I'm wrong, just being able to have a filter mode to change the scope of files shown and amount is enough for people to find this usable.

500 files may seem like not a large increase but that is double the limit, and absolutely plenty. Having a filter to scope between file names or types would allow us to sort the listing and view every file in the directory. Just these 2 things make it usable, and if a full view of the files is desired, SFTP is viable.

[LindaJuffermans]

Alternative solution: A textbox where we can enter a filename filter so it would still only retrieve a limited amount of files.

[anondev-sudo]

It would be nice if there would be pagination buttons (arrows) and like 50 files displayed at once, when you click on the arrow next, next files could be fetched (not all at once to not make it load longer).

[e2489]

It would be nice if there would be pagination buttons (arrows) and like 50 files displayed at once, when you click on the arrow next, next files could be fetched (not all at once to not make it load longer).

Alternative solution: A textbox where we can enter a filename filter so it would still only retrieve a limited amount of files.

These together would be a great solution.

For the time being, it would be nice if we could at least set the file limit ourselves. TCAdmin can load tens of thousands of files just fine. It's slow, but it works. It's much more convenient to have everything in one place, especially on mobile, and especially when this needs to be communicated to admins who are not entirely tech savvy.

[Mechite]

I completely forgot I even made this issue! Yes, no, the original solution proposed by @DaneEveritt is the best one and the one that nobody has the resources or desire to do yet. A virtual scrolling implementation should have been done from the beginning honestly, and pagination is not an alternative when we are dealing with a file system, it is just horrible UX, call me opinionated. I understand people who get frustrated by this kind of thing in, e.g. forums, because it makes total sense why they would prefer an old-school paginated view rather than "auto pagination" i.e. virtual scrolling.

SFTP is a valid solution to this problem, but the problem is that it makes Pterodactyl even less useful, I hardly find anywhere I even need to use Pterodactyl since I usually use FreeBSD jails for all of my applications and just manage things myself, and I don't really want to pollute my node with Laravel and whatnot when the system I am usually running does a better job natively managing containerisation and none of the other management features appeal to me with Pterodactyl since I don't run Minecraft servers but run more distributed, auto scaling systems instead.

Basically, what I am trying to say is that Pterodactyl can't try and be what it is not, and we need to have the most trivial features built into the panel so that the actual target audiences of people running game servers and hosts trying to appeal to those people can actually have something they can appreciate; that is why currently I'd like to see even more of this sort of functionality, or this just feels like a research project otherwise.

[e2489]

pagination is not an alternative when we are dealing with a file system, it is just horrible UX, call me opinionated.

The functionality itself is the important part. Loading a few files at once to not overwhelm the browser, and then allowing a search/filter to do the heavy lifting would solve the current issue. From a UX standpoint, they could implement both once this is in place. When you're at the point of needing multiple pages, it seems less likely you're going to use this method to find files anyway, but maybe that's just me.

SFTP is a valid solution to this problem, but the problem is that it makes Pterodactyl even less useful

Exactly. The only reason I'm using something like Pterodactyl is for the sake of my admins, who are less technically inclined. Otherwise, I'd just use LGSM. When they can't do the basic things they need to do with it, what's the point?

[Mechite]

When you're at the point of needing multiple pages, it seems less likely you're going to use this method to find files anyway, but maybe that's just me.

Virtual scrolling / auto pagination is almost the most common way to deal with long lists like this; you see it absolutely everywhere and it is the way users are used to finding things in modern apps now - it would not feel natural to have pages in a GitHub issue for example. Issues like these need to be the main focus because the whole purpose of this panel is to put power in the hands of idiots.

[LindaJuffermans]

If you have a folder with over 1000 files (e.g. a Minecraft region folder), you'll be scrolling for a long time to get to the file you need. You can't use your browser's find because not the entire list has been loaded, having a search option (like it exists for GitHub issues for example) would help you find r.-6.-14.mca a lot quicker.

[e2489]

Maybe something like this could be implemented: https://github.com/mickael-kerjean/filestash

Works with SFTP, has auto scroll & search. I tested it with a log folder containing over 75,000 files and everything loads up just fine within seconds.

[2001zhaozhao]

I would like to see an option to adjust the limit up to about 500, mostly due to Minecraft modpack servers. A lot of modpacks are in the range of 250-400 mods. Very few have over 500.

[Innesity]

Hi. SFTP cannot be used when running behind cloudflare's proxy.