search

My-Little-Forum / mylittleforum

A simple PHP and MySQL based internet forum that displays the messages in classical threaded view (tree structure)
GNU General Public License v3.0
121 stars 47 forks source link

Critical problem with MyLittleForum and FTP server #641

Open danielb987 opened 1 year ago

danielb987 commented 1 year ago

I have just discovered a critical problem with MyLittleForum and some FTP servers.

Some FTP servers limit the number of records in the folder listing to 10000 records. With the folders . and .., that means that the FTP client will list 9998 files. The critical problem here is that it's impossible for the FTP client to download files that it doesn't see. So I cannot take backups on these files. And our forum is on a web provider so we cannot change the FTP server setting.

On our MyLittleForum, we had 6000 images uploaded for the year 2020 alone. For year 2021 we had 3000 images uploaded. And we have 2000 files uploaded for this year so we will probably get 3000 images uploaded before this year is over.

So on our forum, the limit of 10000 files is the number of images we get uploaded for two - three years.

The simple solution to this problem is to have subfolders for the images. For example the folder 202208 which has the images for August 2022.

auge8472 commented 1 year ago

To be honest, I don't understand your problem description.

Is the list of file entries limited to 10,000 entries in your FTP-client even there are more than 10,000 files in the directory or does your hosting service limit you to upload a maximum of 10,000 files into a single directory and the upload of the 10,001st file will fail?

danielb987 commented 1 year ago

The limit is on the FTP Server. When the FTP client connects to the server, the client request a list of files in a particular folder. The FTP server limits this list to 10 000 files.

There is no limit on the number of files in a folder. But the FTP Server will not show more than 10 000 files in the folder. This is configurable in the FTP server and some web hosting companies have the FTP server configured this way.

Note that the server probably has a standard Linux installation which runs a HTTPS server (for example Apache), a FTP server, and other software. The limit is not in Linux itself, nor on the HTTPS server, but on the FTP server.

auge8472 commented 1 year ago

So the solution was 1. to change the server settings or 2. to prevent the limit by splitting up the image uploads to several sub directories.

For 1. the situation is clear. It would be the better solution because it solves the issue at its source and it would make a script side "solution" obsolete (I will later explain the quotes around solution). The problem is, that not everybody is able to change the setting (as you stated yourself).

For 2. I want to ask a few questions (not to you but to the universe). When we would follow your proposal to split the uploads on a monthly base your personal problem would be solved. But what would happen in the hypothetical case of more that 9998 uploads per month or with another, smaller limit on the FTP-server (say 1000 file entries per directory) with more than 998 uploads per month? These are the questions, that make me enclosing the word solution with quotes. A solution on our side (the side of the script) is by definition incomplete because we are not able to support all possible (edge) cases.

That doesn't mean, that we shouldn't take action. The issue is real and my hypothetical case of the very low limit is not very likely.

auge8472 commented 1 year ago

If we follow your proposal we have the following tasks.

In the case, that I forgot tasks, please feel free to supplement the list with additional entries.

[^1]: at that point we would have the chance to enhance the upload management function by the data of the use of a specific image

auge8472 commented 1 year ago

Found two forgotten (similar) tasks: