SpEcHiDe / UploadGram

https://pypi.org/project/uploadgram/
GNU Affero General Public License v3.0
102 stars 24 forks source link

Found some bugs #1

Open tissole opened 3 years ago

tissole commented 3 years ago

Thank you for developing this script. I have tested on Windows 10 and I want to report some bugs. After the upload videos have some things embedded in them

1 2

In other cases, this remains in chat

3

After the upload is done, in the folder remains some screenshots, they are not cleared.

If videos are numbered 1,2,3..10,11 etc., the script uploads them in this order: 1,10,2,3...Same with the folders, first uploads videos from folder 1 but then uploads videos from folder 10 instead of folder 2.

I also noticed a delay between uploads, can you make an option that users choose to delay or not?

tissole commented 3 years ago

Thanks for the update! I installed the new version and found that the issue with autogenerated files still persists.

9EyUy4I

I also tried to upload a folder with some videos and only the first video is uploaded, and then the script stops with this error https://nekobin.com/tadasacuca

The image from the last line is the auto-generated thumbnail.

I run some tests with a big folder that contains almost 21.000 small pictures, under 1 MB each, about 3 GB in total. I used a user account and set the delay to zero (using a bot account is more limited and the delay has to be set high limiting the number of files uploaded, the user account has fewer limitations). I found these issues:

  1. The script stops with a flood message after uploading about 1900-2000 files - this is the TG limit, same as forwarding. I think 1800 is a safe limit to prevent flood messages. This is beneficial when uploading small files. Is hard to calculate the right delay needed to upload ~1800 files/h because of variable file length, upload speed, etc. Please add an option so that users can choose the number of files uploaded per hour.

  2. If the script stops it does not restart automatically, it has to be restarted manually by the user. Is more comfortable if the script could restart automatically after the waiting period.

  3. After the restart the script uploads files already uploaded, it does not remember what has already been uploaded. This can be fixed by indexing all the files before uploading and keeping a record of uploaded files so the script knows where to restart from next time.

  4. Now the script uploads photos as documents, please make an option so that users could choose between documents or photos. It is possible to upload photos as photos without compression?

tissole commented 2 years ago
  1. The issues with screenshots and progress bar not being cleared after the upload is finished, are caused by a setting in the lines 205 and 257 from file upload.py. The actual setting quote=True must be changed to quote=False.

  2. The actual sorting (10 before 2, filenames with the upper case being uploaded before filenames with lowercase) can be resolved with the library called natsort and the function os_sorted. I installed it and then I modified the file upload.py. First, at the beginning of the file, I added from natsort import os_sorted and then I modified line 45 from dir_contents.sort() to dir_contents = os_sorted(dir_contents). Now the files are sorted like in Windows Explorer. This type of sorting works across all operating systems.

  3. The script can be made to auto-restart after the FloodWait by writing a few lines of codes from Pyrogram

import time
from pyrogram.errors import FloodWait

try:
    ...  # Your code
except FloodWait as e:
    time.sleep(e.x)  # Wait "x" seconds before continuing

The first 2 lines must be written at the beginning of upload.py and the relevant code must be inserted between try and except. The relevant code is that regarding uploading directory content, individual files, video files, and audio files.

For example

        if os.path.isdir(current_name):
            await upload_dir_contents(
                current_name,
                delete_on_success,
                thumbnail_file,
                force_document,
                custom_caption,
                bot_sent_message,
                console_progress,
            )

must be changed with

        if os.path.isdir(current_name):
          try:
            await upload_dir_contents(
                current_name,
                delete_on_success,
                thumbnail_file,
                force_document,
                custom_caption,
                bot_sent_message,
                console_progress,
            )
          except FloodWait as e:
              time.sleep(e.x)

With this change, the script starts automatically after the FloodWait.

  1. Now only one video is uploaded from a folder with many videos. This is caused by a setting in line 195 from upload.py Currently is metadata = extractMetadata(createParser(thumb_nail_img)) and if I change it to metadata = extractMetadata(createParser(file_path)) all videos from a folder are uploaded. This resolves the #3 and #6 too.