maple3142 / aria2c-ariang

310 stars 481 forks source link

double upload issue #19

Open mohitjoshi155 opened 4 years ago

mohitjoshi155 commented 4 years ago

image theres a bug where sometimes the torrent is uploaded twice via rclone...please fix it if u can...tysm

maple3142 commented 4 years ago

I don't have this problem. I think it is actually two files, because you can't have two files with same name coexist under Windows.

mohitjoshi155 commented 4 years ago

the torrent has one file only... i posted your repo as a guide in one of the forums.... there also ppl are telling me that they are getting double files uploaded sometimes too

krishne35 commented 4 years ago

the torrent has one file only... i posted your repo as a guide in one of the forums.... there also ppl are telling me that they are getting double files uploaded sometimes too

Ahhh heroku gonna ban this repo soon then :p Haven't faced this issue myself tho

mohitjoshi155 commented 4 years ago

the torrent has one file only... i posted your repo as a guide in one of the forums.... there also ppl are telling me that they are getting double files uploaded sometimes too

Ahhh heroku gonna ban this repo soon then :p Haven't faced this issue myself tho

Its a closed invite only based forum ...you are right though one guy was saying heroku took down the app as soon as he deployed... this repo comes in top 10 deployed heroku buttons now in heroku elements

himate143 commented 4 years ago

image theres a bug where sometimes the torrent is uploaded twice via rclone...please fix it if u can...tysm

same for me double files is uploaded when i keep other torrent and it finishes the old one also re uploads and also if i kept 2 torrents named 1,2 and 1 is not completed 2 is completed but it is uploading both 1(uncompleted) and 2..pls fix..thank you

krishne35 commented 4 years ago

@maple3142 I guess since the download completion triggers the on-complete.sh it's restarting the rclone copy command I personally use rclone move in my case so I haven't faced this issue

maple3142 commented 4 years ago

I can't fix a bug that I haven't faced. I am using rclone copy, and everything works just fine.

If you have issue, maybe you could adjust your script and push it to your heroku account.

masterrulzz commented 4 years ago

image theres a bug where sometimes the torrent is uploaded twice via rclone...please fix it if u can...tysm

same for me double files is uploaded when i keep other torrent and it finishes the old one also re uploads and also if i kept 2 torrents named 1,2 and 1 is not completed 2 is completed but it is uploading both 1(uncompleted) and 2..pls fix..thank you

same issue..will try rclone copy as suggested

iam143 commented 4 years ago

image theres a bug where sometimes the torrent is uploaded twice via rclone...please fix it if u can...tysm

im also facing this issue

krishne35 commented 4 years ago

here how you could replicate

  1. add two torrent (one small and one big) when the small torrent completes it will start uploading once its done it will upload the partially downloaded 2nd torrent too when the second torrent completes it download it will dupllicate the files
krishne35 commented 4 years ago

this issue is not present with this implementation https://gitlab.com/mega036/aria2cloud-colab/-/blob/master/.aria2/autoupload.sh

maple3142 commented 4 years ago

here how you could replicate

  1. add two torrent (one small and one big) when the small torrent completes it will start uploading once its done it will upload the partially downloaded 2nd torrent too when the second torrent completes it download it will dupllicate the files

Still can't reproduce I will just ignore this until someone send a PR to fix this

mohitjoshi155 commented 4 years ago

here how you could replicate

  1. add two torrent (one small and one big) when the small torrent completes it will start uploading once its done it will upload the partially downloaded 2nd torrent too when the second torrent completes it download it will dupllicate the files

Still can't reproduce I will just ignore this until someone send a PR to fix this

wats pr?u want logs?

maple3142 commented 4 years ago

@mohitjoshi155 Pull Request Someone find where does this bug (exists?) come from, and submit a fix.

mohitjoshi155 commented 4 years ago

I also experience this issue before where it does double upload. Now I used different autoupload.sh, since then I never faced the same issue again.

What did u chg in autoupload.sh...if u hv solved it u can generate pull req and contribute so other ppl can use it

himate143 commented 4 years ago

I also experience this issue before where it does double upload. Now I used different autoupload.sh, since then I never faced the same issue again.

please let us know what you changed so it helps others

N30Z3N commented 4 years ago

It happens because rclone searches directory of cloud for the files that are given as upload task if these files are moved or deleted the rclone will restart upload as if it was never uploaded I'm not a coder so this is what I did after completing upload to the cloud shut down aria2 this helped me

N30Z3N commented 4 years ago

I don't think there can be any solution beacuse this depends on where files are stored since rclone clone is given specific directory to upload to it can't search other directories e.g. / and /AA if files are uploaded in root it won't be able to search in /AA if files are moved or deleted

maple3142 commented 4 years ago

@bnsave100 I don't really understand what you are saying. Do you mean that if user don't try to move or delete files in their drive, then this problem won't happen?

krishne35 commented 4 years ago

@maple3142 using this scrypt solves double upload issue https://gitlab.com/mega036/aria2cloud-colab/-/blob/master/.aria2/autoupload.sh sorry dono how to do PR

maple3142 commented 4 years ago

@krishne35 I won't accept a PR by directly replacing the file. It has to be someone point out what exactly happens, and fix accordingly.

N30Z3N commented 4 years ago

@maple3142 yes

N30Z3N commented 4 years ago

Atleast on heroku

maple3142 commented 4 years ago

So, that is users' problem. It won't happen if user don't try to move or delete them while other file is downloading.

mohitjoshi155 commented 4 years ago

@maple3142 using this scrypt solves double upload issue https://gitlab.com/mega036/aria2cloud-colab/-/blob/master/.aria2/autoupload.sh sorry dono how to do PR

it is for colab repo how can u use it here?is it compatible?how did u do it?

MrRobotGOD commented 4 years ago

The Problem is Happening Due to Improper Filepath or FolderPath passed to rclone on-complete.sh filePath=$3 relativePath=${filepath#./downloads/} topPath=./downloads/${relativePath%%/*}

i fix that problem in my fork https://github.com/MrRobotGOD/heroku-aria2c all can modify the rclone script according to requirements

StreakVilly commented 4 years ago

The Problem is Happening Due to Improper Filepath or FolderPath passed to rclone on-complete.sh filePath=$3 relativePath=${filepath#./downloads/} topPath=./downloads/${relativePath%%/*}

i fix that problem in my fork https://github.com/MrRobotGOD/heroku-aria2c all can modify the rclone script according to requirements

hey, thanks for contributing! I tried your solution for rclone doesnt seem to trigger. Do I have to encode the conf after replacing line breaks, or encode it as it is?

MrRobotGOD commented 4 years ago

The Problem is Happening Due to Improper Filepath or FolderPath passed to rclone on-complete.sh filePath=$3 relativePath=${filepath#./downloads/} topPath=./downloads/${relativePath%%/*} i fix that problem in my fork https://github.com/MrRobotGOD/heroku-aria2c all can modify the rclone script according to requirements

hey, thanks for contributing! I tried your solution for rclone doesnt seem to trigger. Do I have to encode the conf after replacing line breaks, or encode it as it is?

encode as it is

you should have read my README.md example my config is

[test] type = drive scope = drive root_folder_id = XXXXX token = XXX

remove first line [test] then encode as it in base64 no new line break

gautamajay52 commented 4 years ago

The Problem is Happening Due to Improper Filepath or FolderPath passed to rclone on-complete.sh filePath=$3 relativePath=${filepath#./downloads/} topPath=./downloads/${relativePath%%/*} i fix that problem in my fork https://github.com/MrRobotGOD/heroku-aria2c all can modify the rclone script according to requirements

hey, thanks for contributing! I tried your solution for rclone doesnt seem to trigger. Do I have to encode the conf after replacing line breaks, or encode it as it is?

encode as it is

you should have read my README.md example my config is

[test] type = drive scope = drive root_folder_id = XXXXX token = XXX

remove first line [test] then encode as it in base64 no new line break

type = drive\n scope = drive\n root_folder_id = XXXXX\n token = XXX

Ur RCLONE_CONFIG should be like this. Means just add \n at last of each line, nothing else.

yx9-mio commented 4 years ago

RCLONE_DESTINATION What to write

gautamajay52 commented 4 years ago

RCLONE_DESTINATION What to write

A folder name which should be in the same account of ur rclone config. First create a folder in ur gdrive account and put there.

yx9-mio commented 4 years ago

drive:mio add /mio ?

RCLONE_DESTINATION What to write

A folder name which should be in the same account of ur rclone config. First create a folder in ur gdrive account and put there.

drive:mio add /mio Is that right

gautamajay52 commented 4 years ago

drive:mio add /mio ?

RCLONE_DESTINATION What to write

A folder name which should be in the same account of ur rclone config. First create a folder in ur gdrive account and put there.

drive:mio add /mio Is that right

If ur folder name is mio then just put /mio

taylorann11 commented 4 years ago

I'm also having the same issue. and its not just double, sometimes it is uploading the file 3-4 times

maple3142 commented 4 years ago

Please don't post "me too", "I'm having this issue" too, it is what reactions for.

venom3525 commented 4 years ago

The Problem is Happening Due to Improper Filepath or FolderPath passed to rclone on-complete.sh filePath=$3 relativePath=${filepath#./downloads/} topPath=./downloads/${relativePath%%/*}

i fix that problem in my fork https://github.com/MrRobotGOD/heroku-aria2c all can modify the rclone script according to requirements

Tried your fork. it solves the problem with multiple uploads but it seems to skip smaller files

Edit: my bad, u had added the extensions to the list of exclusions. everything is fine