dedupe command not working properly - #RESOLVED

[roshanconnor123]

Hi I had a folder with 2 duplicates When I used the dedupe command it asked for 3 options Delete Confirm Deletion No When I presses Enter after typing Yes..it showed that Deleted succesfully and 0 Folders deleted Thank you

[iwestlin]

hi, can you take a screenshot?

./dedupe will move 2 things into trash

• files with same md5 hash in same folder(will keep one)
• empty folders with same name in same folder(will both trash)

[roshanconnor123]

Here is the screenshot Thank you

[iwestlin]

I see, you are using the colab version...

colab version is very limited, some args in command line is not supported there.

for example, -u, which means update from online

in your screenshot, it says 'found 0 dupe files and 0 dupe folders', may be that's because it's using cached info from local database.

if so, then you'll have to run command like this: ./dedupe folderID -S -u

by the way, your service accounts must be content manager(or manager) to trash files.

[roshanconnor123]

Thank you very much I will try to update the command with -u and try again I will report back

[roshanconnor123]

Hi I was trying to dedupe two duplicate folders..thats why it was not working I tried to dedupe 2 files and it worked...Thank You But I found one issue-How can i choose No instead of Yes When I used down arrow its not changing to No And is there any way to delete the file from Trash too Thank you for all the fast response

[iwestlin]

it seems that colab did not support arrow choose, so you will have to change this function: https://github.com/iwestlin/gd-utils/blob/d9d6ae5efd945278d9af77d415fdbb0a334dd181/src/gd.js#L727 to

async function confirm_dedupe ({ file_number, folder_number }) {
  const answer = await prompts({
    type: 'text',
    name: 'value',
    message: `found ${file_number} dupe files, dupe empty ${folder_number} folders, confirm trash?(yes/no)`,
    validate: value => ['yes', 'no'].includes(value) ? true : 'must enter yes or no'
  })
  return answer.value
}

after that, you may enter yes to confirm trash, and no to cancel.

if you want to delete file instead of move it into trash, you can change this line: https://github.com/iwestlin/gd-utils/blob/d9d6ae5efd945278d9af77d415fdbb0a334dd181/src/gd.js#L804 to

await limit(() => rm_file({ fid: v.id, service_account }))

and make sure your service accounts be the managers of the team-drive.

gd-utils did not support delete files from trash yet, but you can use gclone:

gclone delete DriveName:{team-drive-id} --drive-trashed-only --drive-use-trash=false --verbose=2 --fast-list

replace DriveName to the name in your gclone config file, and team-drive-id of course, then it should work

[roshanconnor]

Thank you very much for you explanation I was making an english friendly colab notebook based on your original notebook I will be happy to share it here once it is done Cheers

[roshanconnor123]

@iwestlin so i forked this repo and made an english version for the same (just used google translate) I changed readme too to english Sharing here as i Promised -https://github.com/roshanconnor123/gd-utils Thank you for such an awesome tool

[iwestlin]

@iwestlin so i forked this repo and made an english version for the same (just used google translate) I changed readme too to english Sharing here as i Promised -https://github.com/roshanconnor123/gd-utils Thank you for such an awesome tool

thank you for your contribution! 😄 I added you repo in readme: https://github.com/iwestlin/gd-utils/commit/e6ebef907805299b35a5fb528ce48523978ed369

[roshanconnor123]

Hi,I am with a new issue

it seems that colab did not support arrow choose, so you will have to change this function: https://github.com/iwestlin/gd-utils/blob/d9d6ae5efd945278d9af77d415fdbb0a334dd181/src/gd.js#L727

to

async function confirm_dedupe ({ file_number, folder_number }) {
  const answer = await prompts({
    type: 'text',
    name: 'value',
    message: `found ${file_number} dupe files, dupe empty ${folder_number} folders, confirm trash?(yes/no)`,
    validate: value => ['yes', 'no'].includes(value) ? true : 'must enter yes or no'
  })
  return answer.value
}

after that, you may enter yes to confirm trash, and no to cancel.

if you want to delete file instead of move it into trash, you can change this line: https://github.com/iwestlin/gd-utils/blob/d9d6ae5efd945278d9af77d415fdbb0a334dd181/src/gd.js#L804

to

await limit(() => rm_file({ fid: v.id, service_account }))

and make sure your service accounts be the managers of the team-drive.

gd-utils did not support delete files from trash yet, but you can use gclone:

gclone delete DriveName:{team-drive-id} --drive-trashed-only --drive-use-trash=false --verbose=2 --fast-list

replace DriveName to the name in your gclone config file, and team-drive-id of course, then it should work

There was one more Scenario Like this In Copy Mode When i was trying to run a copy which was interrupted before it asked me to choose any oone of these resume? restart? exit? Since I am using colab it was difficult for me to choose an option So I edited the code based on your previous reply Like this

async function user_choose () {
  const answer = await prompts({
    type: 'text',
    name: 'value',
    message: 'Do you want to resume your transfer?\n1)Resume\n2)restart\n3)exit',
    validate: value => ['1', '2', '3'].includes(value) ? true : 'must enter 1 or 2 or 3'
  })
  return answer.value
}

When I Ran copy command after Updating the Code (as above).I got 3 options to type The Problem is that when I typed 1 its becoming 178 if 2 then278if3then378` I am attaching a screenshot of both dedupe and Copy commands below

[iwestlin]

I think that's because prompts does not fully support colab...

I just took a test on prompt, it seems ok:

const prompt = require('prompt') // must run `npm install prompt` first

function confirm_dedupe ({ file_number, folder_number }) {
  const property = {
    name: 'answer',
    message: `found ${file_number} dupe files, dupe empty ${folder_number} folders, confirm trash?(yes/no)`,
    validator: word => ['yes', 'no'].includes(word),
    warning: 'Must respond yes or no',
    default: 'yes'
  }
  return new Promise((resolve, reject) => {
    prompt.get(property, function (err, result) {
      if (err) return reject(err)
      resolve(result.answer)
    })
  })
}

[roshanconnor123]

I think that's because prompts does not fully support colab...

I just took a test on prompt, it seems ok:

const prompt = require('prompt') // must run `npm install prompt` first

function confirm_dedupe ({ file_number, folder_number }) {
  const property = {
    name: 'answer',
    message: `found ${file_number} dupe files, dupe empty ${folder_number} folders, confirm trash?(yes/no)`,
    default: 'yes'
  }
  return new Promise((resolve, reject) => {
    prompt.get(property, function (err, result) {
      if (err) return reject(err)
      resolve(result.answer)
    })
  })
}

Thank you for the response Can we do something like this for the Sync function too (resume or restart)

Edit: Nvm I saw the prompts page..gonna Check it out now Cheers

[iwestlin]

Can we do something like this for the Sync function too (resume or restart)

sure:

async function user_choose () {
  const property = {
    name: 'answer',
    message: 'found same task before, what to do?\n(1) continue\n(2) restart\n(3) exit\n',
    validator: word => [1, 2, 3].includes(Number(word)),
    warning: 'Must enter 1/2/3',
    default: 1
  }
  const choices = ['', 'continue', 'restart', 'exit']
  return new Promise((resolve, reject) => {
    prompt.get(property, function (err, result) {
      if (err) return reject(err)
      resolve(choices[result.answer])
    })
  })
}

[roshanconnor123]

YOU ARE AN ABSOLUTE LEGEND !! Awesome

[roshanconnor123]

@iwestlin is there any way for me to contact you in Telegram? Thanks

[iwestlin]

Sorry, I spent too much time on telegram answering questions last few weeks, so I decided simply not use it... Is there anything I can do for you?

[roshanconnor123]

@iwestlin Here is the Telegram Bot Version in English https://github.com/roshanconnor123/Gdutils_Tgbot Thank you

[iwestlin]

@iwestlin Here is the Telegram Bot Version in English https://github.com/roshanconnor123/Gdutils_Tgbot Thank you

Thank you for your contribution. 👍 I added it to readme: https://github.com/iwestlin/gd-utils/commit/0b332504b8b743d1185f0c9b412a66bfc011229d