I recently processed the entire CC and just wanted to point out that this line will cause your script to crash and, in fact, remove all of the downloaded CC data from thetmp_download_dir_name without successfully moving it to download_dir. It's a Linux thing where moving thousands of files with mv can cause an error.
I recently processed the entire CC and just wanted to point out that this line will cause your script to crash and, in fact, remove all of the downloaded CC data from the
tmp_download_dir_namewithout successfully moving it todownload_dir. It's a Linux thing where moving thousands of files withmvcan cause an error.Better solution is mentioned here.
Hope this saves someone some time in future! ✌🏻