search

manojkarthick / reddsaver

CLI tool to download saved and upvoted media from Reddit
Apache License 2.0
144 stars 8 forks source link

Error: IoError(Os { code: 18, kind: Other, message: "Invalid cross-device link" }) #33

Closed wavy-baby closed 2 years ago

wavy-baby commented 2 years ago

Screenshot_2022-01-04_04-46-27

manojkarthick commented 2 years ago

Hi, Can you share your OS info? Also - Can you run it with RUST_LOG=debug reddsaver -d ~/reddit/image -e user.env and paste the output here (removing any sensitive information)? That would help with debugging the issue.

wavy-baby commented 2 years ago

OS is Arch Linux

Edit: This is only the last part, not the full log

[2022-01-04T04:13:26Z DEBUG hyper::proto::h1::io] read 16384 bytes [2022-01-04T04:13:26Z DEBUG hyper::proto::h1::io] read 16384 bytes [2022-01-04T04:13:26Z DEBUG hyper::proto::h1::io] read 6272 bytes [2022-01-04T04:13:26Z DEBUG hyper::proto::h1::io] read 10882 bytes [2022-01-04T04:13:26Z DEBUG hyper::proto::h1::conn] incoming body completed [2022-01-04T04:13:26Z DEBUG hyper::proto::h1::io] read 16384 bytes [2022-01-04T04:13:26Z DEBUG reddsaver::download] Bytes length of the data: 1320322 [2022-01-04T04:13:26Z DEBUG hyper::proto::h1::io] read 16384 bytes [2022-01-04T04:13:26Z DEBUG reddsaver::download] Created a file: /home/wavy/reddit/images/blackmagicfuckery/img-1a0ead8ec3f572fa3e9adaa839ef7893.mp4..mp4 [2022-01-04T04:13:26Z INFO reddsaver::download] Successfully saved media: /home/wavy/reddit/images/blackmagicfuckery/img-1a0ead8ec3f572fa3e9adaa839ef7893.mp4..mp4 from url https://v.redd.it/n1olrcmt9ju71/DASH_audio.mp4 [2022-01-04T04:13:26Z DEBUG reddsaver::download] Media type: RedditVideoWithAudio [2022-01-04T04:13:26Z DEBUG reddsaver::download] Media files: 2 [2022-01-04T04:13:26Z DEBUG reddsaver::download] Locally skipped items: 0 [2022-01-04T04:13:26Z DEBUG reddsaver::download] Assembling components together [2022-01-04T04:13:26Z DEBUG reddsaver::download] Executing command: "ffmpeg" "-i" "/home/wavy/reddit/images/blackmagicfuckery/img-44f5a045a48f97ce18e4abdce5ab9fb4.mp4..mp4" "-i" "/home/wavy/reddit/images/blackmagicfuckery/img-1a0ead8ec3f572fa3e9adaa839ef7893.mp4..mp4" "-c" "copy" "-map" "1:a" "-map" "0:v" "/tmp/.tmp9rp9h0/combined.mp4" [2022-01-04T04:13:26Z DEBUG reddsaver::download] Successfully combined into temporary file: "/tmp/.tmp9rp9h0/combined.mp4" [2022-01-04T04:13:26Z DEBUG reddsaver::download] Renaming file: /tmp/.tmp9rp9h0/combined.mp4 -> /home/wavy/reddit/images/blackmagicfuckery/img-44f5a045a48f97ce18e4abdce5ab9fb4.mp4 [2022-01-04T04:13:26Z DEBUG hyper::proto::h1::io] read 16384 bytes [2022-01-04T04:13:26Z DEBUG hyper::proto::h1::io] read 16384 bytes Error: IoError(Os { code: 18, kind: Other, message: "Invalid cross-device link" })

wavy-baby commented 2 years ago

Btw I checked the downloaded files and sometimes the audio is not merged with the video.

manojkarthick commented 2 years ago

The issue seems to be happening when trying to combine the video with audio. Currently the audio and video is downloaded separately and then combined using ffmpeg into /tmp and then moved to the target location (and delete the individual audio, video components) - are /tmp and /home/wavy on different filesystems? or do you have any non-standard filesystem or partitioning scheme?

wavy-baby commented 2 years ago

I don't think so Here is my "tree -L 1 /" output

/ ├── bin -> usr/bin ├── boot ├── dev ├── etc ├── home ├── lib -> usr/lib ├── lib64 -> usr/lib ├── lost+found ├── mnt ├── opt ├── proc ├── root ├── run ├── sbin -> usr/bin ├── srv ├── sys ├── timeshift ├── tmp ├── usr └── var

and here is my fdisk -l output

Device Start End Sectors Size Type /dev/nvme0n1p1 2048 1048576 1046529 511M EFI System /dev/nvme0n1p2 1050624 1000215182 999164559 476.4G Linux filesystem

manojkarthick commented 2 years ago

Interesting. Can you share few Reddit video links on your saved list (including ones that didn't get merged correctly)? I can try to see if I can reproduce this on a Arch Linux system.

wavy-baby commented 2 years ago

https://v.redd.it/h465tu5issh61/ https://v.redd.it/3x0biapyj5d61/ https://v.redd.it/em4lv7ixeuf61/ https://v.redd.it/i91gkjmw8mk61/ https://v.redd.it/ugwfb0q5o2p61/ Edit: https://v.redd.it/a899lmobkpl61

bradym commented 2 years ago

I ran into this error last night. My /tmp dir and the final destination are on different mount points in my case. Looking at the code I discovered that the temp_dir function looks at the TMPDIR env var.

I then created a tmp directory on the same mount point as the final destination and then when I ran this it worked:

env -i PATH=$PATH TMPDIR=/opt/tmp/ reddsaver --data-dir /opt/stuff
manojkarthick commented 2 years ago

This is great, thanks @bradym. I will add this info to a troubleshooting section on the README.