Not overwriting existing file when monitoring and using `--force`

[Infinitay]

I am still trying to figure out how to properly use this tool as I was under the impression it downloads the space in real-time. When I tried it first using twspace-crawler --env ./.env --config ./config.json --id <space_id> it found the space but did not start downloading it yet. After taking a closer look at the README and reading it in full I misunderstood the functionality.

It's written "Monitor user(s) indefinitely, wait for live Space and download when Space ended". I tried passing in --force to force it to download the space which it did. However, it kept fetching the space and attempting to re-download it as the terminal kept monitoring the same user. Furthermore, the downloaded file did not seem to be re-written after all those calls.

The reason for this I believe is due to the ffmpeg command. ffmpeg by default does not overwrite an existing file and asks you either to rename the file or overwrite it via -y flag which twspace-crawler does not use.

It would be nice if either

• --overwrite
  • We allow overwriting with ffmpeg although I would guess there could be any repercussions with this e.g. terminating mid re-write
  • If a file with the same name exists append "temp" the new file, download it, compare the sizes, and keep the file with the larger size
• --download-segments
  • Download the space's stream in segments and concat them into one file when the space has ended. Delete the segments once concatenated (optional?)

[HitomaruKonpaku]

Hmm the purpose of --force is to run with --id only to force download the live Space, not for monitoring. 🤔

[Infinitay]

Is it possible at all to do what I suggested? I'm worried about the recording not being available on the servers because they usually remove it pretty fast. I'm not sure if I had any issues with this yet using this tool, but manually it has happened. Same with broadcasts.

[HitomaruKonpaku]

Afaik Space master url always stay on their server for at least 30 days.

Unless the owner (host) decide to deactive/delete account.

I have it running for almost 1 year and no issue so far 🤔

[Infinitay]

watch: Request failed with status code 404

FWIW they did not opt to save a recording for this space

[HitomaruKonpaku]

Even when user do not choose to save, server will keep it.

You just need to get the playlist url before Space end.

[Infinitay]

I stand corrected. It worked just fine after I passed in the master_playlist URL. I kept trying the noRedirectPlaybackUrl found in the call to api/1.1/live_video_stream/status/.

Thanks for letting me know.