search

eyal0 / OctoPrint-PrintTimeGenius

Use a gcode pre-analysis to provide better print time estimation
190 stars 32 forks source link

Question: Using RSYNC, watched folder and PrintTimeGenius #303

Open mlewis-everley opened 1 year ago

mlewis-everley commented 1 year ago

As per issue #301 I have now managed to sync a remote gcode folder with the watched folder on multiple clone printers. OctoPrint picks up the new files, moves them about and PrintTimeGenius then analyses them (GREAT :-) ).

But, because OctoPrint moves (emptying the watched folder) all the files out of watched and replaces uploads with them, this means PrintTimeGenius runs on every file. This is not the end of the world, but means I can't run rsync as a scheduled task, as will re-write files that are currently printing.

It also results in an unnecessary amount of overhead, as PrintTimeGenius now re-analyses "new" files that it had technically analysed before.

I am just wondering if this is the best workflow? Is there a better solution? I checked the wiki but couldn't see anything that looked relevant.

eyal0 commented 1 year ago

We might have to hack something up. Let's see.

rsync has an option called --exclude-from that will allow you to choose which files shouldn't be synced. We don't want to sync anything that is already in the remote's uploads folder, right? Also, each exclude must be a regex pattern.

Here's a command that will list all the files in the remote's upload directory:

ssh remote.computer.com ls /home/mlewis/.octoprint/uploads

Here's a command that will take a file of input and stringify all the regular expressions so that they will not have special characters (for example, escaping the period in a filename):

python -c 'import re; import fileinput; print("\n".join(re.escape(x.rstrip("\n")) for x in fileinput.input()))'

Now we combine it as an argument to cat for testing purposes (this assumes that you are on the bash/zsh shell):

cat <(ssh remote.computer.com ls /home/mlewis/.octoprint/uploads | python -c 'import re; import fileinput; print("\n".join(re.escape(x.rstrip("\n")) for x in fileinput.input()))')

Try that command and see if it looks right to you. If you're happy with it then add it into your rsync command:

rsync <your usual arguments> --exclude-from=cat <(ssh remote.computer.com ls /home/mlewis/.octoprint/uploads | python -c 'import re; import fileinput; print("\n".join(re.escape(x.rstrip("\n")) for x in fileinput.input()))') <source path> <dest path>

Something like that might work? What do you think?

mlewis-everley commented 1 year ago

Thanks for the reply, makes sense, I will give it a try!

If it works, maybe I can comp together a Bash or Python script to do this based on this issue (I don't like running long commands as cron, as 6 months later, when I come back to them, I am all like "what the hell is this doing" :-) )

eyal0 commented 1 year ago

If you do make such a script, I'll put it into the wiki here: https://github.com/eyal0/OctoPrint-PrintTimeGenius/wiki/Common-problems