search

goatchurchprime / jupyter_micropython_kernel

Jupyter kernel to interact with a MicroPython/ESP8266 over the serial REPL
BSD 3-Clause "New" or "Revised" License
145 stars 32 forks source link

%sendtofile clean folder uploads #28

Open andrewleech opened 6 years ago

andrewleech commented 6 years ago

When uploading an entire folder to the device, if --mkdir is specified, then delete any existing copy of the folder before uploading new copy.

I think it feels natural that when uploading a folder, it will replace any existing copy of that folder, just like a file upload replaces the existing file. The previous implementation just copies files up, essentially merging the folders together. There was no existing means of deleting files that already existed in the folder.

Clean copies of the folder upload is far more useful in my opinion.

goatchurchprime commented 6 years ago

Have you noticed I've got this function done: %fetchfile [--binary] [--print] [--quiet] [--QUIET]

There were some tough bugs in it, to do with the yieldserialchunk() splitting out the 'OK's https://github.com/goatchurchprime/jupyter_micropython_kernel/blob/master/jupyter_micropython_kernel/deviceconnector.py#L424

Anyway, do you think perhaps we are just haphazardly implementing components of rsync here?
https://en.wikipedia.org/wiki/Rsync

I'm finding that code for scanning through a file to make a hash is very fast, so it makes sense to check for differences before doing up/downloads. Also, rsync has already standardized its meaning for recursion and directory transfers.

andrewleech commented 6 years ago

Nice work! Yes I feel you're probably right, this could be done cleaner & faster following rsync's lead.

The directory traversal stuff seems to be specific to the official rsync application, but at least following the format and conventions of it's source/dest args (trailing slashes etc) would make a lot of sense.

There are a few pure python implementations of rsync (like) hashed file sending: http://code.activestate.com/recipes/577518-rsync-algorithm/ https://blog.liw.fi/posts/rsync-in-python/ https://github.com/skorokithakis/omnisync

goatchurchprime commented 6 years ago

I've written up as the issue here. https://github.com/goatchurchprime/jupyter_micropython_kernel/issues/31

Maybe you could add what you think the rsync command parameters should be, and I'll get round to implementing it.

BTW I've got github support to detach me from the adafruit kernel, which was a year out of date, so now we will show up on searches.