Creation of remote path on Windows

[GoogleCodeExporter]

What steps will reproduce the problem?
1. be on Windows
2. start a synchronization

What is the expected output? What do you see instead?
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "disk\ftpsync.py", line 668, in main
    session = FtpSession(remote_path)
  File "disk\ftpsync.py", line 546, in __init__
    assert remote_path.startswith('/'),`remote_path`
AssertionError: '\\dirtosync'

What version of the product are you using? On what operating system?
Version 1.1-svn on Windows XP SP3

Please provide any additional information below.
I wrote here to try this feature of Google Code! :-)

Original issue reported on code.google.com by Iacopo.M...@gmail.com on 5 Jan 2009 at 11:07

[GoogleCodeExporter]

Thanks for the bug report.
ftpsync2d has been developed under linux and it
has not been tested on Windows. That said, I think
ftpsync2d should work on Windows provided that
the ftp paths contain `/` characters as path separators.
So, could you try using `/' instead of `\` in the
remote path argument of ftpsync.py? Does it work then?
If not, what exact command line are you using (don't send
usernames or passwords or any other sensitive path information)?

Original comment by pearu.peterson on 6 Jan 2009 at 10:02

[GoogleCodeExporter]

Don't mention it. But the solution does'nt seems so simple. I think the problem 
could 
be in using os.path to build remote URLs. I added the line "path = 
path.replace('\\', 
'/')" where path is a remote path in 2 places obtaining the program working on 
the 
download step. But the upload step does not work (generate a ftp filename error 
(550 
file name forbidden), for ex. "/dirtosync/subdir\.listing") probably because of 
os.path.dirname function used in makedirs. So, to give you an idea before 
adding some 
other rash changes I attached the 2-lines-modified program.

Original comment by Iacopo.M...@gmail.com on 8 Jan 2009 at 11:42

Attachments:

• ftpsync11mod.py

[GoogleCodeExporter]

Ok, I agree that using os.path.join for building
up a remote path is not good -- it can only work if
both the local and remote systems are the same.
So, a solution is to use custom path join functions
that takes into account what systems are being used.

To make things simpler, can we assume that the remote
server is always using `/` path separator?
If so, then let's create a function
{{{
  def remote_path_join(*paths):
      # untested code!
      l = []
      for path in paths:
          l.extend(path.split(os.path.sep))
      return '/'.join(l)
}}}
and use it anywhere os.path.join is used for remote paths.
Could you test this approach on Windows?

Original comment by pearu.peterson on 8 Jan 2009 at 2:33

[GoogleCodeExporter]

I'm testing it, but the first synchronization takes more than 1 hour for my 
directory 
and I still found some problem with the paths (damned Windows). I completed it 
right 
now (few seconds ago) for the first time, but since my script is a bit 
different from 
yours I can't immediatly send you the right modificated code.

Anyway the normal 1.1 version seems to work good on Mac OS.

Original comment by Iacopo.M...@gmail.com on 10 Jan 2009 at 12:43

[GoogleCodeExporter]

Windows issue: repeating sinchronization immediatly after a synchronization 
takes the
same time.

Original comment by Iacopo.M...@gmail.com on 12 Jan 2009 at 6:57

[GoogleCodeExporter]

I have applied a not-so-beautiful (read: ugly) patch over the r6 svn version. 
Now it 
works in Windows. I just have added a few extra conditions to the failing 
asserts, 
so now they accept also paths with windows slashes "\".

Original comment by javca...@gmail.com on 12 Feb 2010 at 1:26

Attachments:

• ftpsync.py

[GoogleCodeExporter]

Even with #6s hacked version I couldnt get it to work so I hacked it some more 
and
came up with a version that works for me. But it is a hack. There is some 
serious
code that needs to be rewritten. Wouldnt a switch for remote_os be the solution?

Original comment by nilsma...@gmail.com on 26 May 2010 at 3:17

[GoogleCodeExporter]

I am working on Linux and try to get this working with a remote Windows server.

To sync the current dir with the remote dir /public/downloads the call to the 
script is:
ftpsync.py ftp://<user>:<password>@<server>/public/downloads . --upload

It seems the script does not 'see' the public dir on the remote server, so I 
get a stacktrace as follows:

listing directory '/public/downloads/' <connecting to x.y.nl..<creating 
directory '/public'>Traceback (most recent call last):
  File "/home/nico/develop/projects/python/ftpsync/ftpsync.py", line 504, in <module>
    main()
  File "/home/nico/develop/projects/python/ftpsync/ftpsync.py", line 461, in main
    remote_files = session.get_files(update_listing=options.update_listing)
  File "/home/nico/develop/projects/python/ftpsync/ftpsync.py", line 272, in get_files
    for rfn, mtime in self.get_remote_files(update_listing=update_listing).items():
  File "/home/nico/develop/projects/python/ftpsync/ftpsync.py", line 191, in get_remote_files
    lst = filter(lambda n: n not in ['.', '..'], self.ftp.nlst(wd))
  File "/home/nico/develop/projects/python/ftpsync/ftpsync.py", line 84, in ftp
    self.clock_offset = self.clocksync()
  File "/home/nico/develop/projects/python/ftpsync/ftpsync.py", line 126, in clocksync
    self.makedirs(os.path.dirname(rfn))
  File "/home/nico/develop/projects/python/ftpsync/ftpsync.py", line 313, in makedirs
    self.makedirs(parent, verbose=verbose)
  File "/home/nico/develop/projects/python/ftpsync/ftpsync.py", line 318, in makedirs
    self.ftp.mkd(fullpath)
  File "/usr/lib/python2.6/ftplib.py", line 556, in mkd
    resp = self.sendcmd('MKD ' + dirname)
  File "/usr/lib/python2.6/ftplib.py", line 243, in sendcmd
    return self.getresp()
  File "/usr/lib/python2.6/ftplib.py", line 218, in getresp
    raise error_perm, resp
ftplib.error_perm: 550 Directory already exists

As I am a newbee for Python, it will be a nice exercise to see what is causing 
it. So I hope to post feedback and possibly a patch later.

Original comment by nicodenb...@yahoo.com on 23 Sep 2011 at 9:59

[GoogleCodeExporter]

I found the solution for my case.
The windows server returns a dir listing like:
/public/downloads
for each entry.

In the method makedirs() the checking is done for a dirname with / as a prefix, 
like public

In this method, I have inserted one line at line # 327:
        windows_root = parent == '/' and ("/" + name) in lst
and then changed the next line, by adding 2 conditions:
        if name and name not in lst and fullpath not in lst and not windows_root:

This works for me.

Now I see a next problem. This script modifies my local modified time. No 
problem if you have one server, but I want to use this script to sync my local 
files with multiple mirror servers. This would mean I will need to change the 
mechanism which determines if a file needs to be uploaded, otherwise it will 
always upload everything. However, this is a feature request which I might 
implement myself. So far, so good. One bug fixed ;)

Original comment by nicodenb...@yahoo.com on 23 Sep 2011 at 2:43

[GoogleCodeExporter]

After fixing the problem with the remote Windows server (using a local Linux 
workstation), it turned out that my only remaining problem was; "This script 
modifies my local modified time". Which is killing for other syncs and backups.

Attached the version of the script I am using, with the fixes needed to make it 
work.

I am using this script to upload stuff to a few mirrors. The Linux mirrors can 
be synced with rsync, the Windows mirrors with the attached script.

Pearu Peterson; thanks for this script! It saved me quite some work and as a 
newbee it was educational to go trough the code...

Original comment by nicodenb...@yahoo.com on 25 Sep 2011 at 9:42

Attachments:

• ftpsync11mod.py