File Conveyor is a daemon written in Python to detect, process and sync files. In particular, it's designed to sync files to CDNs. Amazon S3 and Rackspace Cloud Files, as well as any Origin Pull or (S)FTP Push CDN, are supported. Originally written for my bachelor thesis at Hasselt University in Belgium.
In #62, File Conveyor is getting proper Unicode support, from beginning to end. However, the destinations the files are being sent to may not accept Unicode! And if they do, or if some automatic or default encoding of filenames happens, it may result in wrong (or at least: deviating from what we want) filenames.
This should be a non-issue for Amazon S3/CloudFront, which already should be okay. It should also be a non-issue for SFTP — @unn pointed to the documentation of paramiko's _to_unicode(), which certainly suggests it should already be working. It's easily fixable for Rackspace Cloudfiles: the service supports it, the current code does not, but it's already been fixed upstream.
Most likely, that will only leave the FTP transporter as being problematic. Since FTP is insecure, old and rotten anyway, it's unlikely this will be fixed. It's also likely to be an extreme pain in the ass. So, unless somebody contributes it, it's unlikely to happen.
For now, this issue will remain open, unless nobody reports Unicode problems over the next few months for any of the transporters.
In #62, File Conveyor is getting proper Unicode support, from beginning to end. However, the destinations the files are being sent to may not accept Unicode! And if they do, or if some automatic or default encoding of filenames happens, it may result in wrong (or at least: deviating from what we want) filenames.
This should be a non-issue for Amazon S3/CloudFront, which already should be okay. It should also be a non-issue for SFTP — @unn pointed to the documentation of paramiko's
_to_unicode(), which certainly suggests it should already be working. It's easily fixable for Rackspace Cloudfiles: the service supports it, the current code does not, but it's already been fixed upstream.Most likely, that will only leave the FTP transporter as being problematic. Since FTP is insecure, old and rotten anyway, it's unlikely this will be fixed. It's also likely to be an extreme pain in the ass. So, unless somebody contributes it, it's unlikely to happen.
For now, this issue will remain open, unless nobody reports Unicode problems over the next few months for any of the transporters.