fast-data-transfer / fdt

FDT is an Application for Efficient Data Transfers which is capable of reading and writing at disk speed over wide area networks (with standard TCP). It is written in Java, runs an all major platforms and it is easy to use. FDT is based on an asynchronous, flexible multithreaded system and is using the capabilities of the Java NIO libraries.
https://fast-data-transfer.github.io/
Apache License 2.0
198 stars 45 forks source link

Windows Suggested Fixes #30

Open wkwood opened 6 years ago

wkwood commented 6 years ago

I'd like to submit this patchfile that I've developed against 0.26.1 for Windows. It cleans up quite a few problems we were seeing. I can document these further if you'd like.

1) Cleanup the POM and dependencies. Generally speaking, more could be done here. 2) Fix issues with Windows file system nuances vs. *NIX 3) Config. Add a method to find an available port and also don't use the default -tp port which we don't know if it's safe to use or not on the local system. 4) FDTWriterSession don't create the output directory tree. You may want to consider this change that it will break recursion, I'll gladly support -r with another change if you'd like so the directories are built. 5) ControlChannel flush the output stream prior to closing it and then wait a second before trying to close the socket. This cleaned up a Server side issue for us where the server would complain about the remote side disappearing.

patchfile.txt

juztas commented 6 years ago

Can you create pull request for this? checkout master branch and apply your patch, double check for changes and create pull request (p.s. master branch is the development one)

wkwood commented 6 years ago

Sure, I can do that if you’d like. It may take me a few days since I’m off for the holidays though.

Would that be alright?

From: Justas Balčas [mailto:notifications@github.com] Sent: Wednesday, December 13, 2017 9:02 AM To: fast-data-transfer/fdt fdt@noreply.github.com Cc: wkwood wkwood@gmail.com; Author author@noreply.github.com Subject: Re: [fast-data-transfer/fdt] Windows Suggested Fixes (#30)

Can you create pull request for this? checkout master branch and apply your patch, double check for changes and create pull request (p.s. master branch is the development one)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/fast-data-transfer/fdt/issues/30#issuecomment-351416479 , or mute the thread https://github.com/notifications/unsubscribe-auth/AEd0AIcC-OKmo-FcL5YhEm7y8hc8h89Xks5s_-bRgaJpZM4RAnmZ . https://github.com/notifications/beacon/AEd0AGWFa__PLtRDykujov_jT17Svoy0ks5s_-bRgaJpZM4RAnmZ.gif

mark-duggan commented 6 years ago

Hi,

Would this address the issue when I use the filelist in Windows in that it can't create the parent directories? Is there a way to suppress the attempt to create the parent directories?

Mark

mark-duggan commented 6 years ago

Answering my own question here, but applying the patch did indeed seem to resolve the issue with the filelist. Thanks!

mgpeschke commented 5 years ago

Thank you for creating the patch file.

Is there a time frame when the new version of FDT will be available?

From my testing, I notice that the suggested changes will resolve an issue where FDT + GSI in server mode use up a lot of CPU after each transfer.