bentonstark / starksoft-aspen

.net / mono security and cryptography library that provides client support for ftps, gnupg, smartcard, and socks / http proxies
108 stars 49 forks source link

Old _Param TransferProgressEventArgs Equivalent ? #30

Closed ghost closed 6 years ago

ghost commented 7 years ago

Hello

I tried using Starksoft.Aspen to update a project that uses the old Starksoft.Net.Ftp and ...Proxy

The program's code won't compile here: https://pastebin.com/dp8iRzMR

However there seems to be something missing, Parameter and Buffer.

I asked some people and they told me there is no replacement, the feature was removed? Is this true?

Is it possible that this could be brought back or that I could modify the code at my end to adapt to the new version?

And here's the old StarksoftTransferProgressEventArg.cs file, which is also included in the main program's source as a source lib and built along the way. https://pastebin.com/Ezbj5SwM

bentonstark commented 7 years ago

No. It is still supported. The event is called TransferProgress and it sends a TransferProgressEventArgs to the listener.
https://github.com/bentonstark/starksoft-aspen/blob/master/Starksoft.Aspen/Ftps/FtpsBase.cs#L640

There are also other events you can hook into as well. The transferProgressEventArgs is defined here. https://github.com/bentonstark/starksoft-aspen/blob/master/Starksoft.Aspen/Ftps/EventArgs/TransferProgressEventArgs.cs

bentonstark commented 7 years ago

Looking at this again I see you specifically mean the two arguments called Parameter and Buffer. I don't think that was in the original code. Is it possible you might have been using a modified source? I will go back and look at the old source code and see but those parameters do not look familiar but it has been quite a few years.

So here is a fork of my older code and it is not included. I think someone added that in a version I didn't write.
https://github.com/kbrammer/FtpClient/blob/master/Starksoft.Net.Ftp/EventArgs/TransferProgressEventArgs.cs

ghost commented 7 years ago

Oh wow, interesting, thanks a lot, when I tried to download the source to the old ones on various sites it had download errors and it kept redirecting me to this .Aspen site.

Well, hopefully I can get it updated to the newer version since I tried to add a new feature to compare checksum of the remote and local file and the VS2017 debugger reports Starksoft.Net.Ftp thread exceptions quite a few times throught the program, including when trying to get ... but just while writting this and I'm looking that maybe I didn't set the paths on the server correctly (only the base filename), I'll try that first so it's actually a full filepath.

Could be the way the program was written, it's quite a bit old, VS2010 and last updated in 2012.

But it's missing a key feature that I need, it's overwriting existing files without prompt, I'm trying to figure it out to make it skip existing files except if they are corrupted.

The program is a bit unstable as after some 20-30 minutes it will make fatal DL error in the middle of transfer, hence checksum checking.

And I have separated these projects into two, so in one I attempt to do my work with old FtpClient, and the other I try with new .Aspen FtpClient.

Excerpt of my added file skipping code in the main program. https://pastebin.com/gA5xm5fP

As far as my current attempts with the old FtpClient go I guess I'm on my own, as that's obsolete and if I am forced to switch to .Aspen I'll come back to ask about that if I don't figure out the replacement for _Param and _Buffer, if you don't mind, but hopefully I could avoid going there.

bentonstark commented 7 years ago

I think I can add the args you need without too much trouble. I am not sure what the Parameter arg is used for and how they are setting that. Any ideas?

ghost commented 7 years ago

That's exactly what I couldn't figure out, I can fire that project up and take a look again and probably will have to go that way as it seems that getting checksum still doesn't work despite setting proper path.

The file hash comparison actually includes files with different names, otherwise identical, but hashes don't include filenames, right.

Still, the whole thing doesn't complete the condition brackets, it never goes to else or anything so I'm sure if breaking.

System.Windows.Media.Animation Warning: 6 : Unable to perform action because the specified Storyboard was never applied to this object for interactive control.; Action='Stop'; Storyboard='System.Windows.Media.Animation.Storyboard'; Storyboard.HashCode='8409368'; Storyboard.Type='System.Windows.Media.Animation.Storyboard'; TargetElement='System.Windows.Media.Animation.Storyboard'; TargetElement.HashCode='8409368'; TargetElement.Type='System.Windows.Media.Animation.Storyboard'
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Starksoft.Net.Ftp.dll
The thread 0x1168 has exited with code 0 (0x0).
The thread 0x660 has exited with code 0 (0x0).
The thread 0x1c58 has exited with code 0 (0x0).
The thread 0x474 has exited with code 0 (0x0).
The thread 0x22e8 has exited with code 0 (0x0).
The thread 0xc2c has exited with code 0 (0x0).
The thread 0x1314 has exited with code 0 (0x0).
The thread 0x256c has exited with code 0 (0x0).
Exception thrown: 'System.Threading.ThreadAbortException' in Starksoft.Net.Ftp.dll
The thread 0x1618 has exited with code 0 (0x0).
DEBUG - Download: INF Filename Exists - Preparing to Check Integrity
Exception thrown: 'Starksoft.Net.Ftp.FtpResponseException' in Starksoft.Net.Ftp.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
The thread 0x2184 has exited with code 0 (0x0).

Well there's more of these thread exceptions, I could be wrong, maybe it's just a coincidence Stuff happens when I don't even do anything and the program just idles. But the point is, it never goes to the next breakpoint, and it also doesn't properly finish actions which were working before I added checksum checking, so I'm 100% sure something's wrong with the GetChecksum thingy.

But I think I should attempt with the newer .Aspen version and since that what you currently support, should be more proper, ofcourse we have to figure out the Parameter/Buffer thing in order to get it compiled at all first.

bentonstark commented 7 years ago

There were some issues with the older version so I would not try to use it. What FTP server are you connecting to and do you use TLS? The newest version is well tested with FileZilla.

On Mon, Jul 24, 2017 at 3:17 PM, ZexaronS notifications@github.com wrote:

That's exactly what I couldn't figure out, I can fire that project up and take a look again and probably will have to go that way as it seems that getting checksum still doesn't work despite setting proper path.

System.Windows.Media.Animation Warning: 6 : Unable to perform action because the specified Storyboard was never applied to this object for interactive control.; Action='Stop'; Storyboard='System.Windows.Media.Animation.Storyboard'; Storyboard.HashCode='8409368'; Storyboard.Type='System.Windows.Media.Animation.Storyboard'; TargetElement='System.Windows.Media.Animation.Storyboard'; TargetElement.HashCode='8409368'; TargetElement.Type='System.Windows.Media.Animation.Storyboard' Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll Exception thrown: 'System.Threading.ThreadAbortException' in Starksoft.Net.Ftp.dll The thread 0x1168 has exited with code 0 (0x0). The thread 0x660 has exited with code 0 (0x0). The thread 0x1c58 has exited with code 0 (0x0). The thread 0x474 has exited with code 0 (0x0). The thread 0x22e8 has exited with code 0 (0x0). The thread 0xc2c has exited with code 0 (0x0). The thread 0x1314 has exited with code 0 (0x0). The thread 0x256c has exited with code 0 (0x0). Exception thrown: 'System.Threading.ThreadAbortException' in Starksoft.Net.Ftp.dll The thread 0x1618 has exited with code 0 (0x0). DEBUG - Download: INF Filename Exists - Preparing to Check Integrity Exception thrown: 'Starksoft.Net.Ftp.FtpResponseException' in Starksoft.Net.Ftp.dll Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll The thread 0x2184 has exited with code 0 (0x0).

Well there's more of these thread exceptions, I could be wrong, maybe it's just a coincidence Stuff happens when I don't even do anything and the program just idles. But the point is, it never goes to the next breakpoint, and it also doesn't properly finish actions which were working before I added checksum checking, so I'm 100% sure something's wrong with the GetChecksum thingy.

But I think I should attempt with the newer .Aspen version and since that what you currently support, should be more proper, ofcourse we have to figure out the Parameter/Buffer thing in order to get it compiled at all.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bentonstark/starksoft-aspen/issues/30#issuecomment-317526527, or mute the thread https://github.com/notifications/unsubscribe-auth/AK8sQydx2WO2D5-tmSaQFim_byCyZUalks5sRO3PgaJpZM4OgeMw .

ghost commented 7 years ago

No, it's open anonymous ftp, it's also local LAN only, pretty basic imo, vsftpd I think, but I don't have any control over it since it's running inside custom firmware, which is way above my skills, so we can remove that out of the picture as I won't be able to fix anything in there in a timely fashion.

I'm preparing the newer project to reflect other changes, but I got some hicckup with the references and assemblies which I didn't had before, so probably not ready today.

ghost commented 7 years ago

Well after like 2 hrs of setting it all up, I began adapting the code to the .Aspen, but tomorrow, but for some reason some WPFLocalizeExtension reference won't work for some XAML files which is weird as it has nothing to do with Starksoft, but it's not a problem in the hotfix* project with the old Starksoft, could be a coincidence, I must have fiddled with that and forgot what made it work, I have to refiddle before it's buildable, but there's other things:

In the meantime there's other parts besides Parameter and Buffer I have to fix, one of them:

                            if (OTFDecrypt)
                            {
                                _isSynchronized = false;
                                using (MemoryStream tmpMemStr = new MemoryStream())
                                using (FileStream fs = new FileStream(outputFile, FileMode.Create, FileAccess.Write))
                                {
                                    ftpClient.GetFileAsync(currDl.FileName, tmpMemStr, false, new object[] { currDl, fs , downloadList[currDl], tmpMemStr});
                                    _dlCompletedSemaphore.WaitOne();
                                }
                            }
                            else
                            {
                                ftpClient.GetFileAsync(currDl.FileName, outputFile, FileAction.Create, currDl);
                                _dlCompletedSemaphore.WaitOne();
                            }

no overload for method "GetFileAsync" takes 4 arguments

currDl is some kind of a MicroMvvM observable object, some kind of a container, but I'll get in more detail later as I have to actually debug the whole thing to figure out what these things are in detail and for that I'll use the hotfix project with the old Starksoft.

But it shouldn't be much, also VS2017 afaik has that feature of only fully loading up what it's beeing looked at ... I should turn that off and make it traverse all files and point out errors.

But also I apologize if I may mix trivial errors, I don't happen to be an expert on C# or programming in general, and the VS error terminology. I ofcourse will make sure that I only bring up things here that are about .aspen.

ghost commented 7 years ago

Well hmm... file hashing might not be supported by the server, but if this newer .aspen fixes the unstability, this might not be needed so I hope.

Okay, so _handled is pretty self explanatory looking at some of the comments.

        /// <param name="param">Param given by the user to caller</param>
        /// <param name="buffer">The buffer.</param>
        /// <param name="handled">if set to <c>true</c> [handled].</param>

        /// Gets or sets a value indicating whether this <see cref="TransferProgressEventArgs"/> is handled.
        /// </summary>
        /// <value>
        ///   <c>true</c> if handled; otherwise, <c>false</c>.

        object _param;
        bool _handled;
        byte[] _buffer;

............. Actually I kinda went on more and tried implementing the custom changes along with the .Aspen and the more I dig in the more it seems it's quite interwined with the way the old one worked with custom things ontop,

It looks like this may be out of my league to go an adapt .Aspen or the main program it self to work with .Aspen, the GetFileAsync taken me down into a path of having to do some weird void replication with fourth parameter "param" added to it, some major pieces like TransferBytes() ... which doesn't exist anymore, but also I would have to be a lot more familiar with the whole program in order to make change it so it works with .Aspen in a different way so it uses the newer code except only the bits that are directly necessary are adapted into aspen.

So I'm going to go back to the hotfix verion and remove the extra feature of hashing, because it should still work to detect existing files nontheless, I would just have to make sure manually about the integrity by checking the filesize. I might then be able to debug and find the culprit behind the errors in middle of transfer.

Could be many weeks or months before I get enough familiarity of how it works, and ofcourse, fiddling with the original (hotfix) version is the way to go to figure how it it all works first, before trying to move to aspen, obviously that's just me, someone with expert knowledge of programming in general could just skip this step.

ghost commented 7 years ago

Allright, so I found out the old program wasn't properly keeping the connection alive, one of the starksoft components made some connection loss exception, but I decided to rewrite the whole thing in Winforms and under Net 4.7, not implementing the unnecessary features that I don't need and getting rid of complicated threading, so I might come to a point when I will maybe need this "param" and "buffer" thing again.

I finished the GUI, and now coding the first parts to establish a FTP connection.

Now I know this isn't the place to ask general support questions, but I'm just adding this note for an overwite, I'll obviously use other means to try to get to the bottom of this, but just incase I don't figure it out.

Seems like it establishes, but there's an exception thrown in Starksoft.Aspen.Ftps.FtpsFeatureException with the message "epilogue length invalid". http://i.imgur.com/3hM4c4T.jpg

I'll eventually get to the core related issue of Param and Buffer later on when I finish the necessary code prior to it.

EDIT: seems like epilogue length invalid isn't that common message. EDIT2: private const int FEAT_EPILOGUE_LEN = 7; ... this doesn't exist in the old starksoft, so I can't use the old program to help, other people also suggested it's probably valid to open up an issue here about it.

EDIT3: Debug info http://i.imgur.com/HA9ShvH.jpg - And just to repeat, filezilla and the old program still connect perfectly fine, this is just when using .Aspen in my new Solution.

EDIT4: This is what filezilla reports https://pastebin.com/LHRyJkgf

bentonstark commented 7 years ago

The epilogue is the feature set (FEAT command response) returned by the FTP server. While parsing that data each command it is expected to be a specific length of 6 characters or less. It seems your FTP server is giving back a feature command called "REST STREAM" (RESTart stream) which is causing the problem. The code is looking at the command length and if it great than 6 characters (the longest FTP command) it will throw the exception. I can make an adjustment to the code to account for these longer commands.

Since you have the code in a debugger just set the const value FEAT_EPILOGUE_LEN to something like 20 to continue your testing.

https://github.com/bentonstark/starksoft-aspen/blob/master/Starksoft.Aspen/Ftps/FtpsFeatureCollection.cs#L321

ghost commented 7 years ago

Well, now that I'm sure what's going on, I can then simply raise the default value of 7.

I found out a few days ago that the server is a linux version of busybox, not sure which version, but it's definitely somewhere from year 2012 and below.

If you want you can make support for letting the REST STREAM exception, which might be exclusive to busybox, ofcourse it may not be necessary if it's something that was fixed in newer ... err. I just checked it, the latest busybox ELF binary still has the same long REST STREAM thing.

EDIT: Actually it's all different as it seems, it's not testing if the text is more than 7, but less than 7, that's just me I think I didn't look too carefully at the operand, I tried 12, 3, 15, 17 and then finally 128 and it always produces same exact exception. It's always saying that the "fl" value is "211 Ok" when it throws the exception, just as in the original picture, if REST STREAM was the issue wouldn't it show it as the "fl" value in debugger instead, right.

Maybe the value meant there should be 6, and "211 Ok" is exactly 6 so it's not lower than 6.

EDIT-2: After using value 6 it let's it forward and breaks below at the "epilogue invalid". In the debugger I see that it wants the string "211 END" instead of what busybox provides "211 OK" ... I guess I'll have to customize this as well, nothing I can do about the server on that device.