Sometimes ntfs alternate data streams remain in file

T5-R commented 6 years ago

It might be because I had to abort some plots and then replot them, but recently on the plots I did the ADS remained. Normally I would not have noticed, creepminer is the only miner that throws a warning "plot file incomplete" - after a bit of research I found that this is because of the alternate stream. Maybe you could add a check to remove the alternate stream once plotting finishes? Not sure why but it remained on some of my files even though the plotting was complete. Or maybe just explain when this can happen?

I was using the latest version.


PS C:\install\SysinternalsSuite> .\streams.exe -s -d W:\

streams v1.60 - Reveal NTFS alternate streams.
Copyright (C) 2005-2016 Mark Russinovich
Sysinternals - www.sysinternals.com

W:\Plots\11111111111111111111_282296375_1907328:
   Deleted :stream:$DATA
W:\Plots\11111111111111111111_284203703_1907328:
   Deleted :stream:$DATA
W:\Plots\11111111111111111111_289925687_1907328:
   Deleted :stream:$DATA
W:\Plots\11111111111111111111_291833015_1907328:
   Deleted :stream:$DATA
W:\Plots\11111111111111111111_293740343_1907328:
   Deleted :stream:$DATA
W:\Plots\11111111111111111111_295647671_1907328:
   Deleted :stream:$DATA
W:\Plots\11111111111111111111_297554999_1907328:
   Deleted :stream:$DATA
W:\Plots\11111111111111111111_299462327_1907328:
   Deleted :stream:$DATA
W:\Plots\11111111111111111111_301369655_1907328:
   Deleted :stream:$DATA
W:\Plots\11111111111111111111_303276983_1907328:
   Deleted :stream:$DATA`

(replaced the id)

JohnnyFFM commented 6 years ago

Hi, had a quick look at the code. My understanding is that there is no line of code that would cause the stream to be deleted. The stream will remain in the filesystem and wastes 8 byte per file (let me know if I'm wrong here). The stream holds a UInt64 with the number of nonces plotted. What I do not understand is why creep would report some of your files as incomplete, if xplotter reports them as complete.

I would first check if Xplotter reports the file as complete by trying to resume the plot (if yes, it means that the stream holds the total number of nonces).

Then I would have a closer look at creepminer code:

https://github.com/Creepsky/creepMiner/blob/b3bb638cac5826ad44a4513388292b5fa1a7b4e3/src/MinerUtil.cpp, lines 170+

It should perform exactly the same check as xplotter: compare number of nonces with the value stored in the stream. If one succeeds and one fails something is bugged. Looking at the creep code it could be a problem with endianness. it reads some chars in sequence and casts to UInt64. Not sure if this works in nix (big endian) and windows (little endian) at the same time. But I'm not an expert on this matter, someone would need to debug it, I might be completely wrong. Are you on nix or win?

T5-R commented 6 years ago

Hmm ok. I think I plotted the files that do not have the alternate stream with Turboplotter. So it is normal that this remains - good to know. Probably also the reason why xplotter cannot be used on exFat for example. But I suppose it is not needed for mining to work.

Why creepminer was complaining I do not know. I have a few other plots from xplotter that still have the alternate stream (just checked) and are fine in ceepminer. The files did complete according to xplotter... normal complete message. I did try to resume one if I remember correctly, but it did not resume. The files also verified fine in Turboplotter.

Probably nothing to worry about. Thanks for looking into it, but mabye it was some kind of freak-issue and nobody else is seeing this.

JohnnyFFM commented 6 years ago

Oki, I'm closing this! Best, Johnny

JohnnyFFM / XPlotter

Sometimes ntfs alternate data streams remain in file #6