search

SuperGouge / ChanThreadWatch

Fork of the original discontinued ChanThreadWatch.
90 stars 13 forks source link

Threads.txt backup #19

Closed fJo81VzbYs2laS9o closed 10 years ago

fJo81VzbYs2laS9o commented 10 years ago

Can you add an option for CTW to make an autoupdating threads.txt backup everytime a thread is added? I've had CTW crash twice randomly and lost all threads. Upon restarting the list was completely empty.

SuperGouge commented 10 years ago

Was any error logged at the time of the crash? There are safeguards that are supposed to prevent a thread list corruption in case of errors.

fJo81VzbYs2laS9o commented 10 years ago

No, it just crashed with a windows error report to submit to microsoft. There was no CTW error window.

The last time it happened when there was high disk activity. Was moving/copying files, playing music and browsing etc...

fJo81VzbYs2laS9o commented 10 years ago

Just to clarify the feature request: please make an option for the threats.txt to be automatically backed up every time threads are added manually or with auto-follow additions.

SuperGouge commented 10 years ago

I mean was anything logged in the log.txt file in the settings folder?

There is already a timer that auto-saves the thread list when adding threads.

fJo81VzbYs2laS9o commented 10 years ago

The time of the log.txt file was at about the same time it crashed last. Its 0 bytes and completely empty

SuperGouge commented 10 years ago

It seems like this was probably a external problem causing the corruption of the settings folder, probably due the said high disk activity. I can't think of anything in the program resulting in this behavior. Even if errors occur, the files stay untouched.

fJo81VzbYs2laS9o commented 10 years ago

I feel like I have to make a note of these crashes that I've been experiencing because they happen on regular intervals. It happens regularly like once every few days to sometimes two in one day. They happen during low and high disk activity. This last crash, I lost all but one actively watched threads.

At the time I know ctw was actively downloading and writing to disk, because the window froze and I can clearly see it was stuck downloading an image. Everytime this happens I try to pin down what was happening but so far it seems like there was CTW writing activity to threads.txt, at least when losing watched threads are concerned. Upon restarting ctw, almost all my actively watched threads were gone. What was left was one actively watched thread, and a bunch of dead threads that got pruned.

Windows7 generated an error report, which I sent off but there was no solution to the problem.

Is anyone else experiencing crashes at all? It can't be just me, it happens regularly like once every few days to sometimes two in one day.

fJo81VzbYs2laS9o commented 10 years ago

Log file contents

[08/06/2014 - 9:30:13 PM]
System.NullReferenceException: Object reference not set to an instance of an object.
   at JDP.SiteHelper_4chan_org.ResurrectDeadPosts(HTMLParser previousParser, List`1 replaceList)
   at JDP.ThreadWatcher.Check()
-------------------------------
[08/06/2014 - 9:30:49 PM]
System.NullReferenceException: Object reference not set to an instance of an object.
   at JDP.SiteHelper_4chan_org.ResurrectDeadPosts(HTMLParser previousParser, List`1 replaceList)
   at JDP.ThreadWatcher.Check()
-------------------------------
[08/06/2014 - 9:31:05 PM]
System.NullReferenceException: Object reference not set to an instance of an object.
   at JDP.SiteHelper_4chan_org.ResurrectDeadPosts(HTMLParser previousParser, List`1 replaceList)
   at JDP.ThreadWatcher.Check()
-------------------------------
[25/06/2014 - 5:03:55 AM]
System.IO.IOException: Access to the path 'J:\CTW_2014\THREADS\4chan_soc_19596899' is denied.
   at System.IO.Directory.InternalMove(String sourceDirName, String destDirName, Boolean checkHost)
   at JDP.ThreadWatcher.TryRenameThreadDownloadDirectory(Boolean calledFromCheck)
-------------------------------
[26/06/2014 - 5:07:25 AM]
System.IO.IOException: Cannot create a file when that file already exists.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Directory.InternalMove(String sourceDirName, String destDirName, Boolean checkHost)
   at JDP.ThreadWatcher.TryRenameThreadDownloadDirectory(Boolean calledFromCheck)
-------------------------------
[28/06/2014 - 11:18:17 PM]
System.IO.DirectoryNotFoundException: Could not find a part of the path 'J:\CTW_2014\THREADS\soc_misc\4chan_soc_19628465 (4chan_soc_19639505)'.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Directory.InternalMove(String sourceDirName, String destDirName, Boolean checkHost)
   at JDP.ThreadWatcher.TryRenameThreadDownloadDirectory(Boolean calledFromCheck)
-------------------------------
[29/06/2014 - 9:34:01 PM]
System.IO.DirectoryNotFoundException: Could not find a part of the path 'J:\CTW_2014\THREADS\soc_misc\4chan_soc_19628465 (4chan_soc_19639505)'.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Directory.InternalMove(String sourceDirName, String destDirName, Boolean checkHost)
   at JDP.ThreadWatcher.TryRenameThreadDownloadDirectory(Boolean calledFromCheck)
-------------------------------
[29/06/2014 - 10:56:06 PM]
System.IO.DirectoryNotFoundException: Could not find a part of the path 'J:\CTW_2014\THREADS\soc_sma\4chan_soc_19453109 (4chan_soc_19459917_sma-skinny-medium-average-girls-post-itt-females)'.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Directory.InternalMove(String sourceDirName, String destDirName, Boolean checkHost)
   at JDP.ThreadWatcher.TryRenameThreadDownloadDirectory(Boolean calledFromCheck)
-------------------------------
[29/06/2014 - 10:56:06 PM]
System.IO.DirectoryNotFoundException: Could not find a part of the path 'J:\CTW_2014\THREADS\soc_chubby\4chan_soc_19472236 (4chan_soc_19449113_chub-chubby-girls-thread-if-youre-new-please)'.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Directory.InternalMove(String sourceDirName, String destDirName, Boolean checkHost)
   at JDP.ThreadWatcher.TryRenameThreadDownloadDirectory(Boolean calledFromCheck)
-------------------------------
[29/06/2014 - 10:56:59 PM]
System.IO.DirectoryNotFoundException: Could not find a part of the path 'J:\CTW_2014\THREADS\soc_sma\4chan_soc_19453109 (4chan_soc_19459917_sma-skinny-medium-average-girls-post-itt-females)'.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Directory.InternalMove(String sourceDirName, String destDirName, Boolean checkHost)
   at JDP.ThreadWatcher.TryRenameThreadDownloadDirectory(Boolean calledFromCheck)
-------------------------------
[29/06/2014 - 11:00:15 PM]
System.IO.DirectoryNotFoundException: Could not find a part of the path 'J:\CTW_2014\THREADS\soc_hgt\4chan_soc_19453632 (4chan_soc_19400237_hgt-hairy-girls-thread-females-only-old-thread)'.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Directory.InternalMove(String sourceDirName, String destDirName, Boolean checkHost)
   at JDP.ThreadWatcher.TryRenameThreadDownloadDirectory(Boolean calledFromCheck)
-------------------------------
SuperGouge commented 10 years ago

I've tried to fix some bugs that appeared in your log (others being normal activity it seems). However, if the program crashes it seems we are dealing with an unhandled exception and thus it is not logged.

fJo81VzbYs2laS9o commented 10 years ago

Hahaha that's good to hear because I felt like the one boy crying wolf all the time. also, pardon the formatting, it was just acopy/paste, I have no idea how to format on github

fJo81VzbYs2laS9o commented 10 years ago

I wish there was a more detailed log. The times on the logs don't correspond to the time of the crashes

fJo81VzbYs2laS9o commented 10 years ago

FYI, found a couple of threads that caused "stopped: error unknown" Was re-adding some camwhore threads when a couple of linked threads caused this error. http://boards.4chan.org/soc/thread/19600737 http://boards.4chan.org/soc/thread/19622220 edit another "stopped:error unknown" thread just happened http://boards.4chan.org/soc/thread/19605390/sbt-small-breast-thread-tits-are-the-qualifier

do these links help you at all? or am I just cluttering the board?

SuperGouge commented 10 years ago

When a thread is stopped and says "Error writing to disk" or "Unknown error" there should be a log of the exact error that happened. I'd say it is probably due to the error I saw in your log and probably fixed.

If you want you can try the latest debug build (but remember it might not be stable so take your precautions) : http://www.mediafire.com/download/e0eyh4x9p5a2oa1/ChanThreadWatch-Debug.zip Hopefully those errors should be fixed and instead of a crash (if it happens again) you should see more information about the error. And if you generate an error report again try to find in the dialog where it was saved locally and give it to me. Microsoft can't do anything about those reports.

fJo81VzbYs2laS9o commented 10 years ago

Thanks, I'll test it out. Got this when trying to close down ctw after switching from application data folder to executable folder. Then tried to close down ctw..

See the end of this message for details on invoking 
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.FormatException: Input string was not in a correct format.
   at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal)
   at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info)
   at System.Int32.Parse(String s)
   at JDP.frmChanThreadWatch.frmChanThreadWatch_FormClosed(Object sender, FormClosedEventArgs e) in c:\Program Files (x86)\Chan Thread Watch\Source\frmChanThreadWatch.cs:line 154
   at System.Windows.Forms.Form.OnFormClosed(FormClosedEventArgs e)
   at System.Windows.Forms.Form.WmClose(Message& m)
   at System.Windows.Forms.Form.WndProc(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

************** Loaded Assemblies **************
mscorlib
    Assembly Version: 4.0.0.0
    Win32 Version: 4.0.30319.18444 built by: FX451RTMGDR
    CodeBase: file:///C:/Windows/Microsoft.NET/Framework/v4.0.30319/mscorlib.dll
----------------------------------------
ChanThreadWatch
    Assembly Version: 1.10.0.5
    Win32 Version: 1.10.0.5
    CodeBase: file:///E:/ctw-db/ChanThreadWatch.exe
----------------------------------------
System.Windows.Forms
    Assembly Version: 4.0.0.0
    Win32 Version: 4.0.30319.18408 built by: FX451RTMGREL
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System.Drawing
    Assembly Version: 4.0.0.0
    Win32 Version: 4.0.30319.18408 built by: FX451RTMGREL
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
System
    Assembly Version: 4.0.0.0
    Win32 Version: 4.0.30319.18408 built by: FX451RTMGREL
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Configuration
    Assembly Version: 4.0.0.0
    Win32 Version: 4.0.30319.18408 built by: FX451RTMGREL
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Configuration/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
System.Xml
    Assembly Version: 4.0.0.0
    Win32 Version: 4.0.30319.18408 built by: FX451RTMGREL
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------
System.Web
    Assembly Version: 4.0.0.0
    Win32 Version: 4.0.30319.18446 built by: FX451RTMGDR
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_32/System.Web/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Web.dll
----------------------------------------
System.Core
    Assembly Version: 4.0.0.0
    Win32 Version: 4.0.30319.18408 built by: FX451RTMGREL
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
    <system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.
fJo81VzbYs2laS9o commented 10 years ago

SuperGouge,

Here's the last 2 dumps that I snagged. The one dated July 7 is from the release you linked me several days ago. I forget if the one dated July 2 is from the same version.

http://www.mediafire.com/download/msebyh83xhxy2uc/CTW-crashes.zip

SuperGouge commented 10 years ago

Thanks for your time. I finally found where the problem is, however I don't know how to fix it. It's in a very low-level and critical part of the program. Messing with it is really difficult since I haven't yet had to dive into it since I started this fork.

Do you know with which version did you start having those crashes? Are you using the program in any unconventional way? Did you notice a pattern in the crashes?

Also, if anyone else is reading this, do you experience crashes too?

Finally, if any developer want to help on this problem, here is the exception causing the crash: https://github.com/SuperGouge/ChanThreadWatch/blob/master/Classes/Other.cs#L425 Pull requests welcome.

fJo81VzbYs2laS9o commented 10 years ago

I've had noticable crashes since 1.9.x I used to run CTW for weeks with dozens and dozens of threads without crashing. Ever since 1.9.x release they happen anywhere from one or two every couple of days or so. CTW usually behaves well after a restarted but after a couple of days, I would expect a crash, and a crash usually follows +/- a day.

I usually keep a good number of threads watched (anywhere from 30 to 120) but when I catch it right after a crash, I'm either browsing or in some other app. Sorry I know that doesn't really help. But I don't think its disk activity related since there have been a couple of times when I'm just reading news online or a forum and its crashed.

Let me know if you need more info or if theres anything more I can help with.

fJo81VzbYs2laS9o commented 10 years ago

Maybe post on the news or main page of SuperGouge/ChanThreadWatch? or even in /g/? Most people probably don't read through issues part of github

SuperGouge commented 10 years ago

Before 1.9.0 or even with the original version did you use to run CTW for such extended periods of time?

As for people on /g/ or anywhere else I don't expect anyone to be really useful since the problem comes from code written by the original author who seems to be MIA. It would require someone to really take time to fully understand this part of the program. I'll keep on trying but I don't promise anything.

fJo81VzbYs2laS9o commented 10 years ago

Yes I've had CTW run literally for months before. I only closed CTW for system/program updates.

Keep in mind that I was using Jarts version and that it didn't have auto follow threads to keep the program as active, but I very rarely ever closed the program.

fJo81VzbYs2laS9o commented 10 years ago

Had another crash today that cleaned out my threads.txt. Opened up the threads.txt file after the crash and the only text in there was "4".

SuperGouge commented 10 years ago

I'm sorry to hear that. The fact that it wipes the thread list is really weird. I will work on that thread list backup option. How do you think I should make such a feature? Multiple backups, custom time interval, auto-pruning? Any suggestions?

jwshields commented 10 years ago

I'd like to input on the backup option, I personally have never had a crash, but I follow everything on this project.. For the backup, I propose something like a threads.txt.bak or something, which is only updated when a new thread is added/deleted; just a plain text file, similar or a copy of the original. Date: Wed, 9 Jul 2014 18:27:04 -0700 From: notifications@github.com To: ChanThreadWatch@noreply.github.com Subject: Re: [ChanThreadWatch] Threads.txt backup (#19)

I'm sorry to hear that. The fact that it wipes the thread list is really weird. I will work on that thread list backup option. How do you think I should make such a feature? Multiple backups, custom time interval, auto-pruning? Any suggestions?

— Reply to this email directly or view it on GitHub.

fJo81VzbYs2laS9o commented 10 years ago

Yeah it kind of sucks to have 50+ threads from a few websites wiped. It is what it is. But maybe my case is isolated since jwshields has never had a crash! I'm pretty jealous about that.

Anyhow, like jwshields says, backing up threads.txt to threads.txt.bak is simple enough I would think. However since I've lost my threads list a few times I personally prefer something more robust like with your suggestions " Multiple backups, custom time interval, auto-pruning" <---- yes to all those?

I kind of want to see those, but on the flip side I don't want to see CTW get bloated and increase in cpu/io usage.

SuperGouge commented 10 years ago

This feature will be in the next release. When it comes out, let me now if the backup is also corrupted or if you'd really want more options for this feature.