Subscriptions screen freezing - # 2397 re-visited

[hglecke]

After receiving B24 and installing it, I repeated the multiple delete process as described in #2397. It took a while but, unfortunately, finally the screen failed to populate with data. I tried to display previously displayed screens, they too failed. Clicking the yellow triangle had no response. Clearly, a lot of work went into B24. The glitch described here is not an urgent one, as I pointed out when I closed #2397. Ag ain, Thank you for all the effort put into this.

[parg]

Any record of the alerts in either the alerts_1.log, alerts_2.log, debug_1.log or debug_2.log files in the "logs" directory in %APPDATA%\BiglyBT ?

[hglecke]

alerts_1.log alerts_2.log

Lots of 'stuff'. I noticed my install drive (C:) is filling up. Long ago there was a similar problem, solved successfully. This is different in that restarting, closing and loading BiglyBT, re-booting the system did not eliminate the accumulated temp files. I found the 'use/(Name)/appdata/roaming/BiglyBT/torrents/MD directory filled with large number of sub-directories, containing 2 sizable files. The MD directory was over 49GB. Two thirds of the creation dates were older than I would have expected. Related to the display challenges?? I do not know. I did not notice this filling of the drive prior to B24, not saying it started with B24. Sorry to add to the challenge. However, if the system drive runs out of space, the consequences can be severe.

[parg]

The MD directory folders normally correspond to active magnet link downloads - they should be removed when the magnet download is complete. It could be something to do with the issues you are having that is resulting in stuff being left there, can you see if you can figure out under what conditions things aren't being deleted correctly?

[hglecke]

Difficult to determine. Names are not in clear text. Total number of entries do not seem to correlate to active downloads. Dates are little help. Under 'added' in download: oldest Dec 10, newest Jan 12. In MD entries oldest Nov 28, newest Jan 13. Periodically I manually delete from the active downloads screen any entries too old and without activity. Again, my uneducated gut tells me this issue is not (directly) related to B24. The above listed dates would indicate that. Wish I could be of more help with that.

[parg]

I'll look into adding something to remove dead entries anyway

[hglecke]

Sounds like a perfectly good 'catch all' precautionary measure.

[hglecke]

A stab in the dark: Selecting a subscription entry for download adds one to the entry count in MD. Deleting (remove from computer) a "Downloading" entry in library view (because of unwanted file type, for example) did not reduce the entry count in MD. Could be the orphan entry?

[parg]

For me that deletes the entry in the MD folder :(

[parg]

I've been trying to reproduce your "no more handles" error but can't seem to do it.

Can you describe exactly how you manage to create it? For example,

select a Subscription in the sidebar, delete some entries from it via the toolbar/delete-key/right-click menu select another Subscription, do the same

after deleting 1000 results the problem occurs or after switching through 50 Subscriptions the problem occurs

etc.

Hopefully you can produce the problem with some limited set of actions so I can try and do the same!

[hglecke]

FYI The next update to BiglyBT just arrived. I did not load it yet to avoid distorting the indications. In the description of the 'revisited' problem I mentioned that I did get the yellow triangle but cold not open the message, clicking the triangle was without response. The md file size question:

New Text Document.txt

I hope that allows you to replicate the steps I took to see the unchanged number of entries.

I will go now and update BiglyBT with the latest.

[hglecke]

B25 is loaded! The md file was cut way back, from 731 to 35 items. Good job !!!

[parg]

Woo!

The 'md' folder entries (should) only exist while a download is in the "metadata download" state (i.e. using a magnet link to download the .torrent file before commencing the actual download). These are visible in the library with a name prefixed by "Metadata download for " - it is only these that should result in the "md" entry being removed if you delete them. Once they have turned into "real" downloads the "md" entry should be automatically removed.

[hglecke]

OOPS - once again I seem to have boxed myself into a corner by deleting actual downloads, not metadata downloads. That's why you are doing the programming and I do the screwing up. It seems that we have two separate topics intermingled. Your fix in B25 for the orphan entries is effective.

I did manage to freeze the subscription screen, again, even with B25. Here is what I did: Start BiglyBT and start the downloads, as usual. Once that got going I opened one subscription with 'new' entry(ies). Chose for download the ones I wanted and marked the remaining 'new' entries as read. After going through all the subscriptions with 'new' entries I started, one subscription at a time, to delete those entries not marked as 'in library' or 'in history', etc., until no more blanks in the 'in' column were left. I wanted fewer entries to go through, originally trying to combat the GUI errors. During that process of going from subscription to subscription, at no particular entry or number, the screen was painted without the data and locked. Any clicking on that screen, including the yellow triangle, was without response. If I restrained myself and stopped clicking at random and clicked on 'Exit' in the File pull-down, I usually could exit BiglyBT and re-start it normally. If I got carried away with my clicking, the entire BiglyBT could become unresponsive. Only the Ctrl-Alt-Del emergency break would help. This is, as I said before, no longer a major challenge as it requires quite a few specific conditions and user interactions to be triggered. Over the past few beta releases, BiglyBT has become notably more stable, thanks to diligent programming efforts. For that we, the users, owe a big THANK YOU.

[hglecke]

One more question! Currently I show 36 active metadata downloads and 53 items in the md file. Is that an indicator of orphan entries?

[parg]

Not sure - when you restart BiglyBT and there are lots of "metadata downloads" active then to limit resources only there is a maximum number that can be recovered at any given time (config under plugins->Magnet URI Handler). So if you had 100 metadata downloads and the recovery concurrency was, say, 10, then only the first 10 would be added to BiglyBT and the other 90 queued waiting for some of the first 10 to complete. i.e. the "md" folder would have 100 entries but you would only see 10 metadata downloads.

I should rework this some day...

[parg]

Regarding the GUI hang - do you have a rough estimate of the number of different subscriptions you switched through and the total number of entries you deleted? (and how did you delete the entries? delete key, toolbar 'x' icon etc?)

[parg]

B26 should recover metadata downloads better on restart and use less resources to do so.

[hglecke]

Thanks - have installed B26. Will re-run tests I ran with B25. My response will be a little delayed.

[hglecke]

OOPS First action of B26 was to remove ALL Metadata Downloads from the Library screen. Adding a few new ones did not display those either. In the .torrent files save directory I do have a bunch of .torrent files (94) as well as .torrent.imported files (671) plus .torrent.saving files (14). Some date stamps are quite old, some from when I just added the 'invisible' ones. My guess is it needs some additional work.

[hglecke]

Spoke too soon The metadata download is actually displayed during the search for. Once the 'Operation in Progress' And disappears when either none found or an actual download is initiated. Intended?

[parg]

When things go wrong please take a look in the debug_1/2.log files to see if there are any errors there

[hglecke]

New Text Document.txt

Please point my nose to the debug_1/2.log files location.☻

[parg]

All log files are contained in %appdata%\BiglyBT\logs

[parg]

As you say, the lockup may well not be directly connected to the subscription view itself - "no more handles" can mean things other than UI resources have been exhausted. That's why I started looking into the metadata downloads - the ones that you were creating when you selected something from a subscription to download. Each metadata download was causing quite a few threads to be created (each thread using significant system resource) so I worked on reducing the number of threads required in B26.

That being said, everything else was supposed to work as before, so you losing all your existing metadata downloads on installing B26 is odd - I've done quite a bit of testing and things seem fine which is why I am hoping your debug logs show something.

[hglecke]

Located the Debug_1 & _2 files OK, in an obvious and logical place (once you know it). These files contain much info, all gibberish to me since I never used Java myself. Looking at the volume, etc., of this exchange raises the question " is this very public forum the best place for it?' I am reluctant to send the debug files, screen shots, etc. to such public display. I tried using the email reply function only to learn that it ends up in this forum as well. Some more private, maybe by invitation only, location may help to encourage would be contributors, as it would me. Currently still collecting info on the disappearance of metadata downloads in B26. Hope to send soon.

[parg]

Feel free to send me stuff via parg@biglybt.com

Thanks!

[hglecke]

After a lot of work was put into this, BiglyBT is running much quicker for me. Any GUI glitch is very rare and is tolerable. Thanks