[Bug]: RAM 99%, DB no saving data and not possible to open cfg

CyberP1nk commented 4 months ago

This issue respects the following points:

[X] This is a bug, not a question or a configuration issue; Please visit our forum first to troubleshoot with volunteers, before creating a report. The links can be found here.
[X] This issue is not already reported on GitHub (I've searched it).
[X] I'm using an up to date version of OpenBullet2. We generally do not support previous older versions. If possible, please update to the latest version before opening an issue.
[X] I agree to follow OpenBullet's Code of Conduct.
[X] This report addresses only a single issue; If you encounter multiple issues, kindly create separate reports for each one.

Description of the bug

After running in ob job task for several days ram goes higher until rdp not respond, using custom plugins. And if You visit and try to open job in high ram, getting unhanded exception check crash log, also hits not save data. crash log data Unhandled exception thrown on 7/9/2024 10:28:54 AM System.InvalidOperationException: Collection was modified; enumeration operation may not execute. at System.Collections.Generic.List1.Enumerator.MoveNextRare() at System.Linq.Enumerable.WhereSelectListIterator2.MoveNext() at System.Collections.Generic.List1..ctor(IEnumerable1 collection) at OpenBullet2.Native.ViewModels.MultiRunJobViewerViewModel.UpdateHitsCollection() at OpenBullet2.Native.ViewModels.MultiRunJobViewerViewModel..ctor(MultiRunJobViewModel jobVM) at OpenBullet2.Native.Views.Pages.MultiRunJobViewer.BindViewModel(MultiRunJobViewModel jobVM) at OpenBullet2.Native.MainWindow.DisplayJob(JobViewModel jobVM) at System.Windows.RoutedEventArgs.InvokeHandler(Delegate handler, Object target) at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised) at System.Windows.UIElement.RaiseEventImpl(DependencyObject sender, RoutedEventArgs args) at System.Windows.UIElement.RaiseTrustedEvent(RoutedEventArgs args) at System.Windows.Input.InputManager.ProcessStagingArea() at System.Windows.Interop.HwndMouseInputProvider.ReportInput(IntPtr hwnd, InputMode mode, Int32 timestamp, RawMouseActions actions, Int32 x, Int32 y, Int32 wheel) at System.Windows.Interop.HwndMouseInputProvider.FilterMessage(IntPtr hwnd, WindowMessage msg, IntPtr wParam, IntPtr lParam, Boolean& handled) at System.Windows.Interop.HwndSource.InputFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled) at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs) at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)

Reproduction steps

Start ob native/web.
Load big amount of word list
start wait til ram goes up (not sure if it plugin or ob, since using windows server) :(

What is the current bug behavior?

Losing hits, ram overload can crash rdp

What is the expected correct behavior?

Add feature to disable HITS/CUSTOM/TO CHECK data tab, so possible to save only in txt, the thing if hits/custom/to check is more then expected, ram is going high, as You did option "Enable logging for for bots in MultiRunJob" While testing if not use db for saving, ram is stable, will check for few days

Version of the client

0.3 and older version same problem

Type of client

Native client

Environment

- OS:Windows Server 2022
- Virtualization:
- Browser:native client

OpenBullet2 logs

emmpty

Client / Browser logs

No response

Relevant screenshots or videos

No response

Relevant LoliCode if needed

No response

Additional information

No response

openbullet commented 4 months ago

You can remove the "Database" hit output and only leave the "File system" hit output in your job options, so that it will not save hits to the database but only to file.

Also, a few more questions please

how would you define "big amount of word list"? How many wordlists and how many lines?
do you get a huge number of hits? how many per minute?
how many bots do you use?
what's the CPM?

CyberP1nk commented 4 months ago

You can remove the "Database" hit output and only leave the "File system" hit output in your job options, so that it will not save hits to the database but only to file.

Also, a few more questions please
1. how would you define "big amount of word list"? How many wordlists and how many lines?

2. do you get a huge number of hits? how many per minute?

3. how many bots do you use?

4. what's the CPM?

1.2,500000 lines

40-90k + custom 100-700k
200
1200-2000

openbullet commented 4 months ago

40-90k + custom 100-700k

You're getting that many hits per minute? How is it possible with 1200-2000 CPM? Anyways, OB2 is not built to handle this kind of throughput. I suggest you:

disable the database hit output and only use the file system one
decrease the number of bots and see if you can get the same CPM with half the bots or so

Please let me know if it's better.

I agree that the native client has a problem of concurrency where the collection of hits can be modified while it's being enumerated, and I will address that.

CyberP1nk commented 4 months ago

40-90k + custom 100-700k

You're getting that many hits per minute? How is it possible with 1200-2000 CPM? Anyways, OB2 is not built to handle this kind of throughput. I suggest you:
* disable the database hit output and only use the file system one

* decrease the number of bots and see if you can get the same CPM with half the bots or so
Please let me know if it's better.

I have in mind to change the writes of hits to the database by using raw queries, so I will leave this issue open to track this.

Yea, this point is solve my issue "disable the database hit output and only use the file system one"

openbullet / OpenBullet2