lag437289 msg every 30 seconds

[ask2018]

Since I've updated to dlFilter 2.04 I'm getting "Private: -myickname- lag437289 somenumber" every 30 seconds. It was not there with old version 1.16 and its very annoying, because it basically spams me more than it filter the spam. Any idea what can cause this and how to get rid of it?

[Sophist-UK]

Is there a user lag437289 on any of the servers you are connected to?

If you are willing to say, which network / server is this happening on?

Which window is that appearing in? If it is a channel, is lag437289 in that channel also? If you are willing can you say which channel it is in?

(From the limited information you have provided it seems that the current version is not filtering a private message being sent to you by another user, whereas the previous version was filtering this by default. I have no idea why this user is sending you these frequent messages. If you can provide the above information I will look into it. Either way, you should be able to set a custom filter to filter it out.)

[ask2018]

There is no such user on any servers I'm connected to. It's more like some internal notice and it's posted on the current active window - doesn't matter if that's some channel or query window and doesn't matter on what irc server. All the time just the active window.

I'm connected on more mostly private irc servers and I don't think this is somehow related that somebody is sending that to me, because if I disable dlFilter it's gone. Must be some internal message which is dlFilter causing. I'm running MIRC 7.52 with nbs-irc - https://github.com/ElectronicWar/nbs-irc and I'm connected through ZNC 1.6.5

But trying to google that and it seems to be something inside nbs-irc code. So probably some incompatibility between nbs-irc and new dlFilter version Look here https://github.com/ElectronicWar/nbs-irc/blob/master/scripts/main.nbs lines 848, 890 and 1327 So the question is now if you can somehow make this compatible with nbs-irc again or can suggest me some workaround.

[ask2018]

So ticking off "Enable automatic lag check" inside nbs-irc settings fix that. Problem solved, but still not sure why this was not happening with the old 1.16 version of dlFilter.

[Sophist-UK]

I am not quite sure what the reason for this happening in v2 but not v1 is - I can think of a few possible explanations:

1. dlF tries to ensure that it runs before other mIRC scripts - dlF v1 also tried to do that but it is possible that it ran after the nbs-irc code.

2. dlF v2 does something different from v1 for notices that come from self. This is the most likely explanation - that we do something with it in v2 (i.e. echo it to the active window whereas previous we did nothing and it got processed and hidden by nbs-irc.

So - how to fix it:

a. We could change to run dlF last rather than first - not sure what impact this would have if you are also running sbClient, because dlF does stuff to make sbClient run better. We also try to run first in order to filter out the spam before another script processes it - but perhaps we shouldn't do this either. This needs careful thought before we take a decision to make dlF run last instead of first.

b. For this specific nbs-irc case, we can include a filter for it in dlF - or you can add a custom Private Notice filter for "lag437289 *" - but this is an example of a more general case of other scripts processing stuff, so we should probably try to think more widely and have a generic solution (which might be option a. above).

[Sophist-UK]

@DukeLupus Can you give your thoughts on this pls?

[Sophist-UK]

@Dukelupus - the more I think about this the more I think dlFilter should run last.

The purpose of dlFilter is to filter stuff from the mIRC UI not from other scripts. Other scripts tend to look for specific stuff - DLF runs on practically every message.

[Sophist-UK]

I have created a PR #59 to run DLF as the last script rather than the first and so avoid the symptom reported in this issue.

@ask2018 - If you could download this version from https://raw.githubusercontent.com/DukeLupus/dlFilter/62859f7537f907e4ee17b6d3dabfeff2ff94953f/dlFilter.mrc and see if it resolves your issue that would be appreciated.

[ask2018]

@Sophist-UK It doesn't resolve the issue and side effect is dlFilter filtering doesn't work at all in my client with nbs-irc.

[SanderSade]

I remember there were major issues when dlF was loaded to "wherever". I think that caused Invision (or something else) to spit our all messages to channels.

[Sophist-UK]

Well - that is why I did it as a PR and not a commit to the master branch.

If a previous script has halted and event, AFAICT mIRC is supposed to still call the matchig events in later scripts unless they use the "&" prefix, and you can see if they were halted using the $halted identifier.

If $halted is true, then mIRC is not going to display the text on screen - the PR is supposed to respect that the display has been stopped (rather than decide to echo it itself) and still process the message (which could be an advert), but also log it to the Filter window so it is not lost completely.

If dlF is not working at all with nbs-irc, the most obvious guess is that the events are not being called as they should. But it is equally possible that there is something in DLF which is failing to handle them.

[Sophist-UK]

@ask2018 Can you type /DLF.Watch and post a few examples of the debugging output here.

[ask2018]

@Sophist-UK Sure. Log is untouched, just nick, irc, channel etc is censored: It is with "Enable automatic lag check" enabled inside nbs-irc.

(13:06:36)+000 -> irc.server PRIVMSG #channel :testing
(13:07:09)+000 -> irc.server PRIVMSG #channel :123 test
(13:07:16)+000 -> irc.server NOTICE nickname :lag437289 6039079
(13:07:16)+000 <- @time=2018-07-20T11:07:21.554Z :nickname!at@ip NOTICE nickname :lag437289 6039079
(13:07:16)+032 571 ON NOTICE called DLF.Priv.Notice: lag437289 6039079
(13:07:16)+047 571 ON NOTICE called DLF.@find.Response
(13:07:16)+063 571 ON NOTICE called DLF.@find.IsResponse
(13:07:16)+078 571 ON NOTICE called DLF.Custom.Filter
(13:07:16)+110 571 ON NOTICE called DLF.Priv.RegularUser
(13:07:16)+125 571 Not regular user: Me
(13:07:16)+141 571 Not a regular user
(13:07:16)+156 571 DLF.Win.Echo notice Private nickname lag437289 6039079
(13:07:16)+188 571 Echoed: To status window because active window is custom / listbox

And version 2.05 filtering works fine, just there is that annoying every 30s msg if I dont disable "Enable automatic lag check" inside nbs-irc. Only the version with "run DLF as the last script" you posted above, doesn't filter anything with nbs-irc script installed and running.

[Sophist-UK]

Unfortunately this is not a helpful example, because the messages are from yourself. But it does show that at least IRC notices are being processed by DLF.

Any chance of some real-life examples in a real life channel? (Please feel free to obfuscate the actual channel name / or specific text of anything that you don't wish to disclose.)

[Sophist-UK]

P.S. I have made a tweak to the Watch functionality so please use the updated version from https://raw.githubusercontent.com/DukeLupus/dlFilter/04bbc51023ad58eb93de7f5de6199ee1d0af3369/dlFilter.mrc

[ask2018]

Yes, but please specify what exactly you need from the log, because from real life channels there is really lot of output in that window and really lot of text etc I need to obfuscate (I'm not filtering ads, but content). Also specify if you need it with the "Enable automatic lag check" in nbs-irc enabled (even more text) or disabled and specify if you need it from the broken version with "run DLF as the last script" or from the working version 2.05

[Sophist-UK]

I need it from the version last linked to.

I don't need many examples, but it needs to be a channel text message from other users.

And an example of "automatic lag check" would also be useful.

[ask2018]

ok so here is it First of all there is this inside regular filtering window for all channels and thats the main problem probably:

(14:36:41) [#channelname] * Already halted:

this is the lag msg problem:

14(147:377:4814)+000 -> irc.server NOTICE mynick :lag437289 11471689 14(147:377:4914)+000 <- @time=2018-07-20T12:37:53.899Z :mynick!at@ip NOTICE mynick :lag437289 11471689 14(147:377:4914)+016 1001 ON NOTICE called DLF.Priv.Notice: lag437289 11471689 14(147:377:4914)+031 1001 ON NOTICE called DLF.@find.Response 14(147:377:4914)+031 1001 ON NOTICE called DLF.@find.IsResponse 14(147:377:4914)+047 1001 ON NOTICE called DLF.Custom.Filter 14(147:377:4914)+047 1001 ON NOTICE called DLF.Priv.RegularUser 14(147:377:4914)+062 1001 Not regular user: Me 14(147:377:4914)+062 1001 Not a regular user 14(147:377:4914)+062 1001 DLF.Win.Echo notice Private mynick lag437289 11471689 14(147:377:4914)+078 1001 Echoed: To status window because active window is custom / listbox

and this is some text from other user from channel where filtering is enabled: but with this version of dlFilter the text filtering is not working and it shows in both - main channel windows and also filtering window

14(147:487:3814)+000 <- @time=2018-07-20T12:48:43.436Z :otheruser@ip PRIVMSG #channel1 : {b}{k}12{194}{183}{k}{b} 14(147:487:3914)+000 <- @time=2018-07-20T12:48:44.442Z :otheruser2@ip PRIVMSG #channel2 :{k}14[{k}10{b}{b}URL{k}14{b}{b}] this.text.should.be.filteredk}10{b}{b} another.text.should.be.filtered{o} {b}{k}12{194}{183}{k}{b} 14(147:487:3914)+047 843 ON TEXT called DLF.Chan.IsChanEvent 14(147:487:3914)+047 843 ON TEXT called DLF.Stats.Count: #channel2 Total 14(147:487:3914)+062 843 ON TEXT called DLF.Chan.Text 14(147:487:3914)+062 843 ON TEXT called DLF.Custom.Filter 14(147:487:3914)+078 843 Matched in custfilt.chantext by "custom.filter.pattern" 14(147:487:3914)+078 843 Matched in custom.chantext: custom.filter.pattern 14(147:487:3914)+078 843 ON TEXT called DLF.Win.Log: Filter text #channel2 otherusernick 14(147:487:3914)+093 843 ON TEXT called DLF.Stats.Count: #channel2 Filter 14(147:487:3914)+109 843 Filtered: To @dlF.Filter.network text #channel2 otherusernick

[Sophist-UK]

1. I decided we should send halted messages to the filter window on the basis that another script has effectively filtered it already. In my mind that is the purpose of the filter window - to be a record of what has been filtered. If you don't like this, please explain why and I can reconsider.

2. I had forgotten that we echo the text ourselves in some cases and then halt - I will add code to handle that situation which will stop it being displayed in channel.

[Sophist-UK]

Please download and use the revised version for testing from https://raw.githubusercontent.com/DukeLupus/dlFilter/27b22226d826a39d73ce57860a5cd29832a7db3a/dlFilter.mrc

[Sophist-UK]

If it still doesn't work please give examples of both channel / status window output, matching filter window output and matching Watch window output.

[ask2018]

1. Those "[#channelname] * Already halted:" msges was not in the filter window in previous version. Problem with them is it opens filter windows on all irc networks I have. And thats additional unnacesary opened windows, because I don't filter channels on all networks I'm connected. So filter windows are needed only on networks, where filtered channels exists.

2. Last posted version sends the lag437289 msg to filter windows as " -nickname- Already halted: lag437289 43686270" Still not sure why is that msg there, but if thats filtered, it can be considered as fixed I guess.

3. But the main problem with the new version is still there. And that's filtering simply doesn't work. Text which should be filtered is sent correctly to filter window, but it remains also in main channel window as no filter exists.

This is from watch window:

14(237:407:4214)+000 <- @time=2018-07-20T21:40:47.201Z :usernick@ip PRIVMSG #channel :{k}14[{k}{b}{b}text{k}14{b}{b}] text.with.pattern{k}8{b}{b} text{o} {b}{k}12{194}{183}{k}{b} 14(237:407:4314)+031 4152 ON TEXT called DLF.Chan.IsChanEvent 14(237:407:4314)+047 4152 ON TEXT called DLF.Stats.Count: #channel Total 14(237:407:4314)+063 4152 ON TEXT called DLF.Chan.Text 14(237:407:4314)+078 4152 ON TEXT called DLF.Custom.Filter 14(237:407:4314)+078 4152 Matched in custfilt.chantext by "pattern" 14(237:407:4314)+094 4152 Matched in custom.chantext: pattern 14(237:407:4314)+109 4152 ON TEXT called DLF.Win.Log: Filter text #channel nickname 14(237:407:4314)+125 4152 ON TEXT called DLF.Stats.Count: #channel Filter 14(237:407:4314)+141 4152 Filtered: To @dlF.Filter.network text #channel nickname

this is posted msg in main channel window, which should be filtered, but it's not:

(23:40:47) (@nickname) text.with.pattern ·

this is the same in filter window:

(23:40:43) [#channel] text.with.pattern ·

Notice that time. It's posted in channel window 4 seconds after filter window Also text which should not be filtered is sent also to filter window with prefix " Already halted:".

Maybe the problem is nbs-irc is already somehow processing the text and if you don't load dlFilter before it, then it doesn't work.

[Sophist-UK]

Sorry that this is taking several iterations to get right - but we are getting closer.

1. No they weren't previously filtered with "Already halted" because DLF ran first and so other scripts could not have halted them before DLF processed them. DLF either filtered them or explicitly echoed them or left them for other scripts to process. But since DLF echoed and halted some of them, it was thus impossible for later scripts to halt and stop them being displayed. So DLF really does need to run last.

2. On reflection I think we should only send previously halted messages to the Watch window.

3. We still need to track down what is happening because I have no idea where the channel message is coming from. Could the 4 second delay actually be a different message? Also can you confirm whether your own nick or the nicks sending the message are in your mIRC notify list?

4. Acting on non-DLF channels is a bug and I will fix it. I will let you know when the next version is ready for testing as I have other stuff to do for a while.

[Sophist-UK]

Please retest using https://raw.githubusercontent.com/DukeLupus/dlFilter/7b8ad85ca0e689ffadcc3c9e7dfe771811de31eb/dlFilter.mrc

[ask2018]

1. "[#channelname] * Already halted:" are not displayed anymore in filter window. So additional unnacesary opened windows problem is fixed.

2. lag437289 msges are not displayed anymore in filter window or anywhere else. That's also better now, so also fixed.

3. Only problem that remains is filtered msges are still displayed in both windows - main channel window and filter window.

My mIRC notify list is empty and notify is not even enabled in the settings. The 4 second delay was the same msg and there is some delay for all msges in few seconds. It seems to be in +-5 seconds range mostly.

[Sophist-UK]

1. Yes - now sent only to the Watch window if open.

I cannot understand why there would be a lag of several seconds though unless you are running on a fairly low powered machine and DLF and other scripts are making mIRC run at 100% of a CPU core (I don't think mIRC is multi-threaded). Please provide another example with all Watch messages from the original message through until after the time that the message appears in the channel window.

[ask2018]

Also no idea why there is the delay, but it's not cpu issue. The cpu load is around 5-10% included all other processes. Here is the log:

filter window

(14:34:07) [#channel] <@usernick> text text.with.filtered.pattern text ·

main channel window

(14:34:09) (@usernick) text text.with.filtered.pattern text ·

watch window

14(147:347:0614)+000 <- @time=2018-07-22T12:34:08.611Z :usernick!~usernick@ip PRIVMSG #channel :{k}14[{k}{b}{b}text{k}14{b}{b}] text.with.filtered.pattern{k}8{b}{b} text{o} {b}{k}12{194}{183}{k}{b} 14(147:347:0614)+046 683 ON TEXT called DLF.Chan.IsChanEvent: usernick/usernick: 14(147:347:0614)+062 683 Stats: Total 6, Filtered 5: #channel Total 14(147:347:0614)+078 683 Is DLF channel event: usernick in #channel 14(147:347:0614)+093 683 ON TEXT called DLF.Chan.Text: : {k}14[{k}{b}{b}text{k}14{b}{b}] text.with.filtered.pattern{k}8{b}{b} text{o} {b}{k}12{183}{k}{b} 14(147:347:0614)+124 683 Filtered: Already halted by previous script: {k}14[{k}{b}{b}text{k}14{b}{b}] text.with.filtered.pattern{k}8{b}{b} text{o} {b}{k}12{183}{k}{b} 14(147:347:0614)+156 683 ON TEXT called DLF.Custom.Filter: : chantext {k}14[{k}{b}{b}text{k}14{b}{b}] text.with.filtered.pattern{k}8{b}{b} text{o} {b}{k}12{183}{k}{b} 14(147:347:0614)+187 683 Matched in custfilt.chantext by "pattern" 14(147:347:0614)+202 683 Matched in custom.chantext: pattern 14(147:347:0614)+218 683 ON TEXT called DLF.Win.Log: Filter text #channel usernick: {k}14[{k}{b}{b}text{k}14{b}{b}] text.with.filtered.pattern{k}8{b}{b} text{o} {b}{k}12{183}{k}{b} 14(147:347:0614)+249 683 Stats: Total 6, Filtered 6: #channel Filter 14(147:347:0614)+265 683 Filtered: To @dlF.Filter.network text #channel usernick

[Sophist-UK]

I cannot understand where this lag is coming from - unless one of the other scripts is halting the message and using a timer to delay showing the message.

[ask2018]

No idea and only scripts I have here is the already mentioned nbs-irc and then fish 10 from here: https://syndicode.org/fish_10/

[Sophist-UK]

A quick read through of the nbs-irc script shows that it intercepts and halts every channel text message in every channel (see https://github.com/ElectronicWar/nbs-irc/blob/b64f76a02ad59d8b961faa0ef54f3cbf5b198236/scripts/main.nbs#L4225) and then haltdef's it - in other words every line is halted - and then echos it out later - in other words, I would rate nbs-irc as being very poorly written because it is indiscriminate in what it does.

[Sophist-UK]

The bottom line is that DLF and nbs-irc both execute on every mIRC event, and two scripts which both halt the message and then echo it instead of letting mIRC display it are effectively incompatible.

DLF only halts messages that it is displaying in the non-default window. nbs-irc halts everything. DLF deliberately runs last (as per this PR) because it does this, nbs-irc does not consider the consequences and does not position itself cf. other scripts.

In summary, DLF and nbs-irc (as it stands today) are fundamentally technically incompatible. If nbs.irc stops halting everything, it can be made more compatible.

[ask2018]

I know nbs-irc is not ideal, but I've found it as closest replacement for NoNameScript which was no more compatible with latest mirc and there is no other alternative as far as I know. Those few seconds lags are not much problem for me and filtering is working with DLF version 2.05. So my question is now can I just change the loading order back to first and it will work again for me as version 2.05 or the other changes you did are not compatible with load as first as before? And then I guess we can close this issue. Thank you for support.

[Sophist-UK]

If you download and switch back to the version from the master branch, it will load first.

However I am still pondering whether to merge the PR that changes it to load last - because I think that may be the most appropriate approach overall.

My advice to you would be to get nbs-irc fixed - perhaps submit the necessary changes yourself.

(I will raise an issue on their Github repo though.)

[Sophist-UK]

Issue raised on nbs-irc: https://github.com/ElectronicWar/nbs-irc/issues/17 though unsure that this is actively maintained and someone will address it.

My current plan will be to merge the PR except for the bit that moves the script to run last. So for the moment it will continue to run first, but will contain all the other code necessary to handle the situation where another script halts messages first.

I may also add a global option to allow the user to decide whether the script should run first or last, with running first being the default.

[Sophist-UK]

I have now merged #59 and included an option to allow the user to decide whether to run it first or last. I have a couple of quiet days which I can use to code dlF stuff (for other issues) and then I intend to bump the version to 2.07 so updates will be offered to users.

[Sophist-UK]

I am now closing this issue as AFAIK there is no more to be done to fix it - fixes being needed in https://github.com/ElectronicWar/nbs-irc/ to make that script more compatible with other scripts like dlF which process and halt lots of messages.