Auto Substitution on page load causes some functions of a webpage to break, while manual start of the substittution works

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Create four filter rules (in one group) for a webpage like this:
On Tab "URLs" Add Urls:
http://www.kwick.de/forum/topic/
www.kwick.de/forum/topic/

I just added my exported rules. Alternatively here is the description:

On Tab "Substitution" add the following rules (all with match case enabled and 
of type Regular Expression):

Rule 1:
Purpose: insert a fake Metatag in front of each <tr> entry which will be 
deleted using Rule 3 and Rule 4

Regexp:
(<tr *itemprop="comment"[^>]*>[\s\S]*?)(</tr|<meta *itemprop="creator" 
content="([^"]*))

Substitution:
<meta type="TROLLCHECK" istroll="no" creator="$3"/>$1$2

Rule 2:
Purpose: "set" the attribute istroll to true if a special user name appears 
(the example uses just a random user name no troll, but a quite productive one 
which is good for testing purposes):

Regexp:
(<meta type="TROLLCHECK" istroll=)[^>]*"Tesla_Trooper"

Substitution:
$1"yes"

Rule 3:
Purpose: Delete Troll postings (the thr-entries below the fake-metatags with 
istroll="yes")

Regexp:
<meta type="TROLLCHECK" istroll="yes"[\s\S]*?</tr>

Substitution: (empty)

Rule 4:
Purpose: remove the fake metatags for all all other tr entries

Regexp:
<meta type="TROLLCHECK"[^>]*>

Substitution (empty)

2. Now open the following page
http://www.kwick.de/forum/topic/437293/wissenschaftliche-fragen/

3. Navigate to a page on which the user Tesla_Trooper posted something (you can 
see it on the left) using the index buttons on the bottom of the page. At the 
moment there are two postings of this user on page 2 they are from around 25. 
june .2013, 22:27. As new articles are written they might move of course. But 
any article of this user will do. You can of course also alter rule 2 to mark 
any other users postings.

4. While autoreplace is switched off, start the replacement manually when you 
are on a page with postings of this user.

5. The Articles of this user should dissapear. Press on the index buttons to 
switch to another page. They should work.

6. Reload the page, to make the articles "visible" again. Now switch on 
autoreplace. Set the cursor in the adress bar of the browser and hit enter. 

7. Check if the navigation buttons work and if the postings of the user were 
removen. The removal works this way, but very often (but not quite always) the 
navigation buttons on the bottom of the page (i mean the links 1, 2, 3 --> 
which can be used to select a page to jump to) won't work anymore.
I never experienced this behaviour when I started the substitution manually.

What is the expected output? What do you see instead?

1. The substitution should always work (but this is also what happens, when the 
adress is entered in the adress bar).

2. The page functions should still work (navigation links), but in some cases 
they don't work anymore.

What version of the product are you using? On what operating system?

I don't know which version of FoxReplace I am using, but I just installed it 
during the last weekend.

I am using Ubuntu Linux (64 bit version) and Firefox 21

Please provide any additional information below.

Original issue reported on code.google.com by jelz.the...@gmail.com on 26 Jun 2013 at 9:30

Attachments:

• FoxReplace.json

[GoogleCodeExporter]

I just tested it once more. With the same effect (links are not working 
anymore). I recognized that the links are indeed there. At least I can use the 
"copy link" function (I use a german version of firefox, maybe it is named 
differently) and paste the link in an editor to verify it and it is really the 
correct address.

Btw. the substitution is not done automatically when you click on such a link 
(e.g. if you open the page without having autoreplace switched on, then switch 
it on and click on a (index-)link on the bottom, the replacement is not 
performed, ony when you enter the address in the addressbar. But thats another 
toppic I guess.

Original comment by jelz.the...@gmail.com on 26 Jun 2013 at 10:04

[GoogleCodeExporter]

Hi, I can reproduce your problem. It seems that the page is not fully reloaded 
when you click in the pagination links. For example, if you select the text 
"Meet new people" in the left or in the top yellow bar (I don't know if it 
appears when logged in) and then navigate to another page, the text keeps 
selected, when after a full load it shouldn't. So there may be some kind of 
strange ajax going on, because the links look like normal links, as you say.

Original comment by marc.r...@gmail.com on 27 Jun 2013 at 9:01

[GoogleCodeExporter]

I need to clarify, I can reproduce the problem about auto-replace not working, 
but the pagination links have worked every time.

Original comment by marc.r...@gmail.com on 27 Jun 2013 at 9:41

[GoogleCodeExporter]

Hi Marc, thank you for the analysis. You mean, the pagination links work, when 
the two options are not checked (link- and script replacement), right?
Yes, thats what I also experienced. Btw. the problem with the broken links 
seems only to happen, when Replace scripts is checked (with just replace URLs 
it also seems to work).
Is the missing auto-replace also page specific? Isn't there always a 
Page-Loaded event if a new page is displayed in the browse?

Original comment by jelz.the...@gmail.com on 28 Jun 2013 at 8:10

[GoogleCodeExporter]

Yes, I had both options unchecked. If I check "replace scripts" I can reproduce 
the problem of the links not working. There is a JavaScript error logged in the 
error console in that case. I suppose that it is the same problem as in issue 
81 or a similar one.

Autoreplace doesn't work because the page is not fully loaded, only a fragment 
is loaded with AJAX. This can be seen in the web console (Firefox or Tools menu 
-> Web developer -> Web console, or Ctrl+Shift+K):

1. Load any page of the forum
2. Push the Clean button in the console
3. Switch to another page using one of the links at the bottom.
4. In the console you will see the GET petition with a link like this: 
http://www.kwick.de/forum/topic/437293/wissenschaftliche-fragen/5/?__env=ajax&_=
1372408596938

If you follow the link you can see that the response is a JSON object with the 
new content of the page, which is then inserted in place of the previous page 
in the DOM tree.

So, the problem with autoreplace is really the same as in issue 13.

Original comment by marc.r...@gmail.com on 28 Jun 2013 at 8:51

[GoogleCodeExporter]

Hi Marc,

thank you very much for analyzing the problem. I just checked like you describe 
it, you are right. But why do the links in the page source look like ordinary 
links? is the html code replaced and the orignal version is just there for 
compatibility reasons (with browsers that dont support some functions)?
I guess thats no good news for me, is it?

Do you plan to include a solution for ajax? I guess then either the regular 
expressions would have to be applied either just to the loaded fragment, or to 
the source of the whole page after it has been altered, or does the browser not 
really update the source code of the page it is uploading unless the user opens 
the page source view?

J.

Original comment by jelz.the...@gmail.com on 28 Jun 2013 at 8:22

[GoogleCodeExporter]

I suppose the links are there for when JavaScript is disabled. I suppose it is 
somehow intercepted when a link is clicked and then it is converted to the 
other link, but I don't know this is done (I'm not an expert on AJAX).

I would like to include a solution for AJAX (issue 13), but I don't know how to 
solve it for now.

Original comment by marc.r...@gmail.com on 29 Jun 2013 at 2:16

[GoogleCodeExporter]

I found the same problem with a page that is constantly running the linux 
"tail" command. ie auto replace does not work but manual does. Unfortunately it 
is a password protected page so I cannot share it here.

Original comment by sqgl2...@gmail.com on 24 Sep 2013 at 12:22