Readability issue with url having & character

[papriwalprateek]

Reproduce the issue -

• When we try to open url like https://community.topcoder.com/tc?module=Static&d1=tutorials&d2=hungarianAlgorithm with readability. It does not show the result which it should show.
• In the url http://daqwest.com/read?ad=https://community.topcoder.com/tc?module=Static&d1=tutorials&d2=hungarianAlgorithm, there is actually only one parameter but browser splits the url parameter into 2 more parameters.

[papriwalprateek]

This is basically a two step mechanism.

Problem -- Browser splits the parameter when it encounters & character. And on the top of it even if we replace & with %26 it doesn't work because topcoder asks for & character for its own.

Solution -- First encode and then decode. So we have an url for an article.

STEP 1

We have to encode the url and then feed into the parameter.

So if the url is https://community.topcoder.com/tc?module=Static&d1=tutorials&d2=hungarianAlgorithm

then we will feed URI.encode(a,%r{[[:punct:]]}) == https%3A%2F%2Fcommunity%2Etopcoder%2Ecom%2Ftc%3Fmodule=Static%26d1=tutorials%26d2=hungarianAlgorithm as parameter.

That is, localhost:3000/read?ad=https%3A%2F%2Fcommunity%2Etopcoder%2Ecom%2Ftc%3Fmodule=Static%26d1=tutorials%26d2=hungarianAlgorithm

instead of http://localhost:3000/read?ad=https://community.topcoder.com/tc?module=Static&d1=tutorials&d2=hungarianAlgorithm

STEP 2

Now while we feed this into dqreadability, we will decode the url

source = URI.decode(a)

Separate issue

And we have to feed in broswer encoded url ... This i guess u understood

[vinayrajchoudhary]

Currently solved by splitting the request url to get the page

query_page = request.original_url.split("/read?ad=")[1]