Problem with URL encoding (Percent Encoding)

[andrewsz87]

Hi, In this step I have to capture the value wresult, but I have to decode the value why exist &lt and &quot

example "wresult" value "&ltt:RequestSecurityTokenResponse xmlns:t=&quothttp://..."

---

step: Get Check Url Posta url: {REDIRECT} parseresponsePARSEDRESULT2: input type="hidden" name="wresult" value="|"|decode

In this step how do I pass the value of wresult with the values ​​< - " correct and with url encoded?

step: Post Check Url Posta verifypositive1: {PARSEDRESULT2} url: https://server.it:443/.... postbody: wa=wsignin1.0&wresult={PARSEDRESULT2}&wctx=rm%3D0................

Thankyou

[Qarj]

Hi there, what does the wresult value "look like"?

You may not need to decode it in the first place, just post it as is

step: Get Check Url Posta
url: {REDIRECT}
parseresponseWRESULT: input type="hidden" name="wresult" value="|"|

step: Post Check Url Posta
desc: wresult: {WRESULT}
url: https://server.it:443/....
postbody: wa=wsignin1.0&wresult={WRESULT}&wctx=rm%3D0................

Also if the url that you POST to is the same / almost the same as the url you GET from, you might not need to do the parseresponse at all

step: Get Check Url Posta
url: {REDIRECT}

step: Post Check Url Posta
url: {REDIRECT}
postbody: wa=wsignin1.0&wresult={DATA}&wctx=rm%3D0................

[Qarj]

One other comment - if you need to do a url escape, replace the parameter decode in your example with escape (I'm pretty sure this is mentioned in the manual).

[andrewsz87]

wresult result is:

<input type="hidden" name="wresult" value="&ltt:RequestSecurityTokenResponse xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust&quot;>&lt;t:Lifetime>**&lt**w

i want substitution &lt and &quot in < and " before POST. and I want send POST with Percent Encoding

format correct: wa=wsignin1.0&wresult=%3Ct%3ARequestSecurityTokenRespowa=wsignin1.0&wresult=%3Ct%3ARequestSecurityTokenRespo

i have wrong format: wa=wsignin1.0&wresult=%26lt%3bt%3aRequestSecurityTokenResponse%20xmlns%3at%3d%26qu

thankyou

[Qarj]

Can you please put the snippets inside three back ticks, I think your message is being garbled due to special characters, the html snippet you posted isn't valid html so it makes it impossible for me to figure out what is going on ...

[andrewsz87]

Sorry,

'''wresult result (&lt and &quot): <input type="hidden" name="wresult" value="&ltt:RequestSecurityTokenResponse xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust&quot;>&ltt:Lifetime>&lt;w etc....

i want substitution &lt and &quot in < and " before send a POST.

I want send POST with Percent Encoding:

i have wrong format: wa=wsignin1.0&wresult=%26lt%3bt%3aRequestSecurityTokenResponse%20xmlns%3at%3d%26qu.......etc..

format correct: wa=wsignin1.0&wresult=%3Ct%3ARequestSecurityTokenRespowa=wsignin1.0&wresult=%3Ct%3ARequestSecurityTokenRespo

thankyou'''

[Qarj]

You can look here - it will explain what a backtick is https://docs.github.com/en/github/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax

You'll need to use the GitHub issues editor directly probably for it to work, just replying to the email will probably filter out important characters

[Qarj]

If you look at my first response to you, you can see an example of the backtick being used for code samples

[andrewsz87]

wresult: result (&lt and &quot):

<input type="hidden" name="wresult" value="&lt;t:RequestSecurityTokenResponse xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust&quot;>&lt;t:Lifetime>&lt;w etc....

**i want substitution &lt; and &quot; in < and " before send a POST.**

I want send POST with Percent Encoding:

i have wrong format:

wa=wsignin1.0&wresult=%26lt%3bt%3aRequestSecurityTokenResponse%20xmlns%3at%3d%26qu.......etc..

format correct:

wa=wsignin1.0&wresult=%3Ct%3ARequestSecurityTokenRespowa=wsignin1.0&wresult=%3Ct%3ARequestSecurityTokenRespo

thankyou

[Qarj]

The trouble is that the first character you grab in the parse is an ampersand & and the correct uri escape code for that is %26 which is what you are getting, if you wanted to get %3C then the first character you grab would need to be a less than sign < which obviously it isn't.

Also the html for your hidden form value is wrong - the value is <t:RequestSecurityTokenResponse xmlns:t= but after that you have http://schemas.xmlsoap.org/ws/2005/02/trust" which is invalid html at that point.

Here is an example of a valid hidden input element

<input type="hidden" id="isPostBack" value="false" />

Here is an example of an invalid input element - I've made it wrong in the same way yours is wrong

<input type="hidden" id="isPostBack" value="false"http://schemas.xmlsoap.org/ws/2005/02/trust />

You are not allowed to put a url after the closing double quote value

[Qarj]

Ok so it seems that your content is already encoded with html entities, what you are trying to do is decode the html entities and URI encode instead

[Qarj]

What you can do is parse the value twice, the first time you decode it like you are doing.

Then you echo that in a second step, this time doing another parseresponse with escape instead of decode

Finally in the third step you can use the value

[Qarj]

step: URI Escape the value
echo: "{WRESULT}"
parseresponseWRESULT_ESCAPED: "|"|escape

assuming you don't have a double quote in the string, otherwise just pick some other character to wrap the echoed result in

step: URI Escape the value
echo: !{WRESULT}!
parseresponseWRESULT_ESCAPED: !|!|escape

[andrewsz87]

Thanks it worked!

[Qarj]

Great news!

[andrewsz87]

Hi i have other question, I want its output from {WRESULT} :

wresult=%3ct%3aRequestSecurityTokenResponse%20xmlns%3at%3d%22http%3a%2f%2fschemas.xmlsoap....

but I need:

wresult=%3Ct%3ARequestSecurityTokenResponse+xmlns%3At%3D%22http%3A%2F%2Fschemas.xmlsoap...
              %3Ct%3ARequestSecurityTokenResponse%20xmlns%3At%3D%22http%3A%2F%2Fschemas.xmlsoap

I want its output as uppercase.

it's possible?

For some reason Server's OAuth requires the escaped characters to be in uppercase or else it'll give you 401 unauthorized

thank you

I RESOLVE: change 02x in 02X in wl.pl 

sub uri_escape {
    my ($_string) = @_;

    $_string =~ s/([^^A-Za-z0-9\-_.!~*'()])/ sprintf "%%%02X", ord $1 /eg;

    return $_string;
}

[andrewsz87]

Hi i have other question, I want its output from {WRESULT} :

wresult=%3Ct%3ARequestSecurityTokenResponse%20xmlns%3At%3D%22http%3A%2F%2Fschemas.xmlsoap....

but I need "+" not "%20" :

wresult=%3Ct%3ARequestSecurityTokenResponse+xmlns%3At%3D%22http%3A%2F%2Fschemas.xmlsoap...

it's possible?

[Qarj]

What you would normally do in this situation is do two parses - one to the left of the plus and one to the right of it. Then when you use it you just join them together. {WRESULT_LEFT}+{WRESULT_RIGHT}

You can convert stuff to uppercase:

step:                   Convert to uppercase
evalUPPER:              uc(q|abc|)

step:                   Uppercase [{UPPER}]
echo:                   Uppercase {UPPER}
verifypositive1:        Uppercase ABC

[Qarj]

This is also possible:

step:                   Substitute text
evalSUBBED:             sub sb { my ($_s) = @_; $_s =~ s|%20|+|g; return $_s; } sb (q|abc%20qrs|);

step:                   Subbed [{SUBBED}]

[andrewsz87]

It worked, you are great.

Unfortunately I haven't solved my problem.

I am making a robot to verify Outlook Web Access in my company, even with the last correct POST I receive a 401 error (unauthorized) From the Web instead with the same credentials (NO MFA) I can access and all the single steps between webimblaze and browsers are identical.

Are there any limitations? What do you suggest me? Thanks

[Qarj]

Hi there,

When you say the steps are identical, does this include all the request headers including the user-agent and the cookies? If they truly are then there shouldn't be any reason it wouldn't work - unless it was doing something weird like checking for small differences in timings between the two scenarios.

If anything is being calculated by JavaScript based on cookie values or the content of local or session storage, then you would be better off using something like cypress.io to do this. However Cypress also has limitations especially when working with SSL or HSTS. In which case you would be forced to use something like Selenium or Playwright.

Best of luck.

---

From: andrewsz87 @.> Sent: 14 November 2021 10:45 AM To: Qarj/WebImblaze @.> Cc: Tim Buckland @.>; State change @.> Subject: Re: [Qarj/WebImblaze] Problem with URL encoding (Percent Encoding) (Issue #66)

CAUTION: external mail

It worked, you are great.

Unfortunately I haven't solved my problem.

I am making a robot to verify Outlook Web Access in my company, even with the last correct POST I receive a 401 error (unauthorized) From the Web instead with the same credentials (NO MFA) I can access and all the single steps between webimblaze and browsers are identical.

Are there any limitations? What do you suggest me? Thanks

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHubhttps://github.com/Qarj/WebImblaze/issues/66#issuecomment-968265913, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AANJHICHFQGTAGYQYY3ECCTUL6HK3ANCNFSM5H4OSDUQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.