search

paws-r / paws

Paws, a package for Amazon Web Services in R
https://www.paws-r-sdk.com
Other
309 stars 37 forks source link

paws url encoder not encoding protocol #701

Closed DyfanJones closed 8 months ago

DyfanJones commented 8 months ago

Only been able to identify this from cran check failure: https://www.r-project.org/nosvn/R.check/r-devel-linux-x86_64-debian-gcc/paws.common-00check.html

In short this is legacy code from urltools that is designed to work like URLencode(url, reserved = F)

broken_strings <- c(
  "s3://mybucket/",
  "https://www.google.com",
  "ftps://foo bar cho"
)

paws.common:::paws_url_encoder(broken_strings)
#> [1] "s3://mybucket/"         "https://www.google.com" "ftps://foo%20bar%20cho"
curl:::curl_escape(broken_strings)
#> [1] "s3%3A%2F%2Fmybucket%2F"       "https%3A%2F%2Fwww.google.com"
#> [3] "ftps%3A%2F%2Ffoo%20bar%20cho"
URLencode(broken_strings, reserved = T)
#> [1] "s3%3A%2F%2Fmybucket%2F"       "https%3A%2F%2Fwww.google.com"
#> [3] "ftps%3A%2F%2Ffoo%20bar%20cho"

Created on 2023-11-03 with reprex v2.0.2

However we would like to mimic URLencode(url, reserved = T) https://github.com/paws-r/paws/blob/paws.common-v0.3.1/paws.common/R/url.R#L100C1-L113

This shouldn't be a huge issue however it would be nice to fix this for next paws.common release.

DyfanJones commented 8 months ago

Here is an example of a round house trip failing:

string <- "h>/pA30{Jot?K/d7yI72~R1f8E0h)AQ&pr+&*?tga4Z $NN>xK+tC8:N51J$?$w^30t\" ~!i'\"gfyL./<_m5F,FZ'm:gkksP+Uu\"5|:2D]P3#%F$ #HXCwE}u~]4O[SR,;.E:l_%Hg'Sc[]U\"V+n[Jq\"0s#Q+{T'!jA2EA>h&\"/,V']ZuV:3@GLJnfIxK0q'V\\@4+7jV+/)*7m!@i3D<|z0J;Bg$3yzv<mG,274{61NCy8 \\(z>xwKGA;G/A$=i{Vdm~F)'TliLeS\\%+6s&UShzznaFy%]+5FLF|';xs[Lo'%}K}qsOGy41Sj@@=)d' XnU8:#$\\HPQ.T0ql\\3IQC/co*8+SnL$H~r7\"\\{Ek<'QYTKcE#g8LE9.$FLGX64\\dfJ+TZ$Nah#mf~GxPR~^QE`'aa')Z[rOj<GBVWXm/sy31MpYn4on$T?2Z-zX_#fr~)AL(.;\"b'M+y5JHRn/u4aH(m9FVqM\\P8pfej<&&s6}C17219T_4N2WPSu:=*\"?dw?__z'b9:~Em4s ?a+Sc8Sn]FQb+Ju.H?Ho,{xo2EcldB8uHP\\{8Q!%4w,to<9Hs(X)<UyJ#9Lq)4#]0Pa2Q+JI@1U3A(?#|E80q:LE^Uo^QY^mq3KJ%p.*DL0A-STPr:D;F:;p:5[nJ,WF'i,CVu;xWdfSQbchQ`F_W(T#A':Rsp5;hhYJa!g3>7 6o ,!vioTm@*d||vLoAcK~MV>?21iL`0fhYD~N&l,PCjVw5&[\"[PU9RPQ3F}~NTDYk$>y$p!p=B)VikhMdk| >-u;MQz?X9o6 x(_t(T|YxQa&sF%^t`KtidZ-5}c&p>_TzPQ2jNkZq{21Jp\"(P^YFfx/*~{[Nm[G$-</+X>>8Gf<)6OVLm32Y50tJA%bD6[-Kr<T|<! tpo~e12Ls7RFy_[$&ipX39O&-=:O8*UYclb!Af?AJE{P#oI?duZAq+cw\"RWY\\P(@BpQS5>&4BCzL~wtS5C,XTG!M0W%C|<5ziQb_xV>MIdJk,7^a'!,YXfv.D8OwQ>InL{uf#;3TmAv>Sv\\a+[7Q6i&RrPUvWyH@JBHL(L4pr8M$gnY0>GG^ lqtE\\>JV03(}DKO30`Z.zpWKye'_wKAu;Ewxdil6>)1ve~G?8$#.}D\"Fx8FJZXiwn.=(\\h>rbcqx2Q-P kPZH*GIKz\\SO37M1^C,SOq;5{jrT<PiQStfE\\?|<&lTlSN \"4\"4ll+AasDX8g8cda8f|`;[r\"JsTYZhJv?EYBH)%NkW AkNVcv: QKWU>gPxUo!u6p+!2Qtq6$!i&` ?/[oBL~MzLNp:CMo~QL+|cWY]dQ`~N,z&O{0pi\\M2|Zeo,U!$g|{QGTvb\"SyZn5UJe`+%U>Wv4b ThGuT+57koboJAA)t_|yX+0a`9uG6R2r?6{y;|x!})>1S/xZas`.P5Ts?T!Z<@B%fPk]9`PuD>;es:E{,AY#7'8VT@VDUxf)COT9hhO0((9/NYo|LtS1\"nQBONb3r5Q,U`\"g}Nn#eOs0G}W|0xvE.{T!oLLkY.RYsW(]fagW$w\\&YJ)BKSvs'=c=q%S58-c[?.VyeWVA;-Hig(fu*D:~1/0[\"V8DG)l=k.=<s->q%cW;5]6xWg`''(R~AlzpW\\`=e$Z+=|byr9Of`TCdLsV@|zi-jq%a#E4TH?xn:GMZ[UgCpJU:K+{O7m[v!%iNul^\\w9<)[5;}BMfJ2q&KM4|?^60/\\N!rslnk<B]Wca(tO_,%EyzFG{W_)`K`/Pr!&6%IIRTGvOy3?k.tDBGYNbp2)*\\-Nh+-FJ5VqxcQWk;aWSdFIDYI|?wY\\!'[/EQUh-&[M,sF<eF]/o02w!0/s6(J!2@mW5M?=U3T;/xVcwET29C417[2=,]-CsJfD<,N+BI`zk=|.d32+Uadj_ap|lB6$'<h7Fq7X(#8U@LtFf`~kjvdKng #K6V)eKq/://+p~ryVPJJXW]{b\"j%D'?HbB.ilc\\OiO[pUV4.;GG>$ [,o{V!jBZ%PtC#e\\iOP_t#L"

string_encode <- paws.common:::paws_url_encoder(string)
actual <- paws.common:::unescape(string_encode)
testthat::expect_equal(actual, string)
#> Error: `actual` not equal to `string`.
#> 1/1 mismatches
#> x[1]: "h>/pA30{Jot?K/d7yI72~R1f8E0h)AQ&pr+&*?tga4Z $NN>xK+tC8:N51J$?$w^30t\" ~!i
#> x[1]: '\"gfyL./<_m5F,FZ'm:gkksP+Uu\"5|:2D]P3#%F$ #HXCwE}u~]4O[SR,;.E:l_%Hg'Sc[]U
#> x[1]: \"V+n[Jq\"0s#Q+{T'!jA2EA>h&\"/,V']ZuV:3@GLJnfIxK0q'V\\@4+7jV+/)*7m!@i3D<|z
#> x[1]: 0J;Bg$3yzv<mG,274{61NCy8 \\(z>xwKGA;G/A$=i{Vdm~F)'TliLeS\\%+6s&UShzznaFy%]
#> x[1]: +5FLF|';xs[Lo'%}K}qsOGy41Sj@@=)d' XnU8:#$\\HPQ.T0ql\\3IQC/co*8+SnL$H~r7...
#> y[1]: "h>/pA30{Jot?K/d7yI72~R1f8E0h)AQ&pr+&*?tga4Z $NN>xK+tC8:N51J$?$w^30t\" ~!i
#> y[1]: '\"gfyL./<_m5F,FZ'm:gkksP+Uu\"5|:2D]P3#%F$ #HXCwE}u~]4O[SR,;.E:l_%Hg'Sc[]U
#> y[1]: \"V+n[Jq\"0s#Q+{T'!jA2EA>h&\"/,V']ZuV:3@GLJnfIxK0q'V\\@4+7jV+/)*7m!@i3D<|z
#> y[1]: 0J;Bg$3yzv<mG,274{61NCy8 \\(z>xwKGA;G/A$=i{Vdm~F)'TliLeS\\%+6s&UShzznaFy%]
#> y[1]: +5FLF|';xs[Lo'%}K}qsOGy41Sj@@=)d' XnU8:#$\\HPQ.T0ql\\3IQC/co*8+SnL$H~r7...

Created on 2023-11-03 with reprex v2.0.2

DyfanJones commented 8 months ago

PR #702 fixes encoding issue.

Using same round house example

string <- "h>/pA30{Jot?K/d7yI72~R1f8E0h)AQ&pr+&*?tga4Z $NN>xK+tC8:N51J$?$w^30t\" ~!i'\"gfyL./<_m5F,FZ'm:gkksP+Uu\"5|:2D]P3#%F$ #HXCwE}u~]4O[SR,;.E:l_%Hg'Sc[]U\"V+n[Jq\"0s#Q+{T'!jA2EA>h&\"/,V']ZuV:3@GLJnfIxK0q'V\\@4+7jV+/)*7m!@i3D<|z0J;Bg$3yzv<mG,274{61NCy8 \\(z>xwKGA;G/A$=i{Vdm~F)'TliLeS\\%+6s&UShzznaFy%]+5FLF|';xs[Lo'%}K}qsOGy41Sj@@=)d' XnU8:#$\\HPQ.T0ql\\3IQC/co*8+SnL$H~r7\"\\{Ek<'QYTKcE#g8LE9.$FLGX64\\dfJ+TZ$Nah#mf~GxPR~^QE`'aa')Z[rOj<GBVWXm/sy31MpYn4on$T?2Z-zX_#fr~)AL(.;\"b'M+y5JHRn/u4aH(m9FVqM\\P8pfej<&&s6}C17219T_4N2WPSu:=*\"?dw?__z'b9:~Em4s ?a+Sc8Sn]FQb+Ju.H?Ho,{xo2EcldB8uHP\\{8Q!%4w,to<9Hs(X)<UyJ#9Lq)4#]0Pa2Q+JI@1U3A(?#|E80q:LE^Uo^QY^mq3KJ%p.*DL0A-STPr:D;F:;p:5[nJ,WF'i,CVu;xWdfSQbchQ`F_W(T#A':Rsp5;hhYJa!g3>7 6o ,!vioTm@*d||vLoAcK~MV>?21iL`0fhYD~N&l,PCjVw5&[\"[PU9RPQ3F}~NTDYk$>y$p!p=B)VikhMdk| >-u;MQz?X9o6 x(_t(T|YxQa&sF%^t`KtidZ-5}c&p>_TzPQ2jNkZq{21Jp\"(P^YFfx/*~{[Nm[G$-</+X>>8Gf<)6OVLm32Y50tJA%bD6[-Kr<T|<! tpo~e12Ls7RFy_[$&ipX39O&-=:O8*UYclb!Af?AJE{P#oI?duZAq+cw\"RWY\\P(@BpQS5>&4BCzL~wtS5C,XTG!M0W%C|<5ziQb_xV>MIdJk,7^a'!,YXfv.D8OwQ>InL{uf#;3TmAv>Sv\\a+[7Q6i&RrPUvWyH@JBHL(L4pr8M$gnY0>GG^ lqtE\\>JV03(}DKO30`Z.zpWKye'_wKAu;Ewxdil6>)1ve~G?8$#.}D\"Fx8FJZXiwn.=(\\h>rbcqx2Q-P kPZH*GIKz\\SO37M1^C,SOq;5{jrT<PiQStfE\\?|<&lTlSN \"4\"4ll+AasDX8g8cda8f|`;[r\"JsTYZhJv?EYBH)%NkW AkNVcv: QKWU>gPxUo!u6p+!2Qtq6$!i&` ?/[oBL~MzLNp:CMo~QL+|cWY]dQ`~N,z&O{0pi\\M2|Zeo,U!$g|{QGTvb\"SyZn5UJe`+%U>Wv4b ThGuT+57koboJAA)t_|yX+0a`9uG6R2r?6{y;|x!})>1S/xZas`.P5Ts?T!Z<@B%fPk]9`PuD>;es:E{,AY#7'8VT@VDUxf)COT9hhO0((9/NYo|LtS1\"nQBONb3r5Q,U`\"g}Nn#eOs0G}W|0xvE.{T!oLLkY.RYsW(]fagW$w\\&YJ)BKSvs'=c=q%S58-c[?.VyeWVA;-Hig(fu*D:~1/0[\"V8DG)l=k.=<s->q%cW;5]6xWg`''(R~AlzpW\\`=e$Z+=|byr9Of`TCdLsV@|zi-jq%a#E4TH?xn:GMZ[UgCpJU:K+{O7m[v!%iNul^\\w9<)[5;}BMfJ2q&KM4|?^60/\\N!rslnk<B]Wca(tO_,%EyzFG{W_)`K`/Pr!&6%IIRTGvOy3?k.tDBGYNbp2)*\\-Nh+-FJ5VqxcQWk;aWSdFIDYI|?wY\\!'[/EQUh-&[M,sF<eF]/o02w!0/s6(J!2@mW5M?=U3T;/xVcwET29C417[2=,]-CsJfD<,N+BI`zk=|.d32+Uadj_ap|lB6$'<h7Fq7X(#8U@LtFf`~kjvdKng #K6V)eKq/://+p~ryVPJJXW]{b\"j%D'?HbB.ilc\\OiO[pUV4.;GG>$ [,o{V!jBZ%PtC#e\\iOP_t#L"

string_encode <- paws.common:::paws_url_encoder(string)
actual <- paws.common:::unescape(string_encode)
testthat::expect_equal(actual, string)

Created on 2023-11-03 with reprex v2.0.2

DyfanJones commented 8 months ago

Closing this ticket as paws.common 0.6.3 has been released to the cran.