Regexps with suspicious /+ or /* in the codebase

[RReverser]

These were found as part of https://github.com/EFForg/https-everywhere/pull/12294 (although given that it checked only part of rulesets, there might be more). By "suspicious" I mean that in these cases author probably meant /.* or just / - multiple slashes in a row are unlikely to be the target unless it's some exotic case.

• 10gen.xml: Suspicious /+ while traversing ^http://blog\.10gen\.com/+(?:$|\?.*) => https://www.mongodb.com/blog/

• AOE.com.xml: Suspicious /* while traversing ^http://(www\.)?aoe\.com/(?=$|\?|en/*(?:$|\?)|en/press/press-contact\.html|fileadmin/|typo3conf/|typo3temp/|uploads/) => https://$1aoe.com/

• Acrobat.com.xml: Suspicious /+ while traversing ^http://success\.acrobat\.com/+ => https://success.adobe.com/

• Admissions.com.xml: Suspicious /+ while traversing ^http://(?:www\.)?admissions\.com/+ => https://www.fastweb.com/

• Animoto.com.xml: Suspicious /* while traversing ^http://(www\.)?animoto\.com/(?=favicon\.ico|images/|sign_(?:in|up)/*(?:$|\?)) => https://$1animoto.com/

• AskMonty.xml: Suspicious /+ while traversing ^http://(?:www\.)?askmonty\.org/+ => https://mariadb.com/about/

• BBB_Silicon.org.xml: Suspicious /+ while traversing ^http://(?:www\.)?bbbsilicon\.org/+(?:\?.*)? => https://sanjose.bbb.org/

• BE_One_Spark.com.xml: Suspicious /* while traversing ^http://(www\.)?beonespark\.com/(?=assets/|berlin/(?:sign-in|users/password)/*(?:$|\?)) => https://$1beonespark.com/

• BrightTALK.com.xml: Suspicious /+ while traversing ^http://go\.brighttalk\.com/+(?:\?.*)?$ => https://app-sji.marketo.com/

• Bunchball.com.xml: Suspicious /+ while traversing ^http://go\.bunchball\.com/+(?:\?.*)?$ => https://www.bunchball.com/

• Ch9.ms.xml: Suspicious /+ while traversing ^http://(?:www\.)?ch9\.ms/+(?:\?.*)?$ => https://channel9.msdn.com/

• Clear-link.com.xml: Suspicious /+ while traversing ^http://(?:www\.)?clear-link\.com/+ => https://www.clearlink.com/

• Digital_WPC.com.xml: Suspicious /+ while traversing ^http://(?:www\.)?digitalwpc\.com/+ => https://mspartner.microsoft.com/en/us/Pages/WPC/overview.aspx

• EDeveloperz.xml: Suspicious /+ while traversing ^http://(?:version5\.|www\.)?edeveloperz\.com/+([^?]*)(?:\?.*)? => https://version6.edeveloperz.com/$1

• EFinancialCareers.cn.xml: Suspicious /+ while traversing ^http://efinancialcareers\.cn/+ => https://www.efinancialcareers.cn/

• EWebScapes.com.xml: Suspicious /+ while traversing ^http://(?:www\.)?ewebscapes\.com/+ => https://webdevstudios.com/

• Element_5.xml: Suspicious /+ while traversing ^http://www\.element5\.com/+ => https://www.mycommerce.com/share-it

• Elle.com.xml: Suspicious /* while traversing ^http://dating\.elle\.com/(?=assets/|bridge\.js|favicon\.ico|login/*(?:$|\?)|whitelabel/elle/header\.html) => https://dating.elle.com/

• Enphase-Energy.xml: Suspicious /+ while traversing ^http://info\.enphase\.com/+(?:\?.*)?$ => https://enphase.com/

• Exacom.sk.xml: Suspicious /+ while traversing ^http://(?:www\.)?exacom\.sk/+ => https://www.exahost.com/

• FAANCollegeNetwork.xml: Suspicious /+ while traversing ^http://(?:www\.)?faancollegenetwork\.org/+(?:\?.*)?$ => https://www.foodallergy.org/resources-for/colleges-universities/college-food-allergy-program

• FontMarketplace.com.xml: Suspicious /+ while traversing ^http://(?:www\.)?fontmarketplace\.com/+(?=$|\?) => https://www.fonts.com/

• FreeMarket_Lite.cc.xml: Suspicious /+ while traversing ^http://freemarketlite\.cc/+ => https://www.freemarketlite.cc/

• FundaGeek.xml: Suspicious /+ while traversing ^http://(?:www\.)?fundageek\.com/+([^?]*).* => https://gogetfunding.com/$1

• Get_Chef.com.xml: Suspicious /+ while traversing ^http://docs\.getchef\.com/+(?=_static/) => https://d172u545pcyiea.cloudfront.net/

• Get_Rave.com.xml: Suspicious /+ while traversing ^http://getrave\.com/+ => https://www.getrave.com/

• Handy.de.xml: Suspicious /+ while traversing ^http://(?:www\.)?handy\.de/+[^?]*\?(.*) => https://www.etracker.de/lnkcnt.php?et=kbglnx&url=http://www.mondiamedia.com%3FL%3D0%26redirect_from%3Dhandy.de&lnkname=handy&$1

• Haskoin.com.xml: Suspicious /+ while traversing ^http://(?:www\.)?haskoin\.com/+([^?]+) => https://github.com/$1/haskoin

• Hinckley_and_Bosworth_online.org.uk.xml: Suspicious /+ while traversing ^http://(?:www\.)?hinckleyandbosworthonline\.org\.uk/+ => https://www.hinckley-bosworth.gov.uk/

• Innovate_UK.org.xml: Suspicious /+ while traversing ^http://(?:www\.)?innovateuk\.org/+([^?]*)(?:\?.*)? => https://www.gov.uk/government/organisations/innovate-uk$1

• Isle.jp.xml: Suspicious /+ while traversing ^http://(?:www\.)?isle\.jp/+ => https://shared.gmocloud.com/

• Joyent_Cloud.com.xml: Suspicious /+ while traversing ^http://www\.joyentcloud\.com/+ => https://www.joyent.com/

• KTk.de.xml: Suspicious /+ while traversing ^http://(?:www\.)?ktk\.de/+ => https://www.kevag-telekom.de/

• Linerunner.xml: Suspicious /+ while traversing ^http://store\.getcloudapp\.com//+([^?]+) => https://my.cl.ly/plans/$1

• Mamba.xml: Suspicious /+ while traversing ^http://(?:www\.)?corp\.mamba\.ru/+ => https://corp.wamba.com/ru/

• Mapraider.com.xml: Suspicious /+ while traversing ^http://(www\.)?mapraider\.com/+(login|register)(?=$|\?) => https://$1mapraider.com/$2/

• Messagingengine.com.xml: Suspicious /+ while traversing ^http://(?:www\.)?messagingengine\.com/+([^?]*)(?:\?.*)? => https://www.fastmail.com/$1

• Mirror_Bingo.com.xml: Suspicious /+ while traversing ^http://cache(games|www)\.mirrorbingo\.com/+(?!(?!getcss/bingo\.css|html/css/main\.css|html/portlet/journal_content/css/main\.css).+\.css(?:$|\?)) => https://$1.mirrorbingo.com/

• Mobify.me.xml: Suspicious /+ while traversing ^http://(?:www\.)?mobify\.me/+ => https://mobify.com/

• Monty_Program.xml: Suspicious /+ while traversing ^http://(?:www\.)?montyprogram\.com/+ => https://mariadb.com/

• My_phone.coop.xml: Suspicious /+ while traversing ^http://myphone\.coop/+([^?]*).* => https://www.thephone.coop/$1

• Newzsec.com.xml: Suspicious /+ while traversing ^http://(?:www\.)?newzsec\.com/+ => https://saltysailor.github.io/

• OpenMedia_now.net.xml: Suspicious /+ while traversing ^http://(?:www\.)?openmedianow\.net/+ => https://openmedia.org/

• Paymo.xml: Suspicious /+ while traversing ^http://app\.paymo\.biz/+ => https://app.paymoapp.com/

• Pumo.com.tw.xml: Suspicious /* while traversing ^http://www\.pumo\.com\.tw/(?=/*www/+(?:[\w-]+/+)?(?:css/|favicon\.ico|images/|inc/|top/)) => https://www.pumo.com.tw/

• Re-publica.de.xml: Suspicious /+ while traversing ^http://mail\.re-publica\.de/+ => https://ntmx.de/

• Rfecom.com.xml: Suspicious /+ while traversing ^http://www\.rfecom\.com/+(?:$|\?.*) => https://www.myoptimizerplus.com/

• Save_the_Children.org.uk.xml: Suspicious /* while traversing ^http://(?:www\.)?savethechildren\.org\.uk/(?=donate/*(?:$|\?)|favicon\.ico|modules/|sites/) => https://www.savethechildren.org.uk/

• Skoop.xml: Suspicious /+ while traversing ^http://(?:www\.)?theskoop\.ca/+(?:\?.*)?$ => https://www.facebook.com/theskoopinc

• SmashFly.com.xml: Suspicious /* while traversing ^http://go\.smashfly\.com/(?=/*l/) => https://go.pardot.com/

• Stanford.edu-falsemixed.xml: Suspicious /+ while traversing ^http://knight\.stanford\.edu/+ => https://jsk.stanford.edu/

• The-DMA.org.xml: Suspicious /+ while traversing ^http://(?:www\.)?the-dma\.org/+ => https://thedma.org/

• Tmblr.co.xml: Suspicious /+ while traversing ^http://tmblr\.co/+ => https://www.tumblr.com/

• Transmode.se.xml: Suspicious /+ while traversing ^http://(?:www\.)?transmode\.se/+ => https://www.transmode.com/

• UWinnipeg.ca.xml: Suspicious /+ while traversing ^http://indigenous\.uwinnipeg\.ca/+(?:\?.*)?$ => https://www.uwinnipeg.ca/index/indigenous-programs-services

• Unwanted_Witness.or.ug.xml: Suspicious /+ while traversing ^http://(?:www\.)?unwantedwitness\.or\.ug:209[56]/+ => https://hos.hostalite.com:2096/

• Ustream.xml: Suspicious /+ while traversing ^http://support\.ustream\.tv/+(?=$|\?) => https://ustream.zendesk.com/home

• Vdio.xml: Suspicious /+ while traversing ^http://(?:www\.)?vdio\.com/+([^?]*).* => https://www.rdio.com/$1

• Vertical_Response.com.xml: Suspicious /* while traversing ^http://helpcenter\.verticalresponse\.com/(?=/*$|/*\?) => https://vr.force.com/community/Communities/Welcome

• Websterwood.com.xml: Suspicious /+ while traversing ^http://(www\.)?websterwood\.com/+ => https://blog.websterwood.com/

• WiebeTech.com.xml: Suspicious /+ while traversing ^http://(?:www\.)?wiebetech\.com/+ => https://www.cru-inc.com/

• Yandex.com.ua.xml: Suspicious /+ while traversing ^http://(?:www\.)?browser\.yandex\.com\.ua/+ => https://browser.yandex.ua/

• Yandex.net.xml: Suspicious /+ while traversing ^http://(?:www\.)?yandex\.net/+ => https://yandex.ru/

• Zenger.nl.xml: Suspicious /+ while traversing ^http://www\.zenger\.nl/+ => https://www.axelarnbak.nl/

• nlx.org-falsemixed.xml: Suspicious /+ while traversing ^http://(?:www\.)?nlx\.org/+(?:\?.*)?$ => https://us.jobs/

• technologyreview.in.xml: Suspicious /+ while traversing ^http://(?:www\.)?technologyreview\.in/+ => https://www.technologyreview.com/

• uVPN.xml: Suspicious /+ while traversing ^http://(?:www\.)?uvpn\.de/+ => https://fremaks.de/

[ghost]

/+ is intentional, multiple slashes should be treated as one slash. /* is a misspelling of /+.

[RReverser]

Disregard previous comment Why is that needed? (generally curious) How often URLs actually have several slashes after a host that need to be squashed when transforming?

[ghost]

@RReverser Just explaining that /* is an error, but /+ isn't. Do you want me to make a PR to change all instances of /* to /+ or are you going to do that yourself?

[ghost]

@RReverser Actually I can't see any case where /* is used incorrectly here, since in all cases I see it follows a /.

[ghost]

@RReverser Close I guess, unless you can find any place where /* does not follow a slash, allowing URLs like https://example.comexample/.

[RReverser]

I didn't say it's an error, I just said it's questionable because I don't see why these multi-slashes are required ro be matched. If there is no real need for that, many of these rules could be trivialized by the same PR. Hence the question on whether these rules are actually required or they meant to be something else or I can safely trivialize them where possible.

[RReverser]

@bisaloo I see you already started doing that for some rulesets manually, let me know if you want me to run trivializer script with an assumption that /+ can match just /.

[Bisaloo]

I don't really know.

I have seen domains doing strange stuff with multiple slashes. As far as I'm concerned, it's totally possible those /+ and /* (or at least some of them) are here for a reason.

I will let you know if I find an example where they are actually needed.

[RReverser]

@Bisaloo Thank you, that would be helpful!

[Bisaloo]

@RReverser, an example of a double slash in the path:

http://www.imdb.com//images/b.gif

which is loaded on any movie page from IMDb (for example http://www.imdb.com/title/tt0120737/)

In this case, we can still write a rule without /+ or /* because this path is valid with both https and http but as I said before, I would not be surprised if some websites had an unexpected behaviour with multiple slashes.

[RReverser]

@Bisaloo @Hainish Is the decision here that this is not an issue, then? Should I close it?

[Hainish]

I don't necessarily view multiple /'s as suspicious, for the above reasons. Since it is the servers that determine how to respond to multiple /'s, it wouldn't surprise me if there is strange behavior out there from misconfigured or obscure web servers. I'm inclined to keep these. Closing.