Download raw file contents from a specific commit

[GoogleCodeExporter]

Support downloading the contents of a single file "as is" from Gitiles, for 
example:

 https://android.googlesource.com/platform/manifest/+raw/master/default.xml

to get the raw XML rather than it wrapped inside of HTML.

This is a bit of a challenge because the server has cookies, and this is 
possibly unsafe user supplied data. Raw HTML or JavaScript could cause the 
user's session to be able to be hijacked.

Original issue reported on code.google.com by dborowitz@google.com on 11 Nov 2012 at 11:30

[GoogleCodeExporter]

Would providing a json format (issue 27) have less security concerns?  Chrome 
team has tools which need to get current state of certain files.  This is 
currently done via chromium svn servers but I'd like to transition these tools 
to use GoB.  An example file which tools fetch is:  
https://chromium.googlesource.com/chromium/src/+/master/DEPS

Original comment by ilevy@chromium.org on 6 May 2013 at 1:10

[GoogleCodeExporter]

Would it be sufficient to always set the content-type to something like 
"application/octet-stream"? Are there any browsers that would attempt to 
parse/display that, rather than just downloading? That might help avoid the 
browser hijacking issue, while still allowing easy programmatic access from our 
tools.

Original comment by mmoss@google.com on 6 May 2013 at 4:14

[GoogleCodeExporter]

Or another option, maybe always put the file(s) into a zip archive, which would 
force downloading and unzipping, again avoiding browser handling of the 
content, but still being pretty easy for scripts to deal with.

Original comment by mmoss@google.com on 6 May 2013 at 4:18

[GoogleCodeExporter]

#1: JSON is a good idea, that would be easy to implement in the short term. 
Works for the scripting case but we still want true raw file support for e.g. 
viewing in-repo HTML.

#2: Unfortunately if you include a file with an <img> or <script> tag browsers 
are still liable to do content sniffing regardless of Content-Type in the 
header :(. Modern browsers will respect Content-Disposition: Attachment but 
older IEs may not. And again we still want to support browsing raw HTML stored 
in the repo.

#3: Zip archive is another idea, but between that and JSON I think I prefer 
JSON :)

Original comment by dborowitz@google.com on 6 May 2013 at 9:30

[GoogleCodeExporter]

Also an issue for Google's "Skia" open-source project.  Details are at 
https://code.google.com/p/skia/issues/detail?id=1898 ('git transition blocker: 
skia_tools.js needs to download tip-of-tree global_variables.json')

Original comment by epoger@google.com on 13 Dec 2013 at 8:43

[GoogleCodeExporter]

GitHub has this functionality (they serve raw blobs as text/plain). Are they 
just unaware or unconcerned about the content-sniffing issue alluded to in #2?

Original comment by dha...@google.com on 8 Jan 2014 at 1:41

[GoogleCodeExporter]

github solves the problem of cookies by serving the raw URL from a different 
hostname; i.e. https://github.com/dataarts/tailbone/blob/master/Guardfile is 
served raw from https://raw.github.com/dataarts/tailbone/master/Guardfile

Could that work for gitiles?

Original comment by fischman@chromium.org on 6 Feb 2014 at 11:42

[GoogleCodeExporter]

Yes, for googlesource.com we plan to use a new, cookieless domain (and for 
open-source Gitiles the domain will simply be configurable and we trust 
administrators to understand the risks and implement their own cookie policy).

This quarter I plan to do JSON support and at least get the ball rolling with 
our security/domain teams on the cookieless domain part.

Original comment by dborowitz@google.com on 6 Feb 2014 at 11:47

[GoogleCodeExporter]

Blocking:
https://code.google.com/p/chromium/issues/detail?id=337030
https://code.google.com/p/chromium/issues/detail?id=345087
and some aspects of:
https://code.google.com/p/chromium/issues/detail?id=343251

Original comment by mmoss@chromium.org on 10 Mar 2014 at 4:44

[GoogleCodeExporter]

Base64-encoded raw file support is checked in as:
https://gerrit-review.googlesource.com/55202

Example URL (won't work until this is released for googlesource.com):
https://gerrit.googlesource.com/gitiles/+/master/gitiles-servlet/src/main/java/c
om/google/gitiles/PathServlet.java?format=TEXT

Original comment by dborowitz@google.com on 15 Mar 2014 at 4:28

[GoogleCodeExporter]

Original comment by benhe...@google.com on 24 Apr 2014 at 11:33

[GoogleCodeExporter]

Original comment by mmoss@chromium.org on 25 Apr 2014 at 4:17

[GoogleCodeExporter]

FWIW, the base64-encoded delivery is fine for my Chromium use cases. If you 
want to leave this open pending "true" raw file support, that's fine, but I 
don't think it should be a priority.

Original comment by mmoss@chromium.org on 25 Apr 2014 at 4:22

[GoogleCodeExporter]

From the Skia perspective, base64 and JSON cover all of our bases too.

Original comment by epoger@google.com on 25 Apr 2014 at 4:49

[GoogleCodeExporter]

Blocking crbug.com/369329

Original comment by sebmarchand@chromium.org on 15 May 2014 at 7:10

[GoogleCodeExporter]

sebmarchand, that crbug appears to be about a Python script. Any reason base64 
doesn't work for you?

Original comment by dborowitz@google.com on 15 May 2014 at 7:22

[GoogleCodeExporter]

Yeah, I updated that bug with some notes on how to fetch. I don't think this is 
blocking.

Original comment by mmoss@chromium.org on 15 May 2014 at 7:37

[GoogleCodeExporter]

Another issue is with pointing bug reporters and others to download 
bisect-builds.py from the Chromium repo. We used to link to the file on 
src.chromium.org. This page seems to now point to the gitiles page:
http://www.chromium.org/developers/bisect-builds-py

But that's not especially useful as there's no clear way to download it. For 
this use, I think we'd have to be able to download the file itself, and not 
some encoded version.

Original comment by davidben@chromium.org on 4 Sep 2014 at 2:55

[GoogleCodeExporter]

I was able to download bisect-builds.py by adding ?format=TEXT on the end, then 
manually base64 decoding it, but it was a bit of a pain...

Original comment by will@phase.net on 15 Sep 2014 at 1:35

[GoogleCodeExporter]

So does that mean this isn't going to be fixed?

base64 decoding is not a solution.

A normal person should be able to go to 

https://chromium.googlesource.com/chromium/src/+/master/tools/bisect-builds.py

And simply click a link to download the file.

Original comment by mka...@gmail.com on 10 Oct 2014 at 12:38

[GoogleCodeExporter]

This is a real PITA!! Why on earth google doesn't provide a "raw" code link?? 
Absolutely absurd. Please fix!

Original comment by xdae3...@gmail.com on 19 Oct 2014 at 2:11

[GoogleCodeExporter]

please increase the priority of this - it's a real pain to have to attach 
bisect-builds.py to tickets when I want people to help me bisect Chromium.  
Asking them to base64 decode a raw download is too much of a barrier to entry.

Original comment by wfh@chromium.org on 30 Jan 2015 at 9:53

[GoogleCodeExporter]

wfh, the way to bump priority is to raise this with your local chrome-infra 
team member and they can work it into feature planning with our team.

As for bisect-builds.py, I'm not yet convinced base64 decoding is too much of a 
barrier to entry.

The documentation page linked above says, paraphrasing:
1. Download bisect-builds.py from this link.
2. Run this python command.

Would the barrier to entry really be that much higher if the documentation were 
rewritten to say:
1. Run this command to download bisect-builds.py:
$ curl 
'https://chromium.googlesource.com/chromium/src/+/master/tools/bisect-builds.py'
 | base64 -d > bisect-builds.py
2. Run this python command

?

Original comment by dborowitz@google.com on 30 Jan 2015 at 10:00

[GoogleCodeExporter]

d:\src>curl
'curl' is not recognized as an internal or external command,
operable program or batch file.

d:\src>base64
'base64' is not recognized as an internal or external command,
operable program or batch file.

So I think it does for some less technical users (that would be typical of 
bisect-builds users -- if they were devs they'd already have the tree pulled 
and just use it from there.)

Original comment by scottmg@chromium.org on 30 Jan 2015 at 10:08

[GoogleCodeExporter]

Ah. Windows. Point taken.

Original comment by dborowitz@google.com on 30 Jan 2015 at 10:19

[GoogleCodeExporter]

In addition to bisect-builds.py, when reporting compiler regressions I used to 
say "1. Download file <link> 2. Build it with these flags". This is no longer 
possible either. (Again, Windows.)

Original comment by thakis@chromium.org on 29 Apr 2015 at 8:24