search

baoxituo1 / support-tools

Automatically exported from code.google.com/p/support-tools
Apache License 2.0
0 stars 0 forks source link

Some issue attachment links still do not work #50

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Sorry for being picky about this.

I have 249 attachments in my GoogleCodeProjectHosting.json.

Using the info from issue 39 (comment indices are 0-based) I managed to obtain 
links to 122 attachments (listed in good-links.txt).

Apparently another 112 of the attachments actually use 1-based indices in the 
links  (listed in bad-fixable.txt).

Yet there are still 15 attachment links that *do not work at all* (listed in 
bad-unfixable.txt).

See the attached files for the links. I used the projects tint2 and pmn.

--

As a workaround I tried to use directly the links visible on the Google Code 
webpage, which look like this:

https://pmn.googlecode.com/issues/attachment?aid=10004000&name=magic&token=remov
edthesecrettoken

and download all of them to my computer. However I *have to* include the secret 
token parameter in the URL otherwise the script gets redirected to a Sign In 
page which it cannot handle (as it is not a browser). It is not clear how to 
obtain that token.

Finally, I tried copy-pasting the token from the browser to my script but 
apparently a separate token is required for each attachment, so my conclusion 
is that it is impossible to automate this process except maybe with a browser 
extension that crawls the whole issue tracker.

Original issue reported on code.google.com by mrovi9...@gmail.com on 18 Mar 2015 at 12:52

Attachments:

GoogleCodeExporter commented 9 years ago 
And I am not even sure that I have really "fixed" some of the bad links, as I 
sometimes see colliding file names in different comments like this:

User comment 1: Something doesn't work, attaches config.txt
User comment 2: Workaround doesn't work either, attaches config.txt

Original comment by mrovi9...@gmail.com on 18 Mar 2015 at 12:57

GoogleCodeExporter commented 9 years ago 
Thank you for the bug report. I will take a look.

Original comment by chrsm...@google.com on 18 Mar 2015 at 3:21

GoogleCodeExporter commented 9 years ago 
Any update? We're looking to add an "Import from Google Code" feature to GitLab 
and want to import attachments as well, but this issue is standing in our way.

Thank you.

Original comment by do...@gitlab.com on 30 Mar 2015 at 9:16

GoogleCodeExporter commented 9 years ago 
We have added a wiki page detailing how our issue attachment mirroring works. 
You can get issues without the additional token if you go straight to Google 
Cloud Storage.

See: https://code.google.com/p/support-tools/wiki/IssueMirror

However our issue mirroring does have some known issues (we won't fix). We only 
mirror _public_ issues. So attachments for private (Restrict-View-*) issues 
will not be available. Similarly, we don't mirror attachments for deleted 
issues. And, in the occasional situation where you upload two attachments at 
the same commit, both having the same name, we only have one of the files on 
GCS.

As for your bad-unfixable.txt file, you have uncovered something afoot in 
Google Code. For example:

Bad link: 
https://storage.googleapis.com/google-code-attachments/tint2/issue-471/comment-3
4/fedora-2015-03-15T10-43-31-220064000Z.webm for issue 471, comment 35, 
attachment {"mimetype": "application/octet-stream", "attachmentId": 
"4710035000", "fileSize": 10148504, "fileName": 
"fedora-2015-03-15T10-43-31-220064000Z.webm"}

The attachment file is actually found in our mirror, though as comment #31:
https://storage.googleapis.com/google-code-attachments/tint2/issue-471/comment-3
1/fedora-2015-03-15T10-43-31-220064000Z.webm

I have no explanation for the discrepancy, other than I probably wrote the 
errant code.

Could you tell me more about how your "Import from Google Code" feature of 
GitLab works? I take it GitLab supports arbitrary file attachments, so you need 
to download the attachments at the time you do the import? If so, that's a 
great feature. But note that there is a delay between when issue attachments 
are uploaded to Google Code and when they are mirrored onto Google Cloud 
Storage.

Original comment by chrsm...@google.com on 31 Mar 2015 at 12:11

GoogleCodeExporter commented 9 years ago 
The known restrictions to the IssueMirror are not a problem.

As for migrating public attachments, we have two options, since GitLab does 
indeed support arbitrary file attachments:

1. Download all attachments from the IssueMirror and reupload to GitLab when a 
user requests a project import. 
2. Link directly to the attachment on the IssueMirror from the new GitLab issue.

Both options depend on the IssueMirror URLs actually working, which 
mrovi9...@gmail.com reports they aren't currently. The first option also 
requires that we aren't rate limited or otherwise blocked by Google Cloud 
Storage for downloading large numbers of files. The second option is only 
viable if there is a guarantee that the IssueMirror will stay up indefinitely. 

Downsides to the first option would be the storage and bandwidth requirements 
on our side, and the mirror delay you mention (how large is that delay?) The 
main downside to the second option would be the ongoing dependency on Google 
Storage :)

We are ultimately fine with either option, but we need to be sure the 
IssueMirror URLs work.

Original comment by do...@gitlab.com on 31 Mar 2015 at 12:56