Storage: file not found with slash inside the path

[GoogleCodeExporter]

What steps will reproduce the problem?

code snipped:

 var certificate = new
 X509Certificate2(@.....", "notasecret", X509KeyStorageFlags.Exportable);

            ServiceAccountCredential credential = new ServiceAccountCredential(
               new ServiceAccountCredential.Initializer(serviceAccountEmail)
               {
                   Scopes = new[] { StorageService.Scope.DevstorageReadOnly }
               }.FromCertificate(certificate));

            // Create the service.

            var storageService = new StorageService(new BaseClientService.Initializer()
            {
                HttpClientInitializer = credential,
            });

            //this could be downloaded
            //var getRequest = storageService.Objects.Get("bucket_di_prova", "nomesenzaspazi.jpg");

            /this one can not be downloaded
            var getRequest = storageService.Objects.Get("bucket_di_prova", "test/nomesenzaspazi.jpg");
            Console.WriteLine(getRequest.Generation);
            using (var fileStream = new System.IO.FileStream(
                "C:\\Users\\LucaDC\\Downloads\\nomesenzaspazi.jpg",
                System.IO.FileMode.Create,
                System.IO.FileAccess.Write))
            {

                getRequest.Download(fileStream);
            }

What version of the product are you using? Google.Apis.Storage.v1beta2 Client 
Library 1.8.1.410
What is your operating system? windows 7
What is your IDE? Visual Studio express 2010
What is the .NET framework version? 4.5

the object "test/nomesenzaspazi.jpg" saved in the file system contains the 
text: NOT FOUND, the object "nomesenzaspazi.jpg" is saved correctly (an image)

Original issue reported on code.google.com by giordano...@noovle.it on 28 Mar 2014 at 12:15

[GoogleCodeExporter]

Can you try to encode "/" and check if it works?

Original comment by pele...@google.com on 29 Mar 2014 at 6:38

• Changed state: Accepted

[GoogleCodeExporter]

did you mean url encoding? I tried with %2F but without results.

Original comment by giordano...@noovle.it on 29 Mar 2014 at 8:46

[GoogleCodeExporter]

Several questions:

1. 
var getRequest = storageService.Objects.Get("bucket_di_prova", 
"test/nomesenzaspazi.jpg");
var results = getRequest.Execute();
Does that work?

2. 
https://developers.google.com/apis-explorer/#p/storage/v1beta2/storage.objects.g
et?bucket=bucket_di_prova&object=test%252Fnomesenzaspazi.jpg&_h=1&
Does that work? If so it means that it's the client library problem and not the 
API.

Thanks!

Original comment by pele...@google.com on 31 Mar 2014 at 1:00

[GoogleCodeExporter]

Hi,

thanks for your help, here my answers:

1. No, it doesn't work (File not found)

2.

Yes, it works, here the result:

{

 "kind": "storage#object",
 "id": "bucket_di_prova/test/nomesenzaspazi.jpg/1396001144113000",
 "selfLink": "https://content.googleapis.com/storage/v1beta2/b/bucket_di_prova/o/test%2Fnomesenzaspazi.jpg",
 "name": "test/nomesenzaspazi.jpg",
 "bucket": "bucket_di_prova",
 "generation": "1396001144113000",
 "metageneration": "5",
 "contentType": "image/jpeg",
 "updated": "2014-03-28T10:05:43.458Z",
 "storageClass": "STANDARD",
 "size": "174451",
 "md5Hash": "p+XDs4ZHIUYAT7VQPzvjww==",
 "mediaLink": "https://content.googleapis.com/storage/v1beta2/b/bucket_di_prova/o/test%2Fnomesenzaspazi.jpg?generation=1396001144113000&alt=media",
 "owner": {
  "entity": "user-00b4903a97073b5ab401c7b9ddc8946a7a219656abf87cc24ed89f2f639ae872",
  "entityId": "00b4903a97073b5ab401c7b9ddc8946a7a219656abf87cc24ed89f2f639ae872"
 },
 "crc32c": "O8ljow==",
 "etag": "COiOtML9tL0CEAU="
}

Original comment by giordano...@noovle.it on 31 Mar 2014 at 7:54

[GoogleCodeExporter]

1. Can you send the request as well? using the Google API explorer?
2. Can you check in fiddler how the request looks using the library?

It seems like a bug in the client library.
Thanks for your collaboration :)

Original comment by pele...@google.com on 31 Mar 2014 at 12:49

[GoogleCodeExporter]

1. api explorer

Request

GET 
https://www.googleapis.com/storage/v1beta2/b/bucket_di_prova/o/test%2Fnomesenzas
pazi.jpg?key={YOUR_API_KEY}

Authorization:  Bearer 
ya29.1.AADtN_W1etGzuSG9u53bINPuppMV617aSK5AsjsCp3MkG-281rq4aPco-wripkWHhyUc_FA
X-JavaScript-User-Agent:  Google APIs Explorer

Response

200 OK

- Show headers -

{

 "kind": "storage#object",
 "id": "bucket_di_prova/test/nomesenzaspazi.jpg/1396001144113000",
 "selfLink": "https://content.googleapis.com/storage/v1beta2/b/bucket_di_prova/o/test%2Fnomesenzaspazi.jpg",
 "name": "test/nomesenzaspazi.jpg",
 "bucket": "bucket_di_prova",
 "generation": "1396001144113000",
 "metageneration": "5",
 "contentType": "image/jpeg",
 "updated": "2014-03-28T10:05:43.458Z",
 "storageClass": "STANDARD",
 "size": "174451",
 "md5Hash": "p+XDs4ZHIUYAT7VQPzvjww==",
 "mediaLink": "https://content.googleapis.com/storage/v1beta2/b/bucket_di_prova/o/test%2Fnomesenzaspazi.jpg?generation=1396001144113000&alt=media",
 "owner": {
  "entity": "user-00b4903a97073b5ab401c7b9ddc8946a7a219656abf87cc24ed89f2f639ae872",
  "entityId": "00b4903a97073b5ab401c7b9ddc8946a7a219656abf87cc24ed89f2f639ae872"
 },
 "crc32c": "O8ljow==",
 "etag": "COiOtML9tL0CEAU="
}

2.
Did you mean with dot.net client? How Could I check? 

Original comment by giordano...@noovle.it on 31 Mar 2014 at 12:57

[GoogleCodeExporter]

Original comment by pele...@google.com on 2 Apr 2014 at 3:59

• Added labels: Milestone-Release1.8.2

[GoogleCodeExporter]

2. Fiddler.
I think that currently we don't encode the parameters. But I want to make sure.
Anyway if you encode the parameter by yourself:
var getRequest = storageService.Objects.Get("bucket_di_prova", 
"test%2Fnomesenzaspazi.jpg");
var results = getRequest.Execute();
Does it work?

Original comment by pele...@google.com on 2 Apr 2014 at 4:02

[GoogleCodeExporter]

Hi,

no, also encoding the url I have "File not found",with slash instaed an general 
exception

Giordano

Original comment by giordano...@noovle.it on 2 Apr 2014 at 4:31

[GoogleCodeExporter]

Can you check the fiddler output for both cases?
Thanks a lot!
Eyal

Original comment by pele...@google.com on 2 Apr 2014 at 4:36

[GoogleCodeExporter]

Hi,

I am not an fiddler expert but here the log.

Giordano

Original comment by giordano...@noovle.it on 2 Apr 2014 at 5:29

Attachments:

• testwith%2F.saz
• test_with_slash.saz

[GoogleCodeExporter]

Hi,
It actually worked for me.
Using:

var getRequest = storageService.Objects.Get("bucket_di_prova", 
"test/nomesenzaspazi.jpg");
            getRequest.Execute();

I got the following request:
GET /storage/v1beta2/b/bucket_di_prova/o/test%2Fnomesenzaspazi.jpg HTTP/1.1
as expected.

It looks to me that we generate the same request URL as in the Google API 
explorer.
Can you double check your code again, and let me know?

THANKS!
Eyal

Original comment by pele...@google.com on 15 Apr 2014 at 5:36

[GoogleCodeExporter]

Original comment by pele...@google.com on 17 Apr 2014 at 1:36

[GoogleCodeExporter]

no, testing with fiddler I can't create the correct url ( 
/storage/v1beta2/b/bucket_di_prova/o/test%2Fnomesenzaspazi.jpg), the problem is:

/ is not url escaped
% is url escaped

so it's impossible to generate the correct url.

I wonder If we are using the same library, I am using:

Storage
1.8.1.43

Giordano

Original comment by giordano...@noovle.it on 17 Apr 2014 at 2:48

[GoogleCodeExporter]

I used 1.8.1.45 (can you update? it should be transparent).
Now I'm not sure if it's my client problem or something with the .NET 
framework. Did you install the latest updates for the .NET framework? Can I 
also check that with .NET 4.5?

Sorry for the confusion, I just didn't manage to reproduce it, and as you can 
see in the following test: 
https://code.google.com/p/google-api-dotnet-client/source/browse/Src/GoogleApis.
Tests/Apis/Requests/RequestBuilderTest.cs#117 we cover that.

Another suggestion: Can you clone the repository and run this test? It should 
work...

Keep me updated!
Sorry for any inconvenient,
Eyal

Original comment by pele...@google.com on 17 Apr 2014 at 3:50

[GoogleCodeExporter]

Hi,

the test

 Assert.That(request.RequestUri, Is.EqualTo(new Uri("http://www.example.com/?test%20Query%20Param=test%20%25va%2Fue"))); 

doesn't give an Exception

I updated the nuget packages but my version remains the 1.8.43 and not 1.8.45.

Giordano

Original comment by giordano...@noovle.it on 22 Apr 2014 at 2:39

[GoogleCodeExporter]

So although the test works, you still getting 404 for your request?
That's weird.

Can you uninstall the package and then install it get? (and then get 1.8.1.45?

Original comment by pele...@google.com on 22 Apr 2014 at 2:48

[GoogleCodeExporter]

done, but i have a 43 version:
PM> install-Package Google.Apis.Storage.v1beta2
Tentativo di risolvere la dipendenza 'Google.Apis (≥ 1.8.1)'.
Tentativo di risolvere la dipendenza 'Google.Apis.Core (≥ 1.8.1)'.
Tentativo di risolvere la dipendenza 'Newtonsoft.Json (≥ 5.0.5)'.
Tentativo di risolvere la dipendenza 'Microsoft.Bcl (≥ 1.0.19)'.
Tentativo di risolvere la dipendenza 'Microsoft.Bcl.Build (≥ 1.0.14)'.
Tentativo di risolvere la dipendenza 'Microsoft.Bcl.Async (≥ 1.0.16)'.
Tentativo di risolvere la dipendenza 'Microsoft.Net.Http (≥ 2.1.10)'.
Tentativo di risolvere la dipendenza 'Zlib.Portable (≥ 1.9.2)'.
Tentativo di risolvere la dipendenza 'log4net (≥ 2.0.0)'.
Tentativo di risolvere la dipendenza 'Google.Apis.Auth (≥ 1.8.1)'.
Installazione di 'Google.Apis.Storage.v1beta2 1.8.1.430'.
Installazione di 'Google.Apis.Storage.v1beta2 1.8.1.430' completata.
Aggiunta di 'Google.Apis.Storage.v1beta2 1.8.1.430' a Prova.
Aggiunta di 'Google.Apis.Storage.v1beta2 1.8.1.430' a Prova completata.

Original comment by giordano...@noovle.it on 22 Apr 2014 at 3:06

[GoogleCodeExporter]

Sorry, you are right 43 is the latest.

It's really really weird.
Just to confirm:

1. The test passes
2. You still get the error

Right?

On Tue, Apr 22, 2014 at 11:07 AM,
<google-api-dotnet-client@googlecode.com>wrote:

Original comment by pele...@google.com on 22 Apr 2014 at 3:26

[GoogleCodeExporter]

right!

Giordano

Original comment by giordano...@noovle.it on 22 Apr 2014 at 3:36

[GoogleCodeExporter]

I am not an dot net expert but other tests are needed I am avaialble to make 
them.

Your help is very appreciate.

Giordano

Original comment by giordano...@noovle.it on 22 Apr 2014 at 3:39

[GoogleCodeExporter]

It's weird.
Can you meantime work with an object that doesn't contain /?

On Tue, Apr 22, 2014 at 11:39 AM,
<google-api-dotnet-client@googlecode.com>wrote:

Original comment by pele...@google.com on 22 Apr 2014 at 3:42

[GoogleCodeExporter]

yes,

I tested without /and I have no problem to download an object now I am 
preparing a new machine/environment to see if the problem is my machine.

I am developing a very simple application for a customer with millions of 
objects.

Giordano

Original comment by giordano...@noovle.it on 22 Apr 2014 at 3:50

[GoogleCodeExporter]

Thanks! Keep me updated

Original comment by pele...@google.com on 22 Apr 2014 at 4:11

[GoogleCodeExporter]

Hi,

good news, I tested with dotnet versionn 4.5 and version 4.5.1 and the problem 
doesn't occur.

Giordano 

Original comment by giordano...@noovle.it on 23 Apr 2014 at 10:52

[GoogleCodeExporter]

So the problem occurred in .NET 4.0 only?
Our tests are written in 4.0, so I don't understand how it's possible.

Original comment by pele...@google.com on 1 May 2014 at 4:21

[GoogleCodeExporter]

Ping... can we close it?

Original comment by pele...@google.com on 8 May 2014 at 2:11

[GoogleCodeExporter]

sure, the problem is fixed but only with dotnet 4.0, I tried tomorrow with last 
version (v1) but with the same result.

Giordano

Original comment by giordano...@noovle.it on 8 May 2014 at 2:33

[GoogleCodeExporter]

I'm confused.
Does it work in .NET 4.0? Yes\No
Does it work in .NET 4.5? Yes\No

Thanks

Original comment by pele...@google.com on 8 May 2014 at 3:32

[GoogleCodeExporter]

NET 4.0? No
NET 4.5? Yes

Original comment by giordano...@noovle.it on 8 May 2014 at 3:39

[GoogleCodeExporter]

So weird.. Because the tests that are working in your case, are written in .NET 
4.0

Original comment by pele...@google.com on 8 May 2014 at 4:15

[GoogleCodeExporter]

Original comment by pele...@google.com on 20 May 2014 at 4:04

• Removed labels: Milestone-Release1.8.2

[GoogleCodeExporter]

Giordano, been struggling with the same problem myself. Indeed it happens only 
in .NET4, it has do with a security feature from Microsoft that unescaped the 
URL. 
It only occurs in objects that contains characters like '/', probably this is 
the reason why it didn't happen in Eyal's tests.
I just posted about it in my blog: http://blog.techedup.net/gce-dotnet4/

You should change the web.config, add:
<uri>
<schemeSettings>
<add name="https" genericUriParserOptions="DontUnescapePathDotsAndSlashes" />
</schemeSettings>
</uri>

Mashav. 

Original comment by mas...@techedup.net on 10 Jul 2014 at 1:21

[GoogleCodeExporter]

Cool! I'm going to share your post soon.
We still need to investigate why the error response that you got "was so
informative".

Thanks!

On Thu, Jul 10, 2014 at 9:22 AM, <google-api-dotnet-client@googlecode.com>
wrote:

Original comment by pele...@google.com on 10 Jul 2014 at 5:56

[GoogleCodeExporter]

Original comment by pele...@google.com on 28 Jul 2014 at 12:50

• Changed state: WontFix