Implement Object operations for Spaces API

[amoeba]

Description

This implements the first half of the operations listed in https://github.com/sckott/analogsea/issues/140 which are the Spaces API operations related to Objects

Related Issue

https://github.com/sckott/analogsea/issues/140

No tests yet, since all these tests require external calls. See discussion in https://github.com/sckott/analogsea/issues/140

[amoeba]

Thanks for the comments. Will update this PR.

[sckott]

@amoeba sorry for delay - getting to looking this over hopefully today

[sckott]

@amoeba is space_create working for you?

I'm getting an error on every try, checked that my env vars are there.

SO with verbose curl output i'm seeing

space_create('new_space_name', config=httr::verbose())
-> PUT /new_space_name/ HTTP/1.1
-> Host: nyc3.digitaloceanspaces.com
-> User-Agent: libcurl/7.54.0 r-curl/3.1 httr/1.3.1
... cut out some stuff
<- HTTP/1.1 400 Bad Request
<- Date: Thu, 25 Jan 2018 02:37:06 GMT
<- Content-Length: 153
<- Content-Type: text/plain; charset=utf-8

so the URL seems like nyc3.digitaloceanspaces.com/space-name

whereas in creating one in the DO platform, and the API docs say the same, looks like the URL should be of the form

space-name.nyc3.digitaloceanspaces.com

thoughts? does it work for you?

[sckott]

I couldn't get spaces_object_delete to work. I kept getting a The specified bucket does not exist error, even though the bucket defiinitely existed. work for you?

[amoeba]

Thanks for taking a look. I really appreciated the letting us have a go at this and the helpful comments.

I don't think I have ever been able to space_create working. I assumed it was just still in progress. But I think spaces_object_delete was working fine. I'll test now.

[amoeba]

Hrm, spaces_object_delete worked fine for me just now:

> spaces_object_delete("GDP.csv", "test-analogsea")
[1] TRUE

[sckott]

Glad delete works. Wonder if they changed how spaces create works since that fxn was originally made here?

[amoeba]

I wonder. I'll take a look at it.

[amoeba]

Looks like a limitation of aws.s3. In setup_s3_url, which appears to generate the URLs used in put_bucket (which creates new S3 buckets), it hard-codes this logic:

# handle S3-compatible storage URLs
    if (base_url != "s3.amazonaws.com") {
        if (isTRUE(verbose) && url_style != "path") {
            message("Non-AWS base URL requested. Switching to path-style URLs.")
        }
...

which looks like the cause of our problem here.

[sckott]

ah, that could be it, right, i assume then it adds the bucket/space name after the base url

[amoeba]

yup

[sckott]

bump @amoeba , are you waiting on me to do anything here?

[amoeba]

Hey @sckott thanks for the bump. This can be merged if you think so. I looked over all of the comments here and in the related issue and I think it's good to go.

[sckott]

okay, will take another quick look

[sckott]

@amoeba can you add man-roxygen to .Rbuildignore

[amoeba]

Done!

[sckott]

thanks for the rbuildignore fix.

hmm, did you see my comment about space_create not working? https://github.com/sckott/analogsea/pull/146#issuecomment-360343859

[amoeba]

I did. I was thinking that, since this PR is just for the Object API operations that I/we'd work on that elsewhere. It'll require either an upstream aws.s3 fix or moving away from using aws.s3. Is that okay?

[sckott]

right, forgot about that. yep, we'll deal with fixing create later.