Error creating user group

[mkosmala]

I'm trying to create a user group, but am getting a HTTP Error 415: Unsupported Media Type. Here's my code:

#!/usr/bin/python
from urllib2 import Request, urlopen
import subprocess

# this gets my bearer token
token_process = subprocess.Popen(['./get_bearer_token.sh'],shell=False,stdout=subprocess.PIPE)
process_response = token_process.communicate()
bearer_token = process_response[0].rstrip()

# parameters for POST
host = "https://panoptes-staging.zooniverse.org/api/"
headers = {"Accept": "application/vnd.api+json; version=1", "Authorization": "Bearer "+bearer_token}
values = """
    {
        "user_groups": {
            "display_name": "Project Seasons"
        }
    }
    """

# Make the request
request = Request(host+"groups",values,headers)
response_body = urlopen(request).read()
print response_body

[camallen]

add the content-type header to let the system know you're posting application/json.

[mkosmala]

Oh right. So it should be:

headers = { "Content-Type": "application/vnd.api+json", "Accept": "application/vnd.api+json; version=1", "Authorization": "Bearer "+bearer_token}

?

[camallen]

almost

headers = { "Content-Type": "application/json", "Accept": "application/vnd.api+json; version=1", "Authorization": "Bearer "+bearer_token}

[mkosmala]

Hmm. Now I'm getting HTTP Error 400: Bad Request. Only change is that I changed the headers line so it looks like yours, Cam.

[camallen]

I just copied yours but added the json content type header. Here are the headers from my script that i sent you.

#used for non versioned api routes, e.g. /users/sign_up, etc 
json_accept='Accept: application/json'
#used to access the versioned api routes
json_versioned_accept='Accept: application/vnd.api+json; version=1'
#used to specify json content type payloads
json_content_type='Content-Type:application/json'

What route are you posting that to?

[mkosmala]

Oh, should I be using the json_accept or the json_versioned_accept for user group creation? (And how do I know which to use for different things? I don't know what "versioned" and "non-versioned" means.)

I don't really know what a "route" is. I'm sending the request to https://panoptes-staging.zooniverse.org/api/groups

[mkosmala]

Um, now I'm getting HTTP Error 422: Unprocessable Entity. I don't know what's changed between this and the Error 400 above... hmmm.

#!/usr/bin/python
from urllib2 import Request, urlopen
import subprocess

# get bearer token
token_process = subprocess.Popen(['./get_bearer_token.sh'],shell=False,stdout=subprocess.PIPE)
process_response = token_process.communicate()
bearer_token = process_response[0].rstrip()

# parameters
host = "https://panoptes-staging.zooniverse.org/api/"
headers = {"Content-Type": "application/json","Accept": "application/vnd.api+json", "Authorization": "Bearer "+bearer_token}
values = """
    {
        "user_groups": {
            "display_name": "Project Seasons"
        }
    }
    """

# do request
request = Request(host+"groups",values,headers)
response_body = urlopen(request).read()
print response_body

[camallen]

looks like the bearer token is capture the whole output of the ./get_bearer_token.sh script.

#change your bearer token to this
bearer_token = process_response[0].rstrip().split('\n')[-1]

#and i'd update the headers to be more readable

# Make the request
request = Request(host+"groups",values)

#manually set the headers
request.add_header("Content-Type", "application/json")
request.add_header("Accept", "application/vnd.api+json; version=1")
request.add_header("Authorization", "Bearer "+bearer_token)
#request.add_header("other_headers", "other header values")

#DEBUG HEADERS AND REQUEST
#print request.get_full_url()
#print request.get_method()
#print request.header_items()
#print dir(request)  # list lots of other stuff in Request

#carry on as normal

[mkosmala]

Really? It looks fine when I print it. And it worked fine for getting my user ID... I'll give it a go, though...

[camallen]

If you don't change it just print the header debug statements I sent you to see what's being sent. You'll get a very large string in the Authorization bearer token....

[camallen]

For waht it's worth i just created a user group on my dev machine with the modifications i made to your script.

[mkosmala]

Where is this script with your modifications? On my machine, I get the same output with these two print statements:

bearer_token = process_response[0].rstrip()
print "|"+bearer_token+"|\n"
print "|"+bearer_token.split('\n')[-1]+"|\n"

I'm not calling the original bearer_token script you sent me, but rather a modified version of it. I am able to use the bearer_token successfully to get a user. So I don't think it's the token that's the issue.

Will work on the headers... one moment.

[camallen]

I just sent the modifications as code like 5 comments above...

Ok - you've modified the output of the curl script I sent you. I did try and modify the curl scrip to be silent except for the output of the bearer token but you can't gaurantee what curl outputs across platforms / versions of curl. Instead the best way is to just get the last line of output from the bash script (if that's where your token is) hence the bearer_token = process_response[0].rstrip().split('\n')[-1]

[mkosmala]

Here's what I'm getting as output from the four debug print lines. Is this helpful?

https://panoptes-staging.zooniverse.org/api/groups

POST

[('Content-type', 'application/json'), ('Accept', 'application/vnd.api+json; version=1'), ('Authorization', 'Bearer 9f6b0a6dd385be3f96ffeb04fdaab812284aa72478866916afdd46d17a36e1b1')]

['_Request__fragment', '_Requestoriginal', 'doc__', 'getattr', 'init', 'module', '_tunnel_host', 'add_data', 'add_header', 'add_unredirected_header', 'data', 'get_data', 'get_full_url', 'get_header', 'get_host', 'get_method', 'get_origin_req_host', 'get_selector', 'get_type', 'has_data', 'has_header', 'has_proxy', 'header_items', 'headers', 'host', 'is_unverifiable', 'origin_req_host', 'port', 'set_proxy', 'type', 'unredirected_hdrs', 'unverifiable']

[camallen]

Yep that looks right, now what's the request response.

[mkosmala]

HTTP Error 400: Bad Request

[camallen]

I modifed your payload to this, perhaps try that. Also can you send me the timestamp when you run the code next time. I can't see any 400's in the staging logs.

values = """
    {
        "user_groups": {
            "name": "Project Seasons",
            "display_name": "Project_Seasons"
        }
    }
    """

[mkosmala]

Success! I changed it to:

values = """
    {
        "user_groups": {
            "name": "project_seasons",
            "display_name": "Project Seasons"
        }
    }
    """

since my understanding from the API docs is that the name is the lower-case underscored version of the display_name.

[edpaget]

Cool this is an issue that should be fixed in the docs or code (either one I feel would be acceptable).

I think I miswrote the docs to indicate the only display_name was required and, if only that was sent, name would set to the downcased underscored version of it.

At the moment what the code actually does is set display_name equal to name unless it is already present. @mkosmala or @camallen which behaviour do think should actually be correct?

[mkosmala]

I think the former is more user friendly. But either would be fine from a user's perspective.

[edpaget]

Great added a new issue I'll work on https://github.com/zooniverse/Panoptes/issues/283

[camallen]

I think both should be in the docs but with a note saying name can override the display_name. Note display_name is the one that should get created and cleaned of any spaces, etc as it'll be used for the URL's.

Mostly this issue was due to the script not handling HTTP response objects correctly as we include the error information in the HTTP 400 error response (see below for modified script as per python docs). The nice output from the script below says

The server couldn't fulfill the request.
Error code:  400
Error response body:  {"errors":[{"message":"Validation failed: Owner name name can't be blank, Name can't be blank"}]}

#!/usr/bin/python
from urllib2 import Request, urlopen, URLError, HTTPError
import subprocess

# this gets my bearer token
token_process = subprocess.Popen(['./get_bearer_token'],shell=False,stdout=subprocess.PIPE)
process_response = token_process.communicate()
bearer_token = process_response[0].rstrip().split('\n')[-1]

# parameters for POST
host = "https://panoptes-staging.zooniverse.org/api/"
values = """
    {
        "user_groups": {
            "display_name": "Project Seasons"
        }
    }
    """

# Make the request
request = Request(host+"groups",values)
request.add_header("Content-Type", "application/json")
request.add_header("Accept", "application/vnd.api+json; version=1")
request.add_header("Authorization", "Bearer "+bearer_token)
#request.add_header("other_headers", "other header values")

#print request.get_full_url()
#print request.get_method()
#print request.header_items()

try:
  response = urlopen(request)
except HTTPError as e:
    print 'The server couldn\'t fulfill the request.'
    print 'Error code: ', e.code
    print 'Error response body: ', e.read()
except URLError as e:
    print 'We failed to reach a server.'
    print 'Reason: ', e.reason
else:
  # everything is fine
  body = response.read()
  print body

[mkosmala]

Oh, agree with Cam then. If display_name is meant for URLs, it needs to be cleaned of spaces. I had thought that 'display_name' was for the users' benefit and 'name' was used by the system. Whichever way it is, it would be nice to only have to specify one name for the user group -- the one that the user will see on-screen -- and have the other created automatically.

[edpaget]

I think cam has it backwards re name vs display_name.

[camallen]

Yep - i had it backwards.