Getting somewhere with OAuth and Mastodon

[scripting]

Where I'm at.

1. I have a test Mastodon server up.

2. I have a bridge server app I can access from a web browser.

3. I have registered an app with Mastodon.

4. I have made an oauth/authorize call and get back a code. (Took a lot of trial and error!)

The next step is not working.

I am getting a 404 when I call oauth/token on the same server.

Here's what the call looks like:

https://social.scottfr.ee/oauth/token?grant_type=authorization_code&client_id=xxx&client_secret=xxx&redirect_uri=http://scripting.com/&scope=read+write+follow&code=xxx

These are the docs I'm using.

https://docs.joinmastodon.org/methods/apps/oauth/

I feel like I'm almost there, but missing something.

Any help appreciated.

[scripting]

I asked the question on Masto and got the answer. It needed to be a POST, and I used GET.

https://mastodon.social/@danijel@mastodon.green/109365501836831605

Now I'm past that error and on to the next one. It's such a slog but once you get it it tends to stay got. ;-)

"{"error":"invalid_grant","error_description":"The provided authorization grant is invalid, expired, revoked, does not match the redirection URI used in the authorization request, or was issued to another client."}"

I also figured out that I can run the bridge code in the debugger so things are going much faster now.

[scripting]

Bing!

I fiddled around with things and got the access token. So now, theoretically I should be able to post something to our test Masto.

[scripting]

But I'm stuck again. The redirect_url that I send to the "oauth/token" call -- nothing is happening with it.

I end up with the access token on the server, but nothing on the client. It should be getting something it can save in localStorage to use to make future calls.

Nothing like that is happening..

[scripting]

I probably need another fresh start to get over this hump. It feels like the last one, but so did the previous one. ;-)

Someday someone is going to figure out how to make this not take a week.

[jaseg]

Im unsure where your issue stems from, but here are two to me unintuitive observations that I made messing about with mastodons oauth api last week.

1. the redirect url must exactly match the callback url that you registered where you got the app secret and app id.
2. Throughout the oauth redirect, not all browsers preserve client-side http session cookies. Session ids seem to work fine, but if you use a web framework that persists other data in the session cookie, any new data you store shortly before the redirect may be gone afterwards. I think that is because of some tracking prevention feature.

hope that helps :)

[Riff451]

Hi, I'm not sure if I understand this correctly however with OAuth2 nothing is supposed to happen with the redirect_uri when you call the token endpoint. With the auth_code OAuth flow that you're using, what you get in the redirect_uri is just the auth_code that you can exchange for an access_token as you did. If you've got the access_token on the server, it's now a server responsibility to use it. I'm not sure about your current architecture though so I'd need to have a look. :) I hope that helps a little bit.

https://www.rfc-editor.org/rfc/rfc6749#section-4.1.3

[scripting]

Thanks for the help, I’m taking a break to clear my head and my eyes, I’m sure I’ll get it, I’ve had the same thing going with Twitter for years. 😀

[scripting]

I got it. There was a segment of the docs that I had skimmed over when starting, and hadn't gone back to. See screen shot.

So now I have it returning a code to the browser, I assume this will be what I send to the server to post something on behalf of a user.

[scripting]

I'll clean all this up for an upload tomorrow assuming all goes well from here. ;-)

[scripting]

To get my Hello World working, I'm looking at the docs for statuses.

https://docs.joinmastodon.org/methods/statuses/

I can't believe that the stuff about polls is required. I don't want a poll. I just want to post "Hello World". Wft.

I better pick this up tomorrow, I'm overloaded. ;-)

[scripting]

I'm still wandering around in the dark. ;-)

But I am getting somewhere.

[scripting]

Bing! Bing! Bing!

https://www.youtube.com/watch?v=8lejTOhQ59I

[scripting]

Well, I thought I was going to be able to get a Hello World status to post quickly, but I don't know where everything goes. Do I have to send the token as a header. Can't tell for sure.

Why doesn't someone do a freaking Hello World example. The code for the API packages really try to hide the basic stuff that's going over the line, so much ridiculous abstraction, there really isn't very much going on here.

Anyway I'm going to take a nap and a walk, and then come back. I want two basic calls to work and then I'll clean it up and upload the result.

1. Get info about the user, esp their handle. This is what we need the most, but I expect there's other info we can get from Mastodon.

2. Post a hello world message, no polls, no media enclosures. just Hello World.

[scotthansonde]

@scripting I'm not sure it applies to our case, but I found an article on posting a new status on the command line over the api. The curl command (not my real token 😃) was

curl https://social.scottfr.ee/api/v1/statuses  
-H 'Authorization: Bearer WkidfdaswnshRxOeK9FarWcm4K3AnOe1gIwiMg-8C7zTY'  
-F 'status=Is this CLI thing turned on?'

I got some JSON back and the status was posted.

{
  "id": "109371582297744484",
  "created_at": "2022-11-19T17:11:43.714Z",
  "in_reply_to_id": null,
  "in_reply_to_account_id": null,
  "sensitive": false,
  "spoiler_text": "",
  "visibility": "public",
  "language": "en",
  "uri": "https://social.scottfr.ee/users/scott/statuses/109371582297744484",
  "url": "https://social.scottfr.ee/@scott/109371582297744484",
  "replies_count": 0,
  "reblogs_count": 0,
  "favourites_count": 0,
  "edited_at": null,
  "favourited": false,
  "reblogged": false,
  "muted": false,
  "bookmarked": false,
  "pinned": false,
  "content": "\u003cp\u003eIs this CLI thing turned on?\u003c/p\u003e",
  "filtered": [],
  "reblog": null,
  "application": { "name": "Testing CLI", "website": null },
  "account": {
    "id": "109346902734680487",
    "username": "scott",
    "acct": "scott",
    "display_name": "Scott Hanson",
    "locked": false,
    "bot": false,
    "discoverable": false,
    "group": false,
    "created_at": "2022-11-15T00:00:00.000Z",
    "note": "",
    "url": "https://social.scottfr.ee/@scott",
    "avatar": "https://social.scottfr.ee/avatars/original/missing.png",
    "avatar_static": "https://social.scottfr.ee/avatars/original/missing.png",
    "header": "https://social.scottfr.ee/headers/original/missing.png",
    "header_static": "https://social.scottfr.ee/headers/original/missing.png",
    "followers_count": 1,
    "following_count": 1,
    "statuses_count": 5,
    "last_status_at": "2022-11-19",
    "noindex": false,
    "emojis": [],
    "fields": []
  },
  "media_attachments": [],
  "mentions": [],
  "tags": [],
  "emojis": [],
  "card": null,
  "poll": null
}

[scripting]

@scotthansonde — very helpful. I’ll get this in the code and let you know how it goes.

can you find a way to get info about the user?

[scripting]

@scotthansonde -- I need to do a fresh start in the morning. I used the example you provided but something isn't working. It's saying the token I provided isn't valid. That says that the process before trying to post a message has a problem. And it's been a long day. ;-)

[scripting]

We're at the start of another day and still don't have Hello World working. But I think it's really close.

Okay -- so here's where we are, quickly.

I can get the four crucial bits of information you get back from Mastodon when you successfully connect.

{
    "access_token": "redacted",
    "created_at": 1668788407,
    "scope": "read write follow",
    "token_type": "Bearer"
    }

But when I try to use the access_token to post a status, I get back an error saying:

{"error":"This method requires an authenticated user"}

Here's the code I'm using to post the status message.

function postStatus (theMessage, callback) {
    $.ajax ({
        url: "https://social.scottfr.ee/api/v1/statuses",
        type: "POST",
        headers: {
            Authorization: "Bearer " + mastodonMemory.access_token
            },
        data: {
            status: theMessage
            },
        dataType: "json"
        })  
    .success (function (data, status) { 
        callback (undefined, data);
        }) 
    .error (function (status) { 
        callback (err);
        });
    }

PS: Thanks to @scotthansonde for his help and support -- he cruises through stuff in a way I don't, and his input has been very helpful.

[scripting]

So, first thing -- if you have working code that posts a status message, esp if it's JavaScript, could you look at my code above and see if there's anything that might make Masto think that I'm not including the bearer token as it expects it? I'm going to look for examples. I have to believe the access token the server gave me is good.

I'm also going to look for actual JS code that posts a status to see if there are any other ways I might do this.

I'm also going to start over on the server with a new app, and set it up again, just to review that side of it. See if the problem shakes out that way.

Thanks for any help you can offer. :smile:

[ivan3bx]

I don't have JS code, but trying that status API in curl, I noticed it requires form data on the way in (not JSON, as it looks like you're providing based on dataType). Here's a quick test (I'm using a public mastodon instance running v4.0.2. and weirdly, my 'failing' case doesn't return a response payload, so your "This method requires an authenticated user" might be unrelated to this.

# Fails (HTTP/1.1 400 Bad Request)
curl https://${MY_HOST}/api/v1/statuses \
  -X POST \
  -H "Authorization: Bearer ${MY_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{status: "test json"}' -i

# Succeeds (HTTP/1.1 200 OK + json response)
curl https://${MY_HOST}/api/v1/statuses \
  -X POST \
  -H "Authorization: Bearer ${MY_TOKEN}" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d 'status=test form' -i

[scotthansonde]

I found an old library mastodon.js that also uses $.ajax to post statuses. It has a generic post function.

post: function (endpoint) {
            // for POST API calls
            var args = checkArgs(arguments);
            var postData = args.data;
            var callback = args.callback;
            var url = apiBase + endpoint; // apiBase = config.instance + "/api/v1/"

            return $.ajax({
                url: url,
                type: "POST",
                data: postData,
                headers: addAuthorizationHeader({}, config.api_user_token),
                success: onAjaxSuccess(url, "POST", callback, false),
                error: onAjaxError(url, "POST")
            });
        },

With post("statuses",{status:"test from mastodon.js"}, function (data) {console.log(data)}) I was able to successfully post from the browser console.

[scripting]

@scotthansonde -- there's a bunch of stuff in there that's defined externally to the routine like checkArgs, arguments.

Some things we can guess what they are -- but the mystery is what they're passing as data, exactly. I can't find any examples, but it's hard to construçt a search for this stuff because the terms are so generic.

Could you send me the full file via email?

[scripting]

never mind i see you linked to it ;-)

[scripting]

I'm wiped out. Exhausted and need a fresh/fresh start this time. But I'm going to do that after I get this working.

Here's my theory -- Mastodon is insisting that the user doesn't have an token. The error seems to imply that it isn't getting the token, but I'm going on the theory that the message is slightly misleading. Possibly it means "I got the token but it's no good."

So here's what I'm doing. I'm going to stop for now trying to get the code to work. Instead I'm going to get it easily testable, online, and release the source. And then others can join me in testing the same code. And I'm going to start with that assumption that the problem isn't wirh the code that's telling Masto to do a post, rather it's that it's not got a good token.

Back in a bit...

[lrdj]

I'm wiped out. Exhausted and need a fresh/fresh start this time. But I'm going to do that after I get this working.

Ya know, you're like the only person out there trying hard to keep the web working the we all benefitted from it years ago. Keep at it, you're doing awesome. Sending respect and a hug:-)

[scripting]

@lrdj -- thank you. i'm doing it because there's a huge block of new code working that needs to see the rest of the world and the world needs to see it. Feeds are able to do a lot more than people think they can. The pairing betw feeds and Masto is going to be a big boom. I can't wait. That's why I'm pushing. ;-)

[billstclair]

{
    "access_token": "redacted",
    "created_at": 1668788407,
    "scope": "read write follow",
    "token_type": "Bearer"
    }

function postStatus (theMessage, callback) {
  $.ajax ({
      url: "https://social.scottfr.ee/api/v1/statuses",
      type: "POST",
      headers: {
          Authorization: "Bearer " + mastodonMemory.access_token
          },
      data: {
          status: theMessage
          },
      dataType: "json"
      })  
  .success (function (data, status) { 
      callback (undefined, data);
      }) 
  .error (function (status) { 
      callback (err);
      });
  }

First a nit. In the second form above: Authorization: "Bearer " + mastodonMemory.access_token should really be Authorization: mastodonMemory.token_type + " " + mastodonMemory.access_token. I've never see anything but "Bearer" as the token_type, but you never know.

I don't see anything wrong with your code, but I always do GET /api/v1/accounts/verify_credentials before anything else, just to make sure all is well. I think that would be (untested code below; it might not even load):

function getVerifyCredentials (callback) {
    $.ajax ({
        url: "https://social.scottfr.ee/api/v1/accounts/verify_credentials",
        type: "GET",
        headers: {
            Authorization: mastodonMemory.token_type + " " + mastodonMemory.access_token
            },
        })  
    .success (function (data, status) { 
        callback (undefined, data);
        }) 
    .error (function (status) { 
        callback (err);
        });
    }

If you look at this in the "Network" tab of your browser's DevTools window, you should see something like the following request & headers sent (some might be missing. I don't know which are necessary, I just copied this from my "Network" tab):

General
  Request URL: https://impeccable.social/api/v1/accounts/verify_credentials
  Request Method: GET
  Status Code: 200 
  Remote Address: 157.230.215.62:443
  Referrer Policy: strict-origin-when-cross-origin

Request Headers
  :authority: impeccable.social
  :method: GET
  :path: /api/v1/accounts/verify_credentials
  :scheme: https
  accept: */*
  accept-encoding: gzip, deflate, br
  accept-language: en-US,en;q=0.7
  authorization: Bearer BVpNTD...
  origin: https://mammudeck.com
  referer: https://mammudeck.com/
  sec-fetch-dest: empty
  sec-fetch-mode: cors
  sec-fetch-site: cross-site
  sec-gpc: 1
  user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36

And there should be an OPTIONS request sent over the wire, to do the CORS stuff.

Once you get that working, in your original postStatus function, the data type may need to be application/json, not just json, but it's possible that $.ajax prepends application/ for you.

I haven't explained why you're getting {"error":"This method requires an authenticated user"}. Maybe the simpler example, and looking at the "Nework" tab, will help.

You can watch this all happen in the network tab, when sending requests through https://mammudeck.com/?api=login

[scripting]

I got the code mostly cleaned up, and there are current versions of the client and server here:

https://github.com/scripting/reallySimpleActivityPub/tree/main/mastoGlue

So now you can look at both sides of the connection.

[scripting]

Also, no pull requests.

http://scripting.com/2020/05/26/194558.html?title=bugReportsNotPullRequests

[scripting]

I've cleaned up the server side, the test app is now running at this address.

http://test.masto.land/

And the error I'm getting when I try to do a hello world is now a CORS error. Why it should be any different for this domain than scripting.com, well that's a mystery.

I think I'm going to do what I did for Twitter, is build glue scripts that run on the server and call those from the browser software, that way CORS never comes into it.

Also to @billstclair -- I made the changes you suggested, and will upload them to the mastoGlue folder in this repo.

[billstclair]

test.masto.land always connects to https://social.scottfr.ee. I requested an account, but I'm waiting for it to be approved. It would be nice to be able to specify the server.

I don't know what's up with CORS. When I can see it in my browser, I may have an idea. Your idea of funneling requests through the server, which doesn't need CORS, will work, but I'd never do it that way. But I'm weird. My webapps generally don't HAVE a server-side, except to fetch the web page containing the JavasScript.

It also just occurred to me that this repository isn't named right. Not that that really matters, but...

ActivityPub is the name of the server-to-server synchronization interface. This code is not doing that; it's talking the Client API. I've never looked at ActivityPub, and unless you're writing code for a Mastodon instance, which talks the Client API to clients and the AcivityPub API to other instances, you won't need to.

[scripting]

i fixed the problem with CORS.

also i know this thread isn't about activitypub.i did set up another repo about mastodon and we'll switch over to that once the freaking Hello World script works.

read the code, if you have time, if you see anything obviously wrong that might help.

done for the day, see you manana, murphy-willing.

[scotthansonde]

@billstclair https://social.scottfr.ee is a test instance I set up for @scripting and I. I've approved your account, so now there's 3 of us. 😃

[billstclair]

I'm not seeing where the call to https://social.scottfr.ee/oauth/token happens. This turns a ?code into an access token ( It must be from your server at http://dave.masto.land/getaccesstoken?code=r1H..., called by getAccessToken in your code at https://github.com/scripting/reallySimpleActivityPub/blob/main/mastoGlue/home/code.js#L206. My "Network" tab doesn't show me the return value from that, but it must be happening, since it gets to Local Storage.

The server code appears to be the getAccessToken function at https://github.com/scripting/reallySimpleActivityPub/blob/main/mastoGlue/server/mastoserver.js#L89. That sends the token request to the Mastodon server, whose url appears to be stored in a local file, config.json. I don't know why you don't just have the client code send the token request, but it appears to work, for now. If you keep it on the server, you'll need to pass the Mastadon server URL in http://dave.masto.land/getaccesstoken?code=r1H..., not get it from a config file on the server.

In the POST to http://social.scottfr.ee/api/v1/statuses generated by postStatus at https://github.com/scripting/reallySimpleActivityPub/blob/main/mastoGlue/home/code.js#L114, I'm getting a CORS error. The OPTIONS preflight is getting a 301 from the Mastodon server. It should get 200. That causes the error, printed in the Dev Console: "Access to XMLHttpRequest at 'http://social.scottfr.ee/api/v1/statuses' from origin 'http://test.masto.land' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: Redirect is not allowed for a preflight request."

I comparing the OPTIONS preflights from http://test.masto.land and https://mammudeck.com/?api=login, I see the OPTIONS requests as in the linked screenshots below. My guess is that some of the "Request Headers" in the mammudeck.com request are missing from the test.masto.land request, but I don't know which ones are important. Your code didn't generate this, $.ajax() did (I think), so I don't know how you'd fix it. Maybe $.ajax() has a parameter to tune CORS, but I don't see it at https://api.jquery.com/jquery.ajax/. Maybe you need to request from https://social.scottfr.ee/api/v1/..., not http://social.scottfr.ee/api/v1/... (https, not http).

Mammudeck CORS OPTIONS request: https://user-images.githubusercontent.com/40873/202993527-3aa1a180-be15-4036-82f8-cc80ca7c9ac6.jpg test.masto.land CORS OPTIONS request: https://user-images.githubusercontent.com/40873/202994113-59376601-3d73-4994-87f7-968ebeef85a6.jpg

I noticed one bug in your code. In the postStatus function, it's adding the header: Authorization: mastodonMemory.token_type + mastodonMemory.access_token (https://github.com/scripting/reallySimpleActivityPub/blob/main/mastoGlue/home/code.js#L119). This neglects to put a space between "Bearer" and the token. It's done correctly in verifyCredentials: Authorization: mastodonMemory.token_type + " " + mastodonMemory.access_token

Good luck. I have no idea why I'm up at 3am, but I am (I DID sleep 4 hours, so all is not lost).

[billstclair]

Thanks, Scott. I’m in, and was able to do some testing of Dave’s Mastodon Client API code.

On Nov 21, 2022 at 2:37:45 AM, Scott Hanson @.***> wrote:

@billstclair https://github.com/billstclair https://social.scottfr.de is a test instance I set up for @scripting https://github.com/scripting and I. I've approved your account, so now there's 3 of us. 😃

— Reply to this email directly, view it on GitHub https://github.com/scripting/reallySimpleActivityPub/issues/4#issuecomment-1321576437, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAJ7KKXCUXJIDAE466EAMLWJMRETANCNFSM6AAAAAASESRTRE . You are receiving this because you were mentioned.Message ID: @.***>

[Riff451]

Hi, I'm not sure but this part doesn't look quite right to me:

https://github.com/scripting/reallySimpleActivityPub/blob/83bf6480f897ff8e1d0b93085cd4ef7379da41d2/mastoGlue/server/mastoserver.js#L93-L98

If you use the OAuth authorization code flow (as you're doing) the grant type should be authorization_code and not client_credentials. This is maybe what's causing the access token error but to be fair I've not tried to run your code locally. :)

[scotthansonde]

In the POST to http://social.scottfr.ee/api/v1/statuses generated by postStatus at https://github.com/scripting/reallySimpleActivityPub/blob/main/mastoGlue/home/code.js#L114, I'm getting a CORS error. The OPTIONS preflight is getting a 301 from the Mastodon server. It should get 200.

Yes, the nginx web server is configured to redirect all http requests to https.

server {
  listen 80;
  listen [::]:80;
  server_name social.scottfr.ee;
  root /home/mastodon/live/public;
  location /.well-known/acme-challenge/ { allow all; }
  location / { return 301 https://$host$request_uri; }
}

[scripting]

@Riff451 -- that was the problem. I am now able to post to the test instance. Hello World.

I worked around the CORS issue by putting the code that sends the message to the instance in the server app, which doesn't have any CORS restrictions.

Whew! And thank you. ;-)

[scripting]

I updated the code in the mastoland folder.

https://github.com/scripting/reallySimpleActivityPub/tree/main/mastoGlue

There's a lot more work to do, but hopefully now it will go more steadily since we have a template for communicating with Masto.

[scripting]

I've done the factoring that was needed to get ready to add support for a lot of other verbs.

I have two verbs working --

1. mastodon.toot (statusMessage)

2. mastodon.getUserInfo ()

Both use the access_token we have via logging on and it all works as far as I can tell.

You can try it out by going to http://test.masto.land/. To enter a new status message, click on the Masto Toot button and enter the text you want to toot. Click OK, it should appear on the test server. The result will be displayed in the JavaScript console. Any errors will appear in a dialog box.

You can test the second one by opening the JavaScript console and type:

testGetUserInfo ()

You should see a bunch of info about the user in the console.

When you've done this please confirm here.

I have updated the source in this repo.

[scripting]

Next up, I want to upload a picture with a toot.

[scotthansonde]

You can try it out by going to http://test.masto.land/.

To try it out now you need to have an account on the test instance https://social.scottfr.ee/. I've opened up sign-in for new accounts so people can try it out. Just be aware that it is a test instance that will most likely be deleted when this project is complete.

[scripting]

oops sorry about that scott. ;-)

[scotthansonde]

@scripting I can confirm that both mastodon.toot () (via the Masto Toot button) and mastodon.getUserInfo () (in the console) both work for me, returning their results as JSON in the console.

[scripting]

Never mind. I figured it out.

I'm testing some code and have a question about any limits on what characters can be in the text of a status.

If I ask the API to toot this text, it does it, no errors.

Prosecution Rests as Trump Company Trial Moves Faster Than Expected

But if I add a # and a space at the beginning I get a 422 error.

# Prosecution Rests as Trump Company Trial Moves Faster Than Expected

Does anyone here know anything about that?

[scripting]

Tune into the test Masto server.

I got something fun running. When you see the code you'll plotz.

There's a FeedLand feature most people don't know about. :smile:

[billstclair]

Tune into the test Masto server.

I got something fun running. When you see the code you'll plotz.

There's a FeedLand feature most people don't know about. 😄

I don't see it. Your last commit to http://test.masto.land was seven hours ago. Are you talking about https://social.scottfr.ee? I don't see anything there, either.

[scripting]

I haven't released any new code.

The stories flowing through there from the NYT is what (at least I find) interesting.

I will have new code out tomorrow probably.

[billstclair]

Ah. I saw the posts, but I didn't know that they came from an RSS feed. Cool!

[andysylvester]

I created an account on social.scottfr.ee, made a manual toot, then made a toot from test.masto.land and saw the toot on my account on social.scottfr.ee.

[voitto]

I made an account on social.scottfr.ee and went to test.masto.land to make an API-toot - worked nicely!