After today, authenticateWithToken() hangs, produces error.

[Kaidesa]

Ever since the service went down last night, there's been an issue with token authentication. I've done some digging around and it has to do with the response generated by the actual API response.

    async authenticateWithToken(token) {
        if (this.isAuthenticated()) throw Error('Already authenticated');
        if (!token || typeof(token) != 'string') throw Error('Specify a valid token');

        await this.requester.initialize();
        const request = await this.requester.request('https://beta.character.ai/dj-rest-auth/auth0/', {
            method:'POST',
            body:Parser.stringify({
                access_token: token
            }),
            headers:{
                'Content-Type': 'application/json',
            }
        })
        // ...

This will hang for quite a while, unless a subsequent response is sent, where it will crash with a timeout. If left alone for a while, the error is as follows.

        if (request.status() === 200) {
                    ^

TypeError: Cannot read properties of undefined (reading 'status')
    at Client.authenticateWithToken (C:\***\node_modules\node_characterai\client.js:186:21)

[realcoloride]

Hello, I will investigate the issue.

[realcoloride]

Hello, You should try closing all remainings of Chromium instances and setting headless to false temporarily and telling me how it goes

[Kaidesa]

So, once it hits that point, it just sits at the main beta.character.ai website with a very dark (since there's no manual control allowed) screen showing a cookie notification on the website (asking you to click Accept)

It just sits there for seemingly forever, and once the error appears in the terminal, the browser window closes.

Nothing else happens.

I've tried silly things like clearing the cache in Puppeteer out, uninstalling the dependency and reinstalling, remove node_characterai, completely nuking all private browsing data from the system as well as my personal browser, and nothing seems to fix it.

[Parking-Master]

I have been getting the same problem since yesterday.

[Parking-Master]

59 this might be related, but no answer either

[Parking-Master]

@kaidesa the first error you mentioned must be from the request timing out, and not returning the request variable. Which is why it is returning that error that "request" is not defined. So this is definitely from the browser never loading. Basically the problem always leads back to the page not loading on chromium. Let me check to see what the problem is.

[Kaidesa]

@Parking-Master So, that seems to be the thing, yeah. Except the issue happens at that precise call, every time. Oddly enough, connecting as a guest seems to work, but it errors when trying to start or find a conversation.

Did something just to make sure it wasn't something locally messing things up. Uploaded a known working copy to one of my test environments which is off site and the issue persists. Tried doing the same to my PC at home, and the issue yet again persisted.

I'm almost willing to bet that part of the major downtime that was experienced with Character.AI was them changing how something worked, because ever since that spot of downtime they experienced, this has been an issue. Not a fluke that the same thing happens across three different machines on three entirely different networks.

[Parking-Master]

@kaidesa think I found the problem.

node_modules/node_characterai/client.js:175

const request = await this.requester.request('https://beta.character.ai/dj-rest-auth/auth0/', {
  method: 'POST',
  body: Parser.stringify({
    access_token: token
  }),
  headers: {
    'Content-Type': 'application/json',
  }
})

On that request for https://beta.character.ai/dj-rest-auth/auth0/, if you change it to something random (like google.com) the page loads and there is no timeout.

The reason for the page not loading is because the POST request to that URL (https://beta.character.ai/dj-rest-auth/auth0/) is never responding.

I've also noticed that if you use authenticateAsGuest() instead of authenticateWithToken() the API works perfectly fine.

So with this, I found out the problem, and you are correct: Character.AI's official company changed something.

The token in the POST request appears to be invalid even know it worked a few days ago just fine.

[Parking-Master]

So in other words, the project is basically dead until @realcoloride finds a way to authenticate Character.AI because they changed their method where you used to just be able to POST the token and get logged in.

[Kaidesa]

So I checked another project that supposedly uses Character.AI to have a Discord bot do the do.

https://github.com/drizzle-mizzle/CharacterAI.Net-API-Wrapper https://github.com/drizzle-mizzle/CharacterAI-Discord-Bot

Apparently, the method they use for logging in requires the key that's posted from the /auth0 endpoint, which is much smaller.

Mayhaps it'll give @realcoloride a place to look. Their bot is reported as working today, and the project was last updated 11 hours ago as of the writing of this post, which places it after the time of the changes Character.AI made.

[Parking-Master]

@kaidesa thanks for referencing that repository! When he sees this, he should be able to fix it pretty fast. I've done some more research and can't find much more so that should help to fix the issue.

[Parking-Master]

It seems this is also related to #57, and in that issue he said he would patch a fix soon as possible. Just give it some time for now and he should fix it soon.

[realcoloride]

Hello again and thank you massively for the resources I think too that they have changed something in their endpoint.

Proven by the past, CAI have a random tedency to update things here & there.. I will investigate the issue.

[realcoloride]

Ok so I've checked via my browser atleast

the endpoint seems to be https://beta.character.ai/dj-rest-auth/auth0/

Is some kind of header required?

[Parking-Master]

@realcoloride don't think so (except for the "Content-Type" header). In the code I've shown from "client.js" there is just a body with the parameter "access_token".

[realcoloride]

I swear this endpoint gotta be the weirdest. I think there is something to do with the chromium instance itself, maybe some kind of launch parameter is required, but it seems like there was no clear change in how the the endpoint works.

It can be either an issue related to Cloudflare with some kind of restrictions or the chromium instance. EDIT: judging by how it should have atleast received an error or the cloudflare page, maybe the chromium instance has something to do with it.

[Parking-Master]

It could be the browser like you said, because I did a test URL and posted to "https://httpbin.org" and that does the same thing (times out). This is definitely a strange issue, because I didn't update the npm package and chromium didn't update so it had to have been CAI changing some stuff.

There's a chance it is the browser's fault though, especially like you said with cloudfare changing their restrictions.

[realcoloride]

what could be a viable solution then?

[Parking-Master]

@realcoloride it's definitely possible because the login on their website works. Maybe you could look into that, I tried but I can't find anything. Maybe you could try catching any POST requests on the page and see what the URL of the request is, and if you get the auth0 url then try using that in the request instead

[Parking-Master]

And also what @Kaidesa said, there is a Character.AI discord bot that uses this API and it was reported as working still. Try checking there too

https://github.com/drizzle-mizzle/CharacterAI.Net-API-Wrapper https://github.com/drizzle-mizzle/CharacterAI-Discord-Bot

[Parking-Master]

Ok LOL, I found a solution but still don't know what the problem is. Must be because of old site data from before C.AI updated their API or something.

1. Go to _node_modules/nodecharacterai/requester.js 2. Go to line 10 and set "headless" to "false" (as a Boolean, not string) 3. Save the file and run your script again. 4. When Chromium opens up, Type "CMD+Alt+I"/"Ctrl+Shift+I" In the current page 5. Wait until the console opens up, then navigate to "Application > Storage" 6. Click "Clear site data" then exit chromium 7. Set line 10 of requester.js as "new" (string) or "true" (boolean) 8. Save the file and start your script.

By the fact that clearing storage data is all it takes, it was definetely a problem with chromium.

[realcoloride]

Nice stuff!

Do you think there is a way to do that directly via code instead of doing it manually?

[Parking-Master]

@realcoloride indeed there is!

Using puppeteer, add this to line 103 in requester.js right after this.page = page:

const client = await page.target().createCDPSession();
await client.send('Network.clearBrowserCookies');
await client.send('Network.clearBrowserCache');

This will clear all browser data right before starting the session.

You could also try adding this to the code in a new commit

[realcoloride]

Could you try it? I will push the commit soon if it works correctly

[Parking-Master]

I already have tried it, works like a charm :)

[Kaidesa]

Just tried it myself and can confirm this works fine. Great stuff, you two!

Do find it odd, however, that manually deleting the cache from the userdir didn't fix the issue the first time,. Not complaining, mind you. Just odd.

[Parking-Master]

@Kaidesa thanks! I urgently needed this project too so might as well opt-in to help. The problem should be globally fixed once the commit goes through.

[realcoloride]

Hello I've pushed commits https://github.com/realcoloride/node_characterai/commit/910cee0658e0e7842af8b58006bf1f79fc426c14 and https://github.com/realcoloride/node_characterai/commit/ec8a4f97ed68ed4739bf96bc19c6f3d559f7d7be

I've pushed the update as of 1.1.6, so please let know how it goes.

[Parking-Master]

@realcoloride thank you for updating it. I will try it again when I get the chance, I'm pretty sure it would work

[Kaidesa]

@realcoloride @Parking-Master So, while clearing it manually works fine, I ran the code on a second machine after updating the installed version of node_characterai. The updated code would work, except it can't with the old browser data in tact.

As it stands, anyone affected by this issue would have to do the manual fix regardless. Likely just an order of operations issue here. I won't be in front of the last machine with issues to test my theory for about an hour, though. I'll check my proposed fix as soon as I'm able and report back.

[Parking-Master]

@Kaidesa strange, seems to work fine after re-installing it on my Mac.

[Kaidesa]

@realcoloride @Parking-Master So, apparently I was mistaken. It WILL work, but apparently it requires a full uninstall and reinstall of the node_characterai package. I'm not sure if it's just a weird fluke on my end, but this is the results.

Machine 1: Manually fixed the issue, so couldn't test. Running the package works as intended. Machine 2: Just updating node_characterai didn't fix the issue. Because of the above code, it appeared to try loading the page prior to clearing the cache. Workaround: Load another page, clear the cache via code, then load the requested page. Worked fine. Machine 3: Updated package. No dice. Uninstalled package, reinstalled package, and it worked.

Either way, the code works fine now, but you may want to throw in a notice that deleting and reinstalling might be the solution if a straight update doesn't work.

[realcoloride]

Thank you for more clarity and the time you spent on it!

I will probably make a section in the FAQ of the README.md file if needed.

[Parking-Master]

@Kaidesa like realcoloride said, thank you for clarifying that! Even if the update doesn't work for future users, they can simply clear the data manually which is pretty easy. Either way, it works lol.

[realcoloride]

Do you have a clear guide of instructions for people passing by that may be experiencing issues from older versions? like how to clear the data?

If needed I'll specify it in the README.md

[Parking-Master]

@realcoloride from the list I gave earlier, that should work with all the versions. If you meant older versions of browsers, you will need to check the chromium docs to see how to clear the data in earlier versions

[realcoloride]

I meant as of old package versions, but thank you again o7

[Parking-Master]

No problem! If a future user's package version is older then they should update it to the latest version. They would also get more benefits like better security, etc.

[Parking-Master]

Hello! I fixed the problem again with:

page.deleteCookie();

Right after this line:

const client = await page.target().createCDPSession();
await client.send('Network.clearBrowserCookies');
await client.send('Network.clearBrowserCache');

In requester.js line 108.

I did not have to manually clear the data through the dev console after I put that line there.

Let me know if there are any more problems!

[realcoloride]

Hello, could you explain to me whats going on?

[Parking-Master]

@realcoloride instead of editing the whole documentation and for future users to go through the hassle of clearing the site data manually, you could try following those steps and see if it works. I don't know why the previous method I wrote doesn't clear the site data already but this code works

[realcoloride]

Issue solved in #68, see https://github.com/realcoloride/node_characterai/issues/68#issuecomment-1637297081

Please re-open an issue if the problem persists.

[Parking-Master]

Thanks, I knew this would solve the problem :D