Chocobozzz / PeerTube

ActivityPub-federated video streaming platform using P2P directly in your web browser
https://joinpeertube.org/
GNU Affero General Public License v3.0
13.08k stars 1.51k forks source link

Followers disappearing after upgrade to 6.0 #6131

Closed FediVideos closed 10 months ago

FediVideos commented 10 months ago

Describe the current behavior

I'm posting on behalf of instance owner https://peertube.lon.tv/a/lonseidman/video-channels, they have recently upgraded to PeerTube 6.0 and their followers started disappearing. They were on around 100 followers or so before the upgrade?

Steps to reproduce

(Not applicable)

Describe the expected behavior

The follower numbers should only drop if people unfollow. As far as I can tell, people have not been unfollowing.

Additional information

I can request logs from owner if it would be useful?

kodxana commented 10 months ago

From what I see from my instance it look like lon.tv has some issues

peert-peertube-1  |     "stack": "Error: Cannot fetch remote actor https://peertube.lon.tv/accounts/peertube\n    at getOrCreateAPActor (file:///app/dist/core/lib/activitypub/actors/get.js:18:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async addVideoShare (file:///app/dist/core/lib/activitypub/share.js:43:19)\n    at async Bluebird.map.concurrency (file:///app/dist/core/lib/activitypub/share.js:27:13)",
peert-peertube-1  |     "message": "Cannot fetch remote actor https://peertube.lon.tv/accounts/peertube"
peert-peertube-1  |   }
peert-peertube-1  | }

I would check if private/public keys are there as I had similar errors after upgrade to 6.0

  1. Check database for missing keys:

    select * from actor where "preferredUsername" = 'peertube' and "serverId" is null;
  2. If there are missing privkeys you want generate them

create js script example script.js

const options = {
  modulusLength: 2048,
  publicKeyEncoding: {
    type: 'spki',
    format: 'pem'
  },
  privateKeyEncoding: {
    type: 'pkcs1',
    format: 'pem'
  }
}

require('crypto').generateKeyPair('rsa', options, (err, publicKey, privateKey) => {
  if (err) return console.error(err)

  console.log('Public key')
  console.log(publicKey)
  console.log('Private key')
  console.log(privateKey)
})

After you run this it should give priv pub keys

  1. Add key back to database
UPDATE actor
SET "publicKey" = 'your_public_key_string', "privateKey" = 'your_private_key_string'
WHERE id = 1;

That should fix federation.

Chocobozzz commented 10 months ago

Hi,

@kodwana is right: we can't follow the [peertube.lon.tv](https://peertube.lon.tv/) instance and the database manipulation should fix that.

But I think there's a different issue there, because https://peertube.lon.tv/a/lonseidman account doesn't have a missing public key. It may be related to https://github.com/Chocobozzz/PeerTube/issues/6113 and the upgrade to Node 18, where PeerTube can't do requests to followers and so remove them after multiple request failures.

Can you ask the admin to provide some logs?

FediVideos commented 10 months ago

Thanks both for answers, I've messaged the instance owner and asked if they can supply logs.

lonseidman commented 10 months ago

Hi everyone thanks for offering to help figure this out. I am happy to provide logs, just let me know which logs you're looking for.

Also of note - I am running Peertube via Docker.

-Lon

kodxana commented 10 months ago

Hi everyone thanks for offering to help figure this out. I am happy to provide logs, just let me know which logs you're looking for.

Also of note - I am running Peertube via Docker.

-Lon

I also run via Docker. Btw you can reach me via Matrix if you want to: @madiator2011:matrix.madiator.dev

Chocobozzz commented 10 months ago

Hi everyone thanks for offering to help figure this out. I am happy to provide logs, just let me know which logs you're looking for.

Please paste warn logs from Administration -> System -> Logs (select Standard Logs).

You can also paste failed Local Jobs error (activitypub-http-broadcast or activitypub-http-unicast jobs for example) to see what is the root cause of the federation problem

lonseidman commented 10 months ago

I have a bunch of these, looks like there's a key it's missing:

warn[12/27/2023, 8:13:05 AM] Cannot process activity View. { "err": { "stack": "RequestError: options.key must be a sshpk.PrivateKey\n at Request._beforeError (file:///app/node_modules/got/dist/source/core/index.js:310:21)\n at Request.flush (file:///app/node_modules/got/dist/source/core/index.js:299:18)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at Object.signRequest (/app/node_modules/@peertube/http-signature/lib/signer.js:362:14)\n at got.extend.Object.assign.hooks.beforeRequest (file:///app/dist/core/helpers/requests.js:49:35)\n at Request._makeRequest (file:///app/node_modules/got/dist/source/core/index.js:942:34)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async Request.flush (file:///app/node_modules/got/dist/source/core/index.js:285:13)", "message": "options.key must be a sshpk.PrivateKey", "name": "RequestError", "requestHeaders": { "user-agent": "PeerTube/6.0.2 (+https://peertube.lon.tv)", "date": "Wed, 27 Dec 2023 13:13:05 GMT", "accept": "application/activity+json, application/ld+json", "accept-encoding": "gzip, deflate, br", "host": "neat.tube" } } }

kodxana commented 10 months ago

I have a bunch of these, looks like there's a key it's missing:

`warn[12/27/2023, 8:13:05 AM] Cannot process activity View.

{

"err": {

"stack": "RequestError: options.key must be a sshpk.PrivateKey\n    at Request._beforeError (file:///app/node_modules/got/dist/source/core/index.js:310:21)\n    at Request.flush (file:///app/node_modules/got/dist/source/core/index.js:299:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at Object.signRequest (/app/node_modules/@peertube/http-signature/lib/signer.js:362:14)\n    at got.extend.Object.assign.hooks.beforeRequest (file:///app/dist/core/helpers/requests.js:49:35)\n    at Request._makeRequest (file:///app/node_modules/got/dist/source/core/index.js:942:34)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async Request.flush (file:///app/node_modules/got/dist/source/core/index.js:285:13)",

"message": "options.key must be a sshpk.PrivateKey",

"name": "RequestError",

"requestHeaders": {

  "user-agent": "PeerTube/6.0.2 (+https://peertube.lon.tv)",

  "date": "Wed, 27 Dec 2023 13:13:05 GMT",

  "accept": "application/activity+json, application/ld+json",

  "accept-encoding": "gzip, deflate, br",

  "host": "neat.tube"

}

}

}`

Yes that what i suspected you got private key issue one I got.

kodxana commented 10 months ago

@lonseidman so if you run peertube inside docker you need to ssh into pogester container for your instance

docker exec -it [name of your postgres docker container] /bin/bash In my case I had it

Once you get terminal from container you need to open psql tool:

psql -d DATABASENAME -U username example: psql -d peertube -U madiator (this one usually is set in dockerf-compose file)

Then I recomend sending this command first (it makes output more readable in terminal \x

You will get infor that Expanded display is on

Then you want to check for channels with missing private public keys:

select * from actor where "preferredUsername" = 'peertube' and "serverId" is null;

You might want to save id for channel without keys.

If any channel has key missing you need to generate priv public key

with the script I have shared https://github.com/Chocobozzz/PeerTube/issues/6131#issuecomment-1865193213 just make sure to set correct user ID

WHERE id = 1;

kodxana commented 10 months ago

@lonseidman Looks like there are still issues :)

{
  "payload": {
    "host": "peertube.lon.tv:443",
    "name": "peertube",
    "followerActorId": 1
  },
  "err": {
    "stack": "Error: Cannot fetch remote actor https://peertube.lon.tv/accounts/peertube\n    at getOrCreateAPActor (file:///app/dist/core/lib/activitypub/actors/get.js:18:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async Object.processActivityPubFollow [as activitypub-follow] (file:///app/dist/core/lib/job-queue/handlers/activitypub-follow.js:23:23)\n    at async Object.wrapPromiseFun (file:///app/dist/core/lib/plugins/hooks.js:8:24)\n    at async Worker.processJob (/app/node_modules/bullmq/dist/cjs/classes/worker.js:350:28)\n    at async Worker.retryIfFailed (/app/node_modules/bullmq/dist/cjs/classes/worker.js:535:24)",
    "message": "Cannot fetch remote actor https://peertube.lon.tv/accounts/peertube"
  }
}
lonseidman commented 10 months ago

Yeah I haven't had time to address this.. Hopefully next week.

On Thu, Jan 11, 2024 at 1:27 AM kodxana @.***> wrote:

@lonseidman https://github.com/lonseidman Looks like there are still issues :)

{ "payload": { "host": "peertube.lon.tv:443", "name": "peertube", "followerActorId": 1 }, "err": { "stack": "Error: Cannot fetch remote actor https://peertube.lon.tv/accounts/peertube\n at getOrCreateAPActor (file:///app/dist/core/lib/activitypub/actors/get.js:18:19)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async Object.processActivityPubFollow [as activitypub-follow] (file:///app/dist/core/lib/job-queue/handlers/activitypub-follow.js:23:23)\n at async Object.wrapPromiseFun (file:///app/dist/core/lib/plugins/hooks.js:8:24)\n at async Worker.processJob (/app/node_modules/bullmq/dist/cjs/classes/worker.js:350:28)\n at async Worker.retryIfFailed (/app/node_modules/bullmq/dist/cjs/classes/worker.js:535:24)", "message": "Cannot fetch remote actor https://peertube.lon.tv/accounts/peertube" } }

— Reply to this email directly, view it on GitHub https://github.com/Chocobozzz/PeerTube/issues/6131#issuecomment-1886706178, or unsubscribe https://github.com/notifications/unsubscribe-auth/AORPSHE6TVXLM7WVW3ZI7FLYN6WAZAVCNFSM6AAAAABA5II33OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBWG4YDMMJXHA . You are receiving this because you were mentioned.Message ID: @.***>

--

Lon Seidman 860-577-0566 @.***

lonseidman commented 9 months ago

So I finally had some time to tackle this - as expected the keys were missing from the database record which I generated and inserted. I've recycled the Docker containers too - hopefully this rectifies it!

kodxana commented 9 months ago

Make sure to check if keys are there if not you need to generate them manually

lonseidman commented 9 months ago

Yes I generated them - if you can test and see if we're able to connect properly that would be helpful!