Support for following Lemmy Communitites

[rcarmo]

Hi there,

I think it would be great to be able to follow Lemmy communities via Takahe. I've already tried adding @3DPrinting@lemmy.world to my follows on my test instance, but it seems to blow up due to schema validation when receiving the initial update:

wsgi.1   | 2001:818:e801:c000:79f9:2bd7:d402:dd67 - - [19/Jun/2023:12:57:18 +0000] "POST /api/v1/accounts/193604962194743690/follow HTTP/2.0" 200 528 "-" "Ivory/14100 CFNetwork/1408.0.4 Darwin/22.5.0" 32ms
wsgi.1   | 2001:818:e801:c000:79f9:2bd7:d402:dd67 - - [19/Jun/2023:12:57:18 +0000] "GET /api/v1/accounts/familiar_followers?id=193604962194743690&limit=16 HTTP/2.0" 200 294 "-" "Ivory/14100 CFNetwork/1408.0.4 Darwin/22.5.0" 14ms
worker.1 | Attempting transition on users.identity#152165363235847338 from state outdated
worker.1 | Attempting transition on users.identity#153015956477014738 from state updated
worker.1 | Attempting transition on users.identity#140816249298612706 from state updated
worker.1 | Attempting transition on users.follow#193605039960699596 from state unrequested
wsgi.1   | 135.181.143.230 - - [19/Jun/2023:12:57:23 +0000] "GET /@rcarmo@insightful.systems/ HTTP/2.0" 200 2955 "-" "Lemmy/0.17.4; +https://lemmy.world" 20ms
worker.1 | Traceback (most recent call last):
wsgi.1   | 135.181.143.230 - - [19/Jun/2023:12:57:23 +0000] "GET / HTTP/2.0" 200 29592 "-" "Lemmy/0.17.4; +https://lemmy.world" 57ms
worker.1 |   File "/home/piku/.piku/apps/takahe/stator/models.py", line 234, in atransition_attempt
worker.1 |     next_state = await current_state.handler(self)  # type: ignore
worker.1 |   File "/home/piku/.piku/apps/takahe/users/models/identity.py", line 118, in handle_outdated
worker.1 |     if await identity.fetch_actor():
worker.1 |   File "/home/piku/.piku/apps/takahe/users/models/identity.py", line 891, in fetch_actor
worker.1 |     await sync_to_async(self.save)()
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/asgiref/sync.py", line 448, in __call__
worker.1 |     ret = await asyncio.wait_for(future, timeout=None)
worker.1 |   File "/usr/lib/python3.10/asyncio/tasks.py", line 408, in wait_for
worker.1 |     return await fut
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/asgiref/current_thread_executor.py", line 22, in run
worker.1 |     result = self.fn(*self.args, **self.kwargs)
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/asgiref/sync.py", line 490, in thread_handler
worker.1 |     return func(*args, **kwargs)
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/django/db/models/base.py", line 814, in save
worker.1 |     self.save_base(
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/django/db/models/base.py", line 877, in save_base
worker.1 |     updated = self._save_table(
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/django/db/models/base.py", line 990, in _save_table
worker.1 |     updated = self._do_update(
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/django/db/models/base.py", line 1054, in _do_update
worker.1 |     return filtered._update(values) > 0
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/django/db/models/query.py", line 1231, in _update
worker.1 |     return query.get_compiler(self.db).execute_sql(CURSOR)
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1982, in execute_sql
worker.1 |     cursor = super().execute_sql(result_type)
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1547, in execute_sql
worker.1 |     sql, params = self.as_sql()
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1945, in as_sql
worker.1 |     val = field.get_db_prep_save(val, connection=self.connection)
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/django/db/models/fields/__init__.py", line 954, in get_db_prep_save
worker.1 |     return self.get_db_prep_value(value, connection=connection, prepared=False)
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/django/db/models/fields/__init__.py", line 947, in get_db_prep_value
worker.1 |     value = self.get_prep_value(value)
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/django/db/models/fields/__init__.py", line 1122, in get_prep_value
worker.1 |     return self.to_python(value)
worker.1 |   File "/home/piku/.piku/envs/takahe/lib/python3.10/site-packages/django/db/models/fields/__init__.py", line 1112, in to_python
worker.1 |     raise exceptions.ValidationError(
worker.1 | django.core.exceptions.ValidationError: ["“{'type': 'xsd:boolean', '@value': True}” value must be either True, False, or None."]
worker.1 | Attempting transition on users.identity#153015956477014738 from state outdated
worker.1 | Attempting transition on users.identity#140816249298612706 from state outdated
worker.1 | Attempting transition on users.follow#193605039960699596 from state local_requested
wsgi.1   | 135.181.143.230 - - [19/Jun/2023:12:57:34 +0000] "POST /inbox/ HTTP/2.0" 202 261 "-" "Lemmy/0.17.4; +https://lemmy.world" 23ms
worker.1 | Attempting transition on users.inboxmessage#90203 from state received
worker.1 | Running cleaning and scheduling
worker.1 | Tasks processed since last flush:
worker.1 |   users.identity: 5
worker.1 |   users.follow: 2
worker.1 |   users.inboxmessage: 1

[andrewgodwin]

Yeah, I'll have to go capture some example on-the-wire communication and then figure out what the mismatch is.

[osmaa]

I believe this is it (redacting rest of the identity for personal data):

'toot:discoverable': {'type': 'xsd:boolean', '@value': True},

[andrewgodwin]

I've also seen a bunch of complaints from our json-ld parser saying that there's an ID declared where it shouldn't be - as of yesterday, any incoming messages that we can't parse are shunted to a new errored state rather than being retried for ages, so at least there's now a way to easily see in the django admin everything that Takahē can't understand.

[osmaa]

this, or at least the most common case of difference in tag encoding, may have been fixed by #596