Open API

[charlotte-chiang]

High level: Permit community contributions for integrations, plugins, imports

Related Threads/Feature Requests: https://community.anytype.io/t/future-plugin-extension-api-ideas/1397 https://community.anytype.io/t/how-will-should-the-plugins-system-work-for-developers-and-users/2485 https://community.anytype.io/t/plugin-architecture-discussion/6790 https://community.anytype.io/t/live-export-plugin/7179/9

[Narvey]

This is huge to me and more important than any other single item on the roadmap!

[Dorifor]

I'd love to see that, opens many opportunities for contributions !

[derekbelrose]

Would your interpretation of an Open API allow for automation or is this purely for plugins? I'm playing around with a home project management setup so the family can quickly fill out a form on a web page and it creates a new project in my anytype setup.

[waldentoo]

@derekbelrose How can that be done? Do you mind going over that? I really like the form idea. could the form do appending or even adding a new relationship?

[derekbelrose]

@waldentoo If the client were documented and could be recreated in a self-hosted server application that was accessible by an API, any application that could call the API could create objects and add them to the store...

Then a javascript application that talks directly to the API and runs in a browser would take form input and do the rest.

Granted this wasn't fully thought out yet, but I'm curious if it's feasible to build out a limited functionality client that could respond to HTTP calls and create objects.

[hassanzouhar]

Really you guys had me at interlinked human knowledge... but an OpenAPI would make this kill like a dozen other apps for me.

[derekbelrose]

I just think that if this is supposed to be the operating system for my life, I should be able to tailor how I get data into it. Automation is key to me getting things accomplished.

[hassanzouhar]

I just think that if this is supposed to be the operating system for my life, I should be able to tailor how I get data into it. Automation is key to me getting things accomplished.

Agree, but its on the roadmap so we can only give them time to properly implement it now :) Really looking forward to it :)

[timolins]

Big +1 to go beyond the the Anytype app when it comes to the API. Besides plugins I'd love to be able to query/write data via:

• Either a self-hosted REST API
• An API package. Think npm install anytype

import anytype from "anytype"

const store = antype.auth("SECRET")

store.createObject()
store.getObjects()

That would be insanely powerful if you want to build on top of Anytype. E.g. I want to build my own journaling app, but still want it's data to be accessible in AnyType.

[Narvey]

An API package. Think npm install anytype

Sounds great as long as it can be accessed from other languages (or no language like curl) too.

I'd also like to see the ability to use the API offline, for instance, by having the API be run by the app at localhost.

[jbriales]

So this is in the roadmap, and that is great! But do we get visibility into the planned feature and capabilities beforehand? I guess it's going to end up being part of the opensource repos as well, right? So it'd be great to be able to get an early perspective of the feature, chime in and weigh in from the community side to make sure the most important capabilities for users (and especially developers/contributors) are prioritized.

[fuksman]

@jbriales Yes, this will be a part of our published repositories, just like everything else we do. I also think that in September, we will start publishing our ideas/proposals about features involving contributors in our GitHub Discussions to gather comments and ideas from the community as early as possible. Stay tuned! :)

[LaTrissTitude]

Hello, same here, an API to import, export and interact with text content and tasks would be a game changer.

[aadilayub]

Since AnyType is local-first, I'm wondering how the API would work.

Like, if I'm writing a request to pull data from AnyType, what server would that request be going to? My understanding is that AnyType is p2p, so there isn't a central server to pull data from.

[fuksman]

@aadilayub, you're right, the local-first approach and encrypted external infrastructure pose challenges for implementing APIs. We see two approaches for different use cases here:

• API at the local client level, which could help create integrations with external services and will only work when the client is running.
• Anytype-heart node, a sort of headless client, which could be deployed next to any-sync nodes (on Anytype infrastructure or your self-hosted network) and used for continuously running bots to automate data reading or writing in your spaces.

If you have any other suggestions or ideas, please share.

[aadilayub]

That makes sense!

Regarding the first option, it might be worth looking at the implementation in Logseq..

fireSeqSearch is a good example of a tool built on top of Logseq's local HTTP server.

There's also nbb-logseq which is a tool for querying Logseq's local database (from what I can tell, this doesn't rely on logseq's HTTP server).

I have no idea if there are any structural similarities between Logseq and AnyType, but it felt worth mentioning these examples since Logseq is also a local-first knowledge management app.

[xtagon]

I'm really excited about this feature more than anything. It's the one part of Anytype that doesn't fit well with my idea development process (yet!)

Example use cases that I would want to develop plugins for myself:

• One-click to start a build process to publish my website based based on a filtered set of public pages

• One-click exports that generate source code skeletons from anyblocks relating my software design notes.

• Domain-specific rendering for custom types in pages (and icon in the graph) (e.g. I could implement a Chess Game type that renders the chess board, and relate that to following chess positions that I'm analyzing, just one example)

• Ability to paste things in and parse various custom file types (for example I could make a plugin where if you paste in a IRI it fetches it, determines if it's RDF, and imports RDF graph as anyblocks into wherever you specifically pasted or drag/dropped)

• Custom type transform actions (e.g. a plugin where with the click of a button I can transform a subgraph of anyblocks into another graph of anyblocks, or a single object into another object of a different type, in ways specific to the types I use)

Notably, for my use cases, plugins should be able to depend on and call queries or actions in other plugins, as well as standard libraries for things like HTTP and filesystem access. For example, my transform plugin would be used by my web publishing plugin in order to convert the data, and then it might call a transform plugin that generates HTML from the anyblocks, and publishing to AWS Amplify, etc.

If either Javascript or Webassembly are supported, then the sky is the limit.

[swarnimarun]

I can finally start using something other than logseq if this becomes available even as a local plugin api. I just need something to connect to my feeds and note taking apps.

[5p4r74cu5]

Anytype combines all the features I've been looking for in one application, except for customisability, so this is the only feature stopping me from migrating everything to Anytype. I wouldn't be suprised if this finally pulls alot of people away from Obsidian.

[Grygon]

This would be my number 1 thing I'd need to make it worth the hurdle of switching, assuming it's a reasonable implementation it would (hopefully) help me tie in a lot of workflows, external data sources, etc that I'm struggling to do properly with logseq today.

Are there any details about what this will look like? Just a local server an external program can make requests against? Or something more advanced like Trilium Scripts?

[developerisnow]

Can't wait hearings about it!

[jreus]

Would love this! I'm especially interested in developing my own custom set/collection views. None of the standard ones ( Graph, List, etc.. ) really do what I need - which is to be able to handle Tasks that have accompanying notes ( e.g. a Checkbox/Todo followed by a text block ) .... I'm trying to make linear timelines / work logs from such objects.

Would be very happy also to contribute any new set/collection view formats back to the community however possible.

[mifraburneo]

So sad this has been pushed so far into the year 🤕 I'd love to contribute to this feature, although it'd be my first open source project to work in...

I think the "headless client" that @fuksman suggested should be the way to go... maybe on a docker container, that would be awesome!

[stefanku]

I’m sad to read that this is pushed to Q4. An API would be an absolute game changer and turn Anytype into a truly interactive space to manage anything.

[fraschm1998]

I’m sad to read that this is pushed to Q4. An API would be an absolute game changer and turn Anytype into a truly interactive space to manage anything.

I completely agree, I think the main barrier preventing users from adopting anytype is the lack of plugins or AI integrations. Having an API will hugely increase community contributions and further anytype development

[derekbelrose]

I completely agree, I think the main barrier preventing users from adopting anytype is the lack of plugins or AI integrations. Having an API will hugely increase community contributions and further anytype development

It's why I have not adopted this fully. Until I can automate some stuff using an API, I can't use it day to day.

[Coriakin]

I completely agree, I think the main barrier preventing users from adopting anytype is the lack of plugins or AI integrations. Having an API will hugely increase community contributions and further anytype development

It's why I have not adopted this fully. Until I can automate some stuff using an API, I can't use it day to day.

Same here. I am sure there is a good reason for postponing it, but it truly is an enabler for so many other things that could accelerate AT traction as well as setting it apart from other competitors in a very clear and beneficial way. A local Notion/Capacities/etc but where you have deep control over your data; to automate, bulk update/create and even extract data from; that will be a game changer.

[fuksman]

Hey everyone, we hope to introduce some part of API capabilities much earlier. However, as this feature has slightly lower priority than others, we don't think we'll be able to fully focus on it until the end of the year.

[jbriales]

I understand that it's hard to shuffle priorities. Still, considering the Open-Source direction Anytype adopted (which makes it IMO stand out from the competition) I'm a bit surprised (disappointed?) that sth like an API hasn't higher priority.

Also, considering the answer I got half a year ago, I'm surprised sth like starting a more serious brainstorming effort and publishing some initial ideas didn't happen yet at all (AFAIK). There are a lot of ambiguity and mystery about "some API things that will be done" but I couldn't see any details on those plans to contribute or give early feedback:

@jbriales Yes, this will be a part of our published repositories, just like everything else we do. I also think that in September, we will start publishing our ideas/proposals about features involving contributors in our GitHub Discussions to gather comments and ideas from the community as early as possible. Stay tuned! :)

I wish there would be some more structured discussion/material to refer to around this at this point...

[aapjeisbaas]

I didn't read through all the comments and proposals but maybe a webhook trigger based on a set + filter could be an easy way to trigger external systems based on local changes.

[derekbelrose]

My initial intent is to do the opposite. I’d like external systems to be able to affect data in AnyType. The reason for this is I want to have external input such as a web form kick off a pipeline that ultimately ends up with documentation within AnyType.  On Mar 12, 2024, at 6:01 AM, Stein van Broekhoven @.***> wrote: I didn't read through all the comments and proposals but maybe a webhook trigger based on a set + filter could be an easy way to trigger external systems based on local changes.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

[anzenketh]

I am excited for this to be implemented. Hopefully it is for what I think it is. Plugins (Which API is a first step to having) was a game changes in a lot of ways for integrating other digital systems. For example loading notes from systems like https://omnivore.app/ or pulling notes from kindle or moon reader. Or creating a system that pops up a window that asks questions in a different way then what AnyType supports but data can be stored as a value.

Basically a API will allow for some edge case use cases to be fulfilled without relying upon the Developers of AnyType to implement integrations.

[ebanDev]

I can't wait for this system to be implemented it's the last obstacle for me to fully adopt anytype !

[A-Z-X-R]

This will be what would finally get hooked up into Anytype, if possible. I love my Logseq/Obsidian workflow(Yes, I use both), but I crave my Anytype sort of writing style (and being able to drag my blocks anywhere I want, and object types is goated fr). The only reason I'm not leaving Logseq is because:

• No Vim support on Anytype
• No Anki sync
• No Excalidraw
• Outlining. While it does get in the way if it get's forced all over the place like Logseq, it's useful in journalling

[micklat]

Everybody want plugins and I do too, but I'm concerned that rushing this without giving due considerations to security will lead quickly to the failed app security model that is prevalent in computing today, where basically any plugin can do anything and it is up to users to decide which plugins should be trusted. That's a bad outcome because ordinary users can't reasonably be expected to vet every plugin they install (and I assure you they won't do anything of the sort). This sets them up for exploitation by malware authors, or creates a high vetting burden on the plugin repository maintainers.

If anybody wants to discuss this further, and work with me towards making plugins safer, please join the discussion here, or contact me directly, on community.anytype.io.

[hellodword]

In my opinion, language-based security may hurt the enthusiasm of plugin contributors because they would have to learn a new language (a pure language is always hard to learn, for me).

I prefer a sandbox-based plugin system that defines all the boundaries, exposes common APIs, without being concerned about the plugin languages. I'm not saying the language is unimportant; I think WebAssembly (Wasm) is a good choice for the sandbox runtime. This way, people can use Python, JavaScript, Java, Go, or whatever they prefer, and all the source code will be compiled to Wasm.

Actually, even with a language designed for security, I believe a sandbox is absolutely required for the real-world to mitigate the worst situations and ensure we can sleep well at night.

(@micklat Sorry for replying here, but I do not want to to register one more discourse...)

[hellodword]

I searched around and found that Extism could be a good choice.

• It implements a kernel in Rust and builds it to kernel.wasm, then loads the kernel.wasm in the Wasm runtime. I think we can build a very secure plug-in system on it.
• It provides an official Go SDK (non-CGO), with very few dependencies, which anytype-heart will love.
• This means that all languages that support compilation to Wasm can be used as plugin languages.

It’s pretty easy to use: https://github.com/hellodword/anytype-heart/commit/81a54c531b2abb246a626c203916b20d1b205dd1

I'll keep learning and write my thoughts here.

I'd be glad to hear what anyone thinks.

[bhelx]

Extism contributor here. We're happy to provide some advice. Extism takes care of a lot of the resource level security for you (in terms of access to memory, disk, networks, etc). Obviously it can't prevent everything bad from happening, and some care must be taken to treat plug-ins as untrusted, but it gives you the tools to control how much power you give any individual plug-in.

Regarding OpenAPI: we're also working on a new bindgen system derived from OpenAPI to generate language agnostic bindings across the Wasm boundary.

[hellodword]

After spending nearly three weeks learning and thinking during my spare time, I haven't come up with a complete and stable design yet. Nevertheless, I've decided to start coding based on my notes here, with the branches here and here.

I'm hoping that experienced individuals can assist with the overall design, especially the API, as I'm not particularly good at this aspect but am willing to learn.

Additionally, I'd love to hear the official stance on the extension system I've described. @charlotte-chiang @fuksman

[otherAdamTHM]

+1 Would be very interesting to work with this and write my own integrations, would allow automation at a decent level. Would like to learn more in how these integrations will be implemented into the testing releases. @hellodword's description in the repo seems pretty awesome maybe a few tweaks could happen but it all seems fine to me.

[JonathanReeve]

I've been watching this issue with great interest for over a year now. I was really hoping it'd be implemented when it was originally scheduled to be, I think back in 2023, if I recall correctly, since, like many others have commented here, this is the one thing preventing me from adopting Anytype.

The proposals here for using Extism and WASM as an extension system I think are really interesting. Although for me, I'd really appreciate just a command line interface. That would allow me to script up something like this:

• anytype add --type=book --relation title="Atomic Habits"

The ability to read and write from standard graph serialization formats would be a huge win, too. RDF would be best, since so much of the semantic web is already in that format. Anything that has a Wikipedia or Wikidata page has RDF data that can be downloaded, so adding books or movies with lots of metadata could be trivial:

• anytype import --type=book someBook.rdf.ttl
• anytype export --space=someSpace myGraph.rdf.ttl