500€ | Add webdav support for canvas-server/transports

[idncsk]

We need to implement a webdav server (in canvas-server/transports or as a standalone role (go/rust) using socket.io to connect to the canvas-server instance) that would export the global context tree so that individual layers and all the content they index can be accessed and managed using OS's file manager.

DoD

• Each context "layer" should be made available as a folder (universe://foo/bar/baz would represent a standard directory tree Universe/foo/bar/baz)
• listDocuments() with the context url (for example /work/customer-foo/reports) will return document metadata of all documents within the context - webdav server implementation should create a subfolder for each data type returned (folder work/customer-foo/reports would contain subfolders "Tabs", "Notes" and "Files" as an example)
• Each document is stored as a JSON object, webdav-server implementation will need to have handlers for each data type to correctly generate files in the webdav tree (to illustrate, a "Note" JSON document has 2 main fields, subject and body, those should be converted to a markdown or a plain txt file when shown in the directory tree and parsed to JSON on file save).

List of data types that need to be implemented as part of this ticket(internally called data abstractions, feature bitmap data/abstraction/):

• ~App (you can freely assign apps to contexts; if you work on a random task related to LDAP (work/customer-foo/infra/jira-1234 as an example), you could link your apacheDirectoryStudio or some ldap web apps to it so that when you switch your session context to this task, your tray menu / toolbox and webdav should provide a launcher / .desktop with only the apps (aot) that are needed for that particular task)~
• File (should be fairly easy to implement - for files)
• Note
• Tab
• Todo Optionally
• Buckets (folders) - details how to handle those in terms of indexing are not yet clear

There should be 2 dynamic directory trees made available in the webdav root:

• /Universe (global traversable context tree

  • work
  • customer-foo
    • reports
    • notes
    • tabs
  • customer-bar
    • dev
    • task-1234
      • Tabs
      • Notes
      • Todo
  • home

• /Context (~the current context for the session~)

  • Session ID (Updated 30.7.24)
  • ~Apps~
    • ~List of apps for the current context~
  • Notes
    • List of notes for the current context
  • Files
  • Emails
  • Tabs

Both trees need to be refreshed whenever the global tree is changed or the user changes context Switching from work/customer-foo/dev/taskA to work/customer-foo/dev/task-b with the webdav folder path "Context" opened should should automatically update the folder

Implementation should support all basic FS operations (copy, move, delete, rename, for Notes and Todo update)

[CryptoNinja0331]

Hello, I am interested in this issue.

[idncsk]

I don't see any projects related to webdav(js, rust or go) in your portfolio This task was also posted on upwork, let me refine the description a little too

[luckyman9339]

Hello, I just prepared a demo application for this issue. I applied to your job post with demo application. My name is Dmytro

[Kunkka0822]

@idncsk I am really interested in your project. I've gone through all sub-repositories. Before we discuss the current issue, I'd like to understand this project's functionalities better by testing it.

Is there any documentation that you can share with me, please?

[idncsk]

Perfect, thanks for the effort! especially given that the canvas-server code is in the middle of a big refactor. I tried to explain the base concept in https://github.com/canvas-ai/canvas but we can also have a short call, I can quickly show you where we are heading (should probably finally do a video too :P)

[idncsk]

The missing part in your screenshot / setup I assume is the bash CLI client(or the repl nodejs app - not really maintained but works)

In general, you have your directory-like context tree where each directory is tied to a bitmap index indexing all your data (notes, contacts, emails, file) to a given context

Selecting universe://work/customer-foo/dev/jira-1234 should automagically return all data linked to this particular context (notes, tabs etc) Selecting/creating universe://customer-foo will show you all data linked to customer-foo Selecting/creating universe://reports/2024/customer-foo will do a ligical AND for reports AND 2024 AND customer-foo and show you only data that was linked/commited to all 3 layers (one thing I'll probably implement today is the commitLayers functionality)

You can have your canvas-server sitting somewhere in your home NAS in a docker container(optionally running a minio S3 role), holding all your canvas index data, and a session "work" from your work nb tied to a base URL /work (so as to only navigate within the /work subtree) and a default session with base URL / (universe) on your workstation and personal nb, transparently indexing all your data (browser tabs are notoriously entropy-producing so prioritized that extension as a PoC - big thanks to @tarborlevan for his work)

[idncsk]

[Kunkka0822]

I see.

[idncsk]

you can now use $ context list | grep url # to check if your urls were stored correctly $ context bitmaps # to check what bitmap-based filters are applied to your current context well, those 2 are most practical when developing/testing $ context sessions # the whole submenu (really badly named/implemented) if you need to test the sessions functionality

[Kunkka0822]

@idncsk All things are clear. I just applied to your job on Upwork. Please check.

[idncsk]

@Kunkka0822 whats your initials

[Kunkka0822]

@idncsk My name is Stepan.

[BigHouse0405]

Hello, My name is Rodney, i'm ready

[idncsk]

@BigHouse0405 Hello Rodney, once the grunt work is done, we will start working on llm agents, already did some tests with fine-tuning with unsloth and lora adapters. Internally we use bitmap indexes which I think are a great fit and have huge potential optimizing RAG and even inference(depending on how deep one wants to integrate them), Ill probably also use lancedb in the synapsdb module which is undergoing a rewrite/cleanup => I dont think this webdav module is a good fit but stay tuned for the llm part!

[BigHouse0405]

@idncsk Hello, sir. i'm interesting about your project, so could you invite me? if there's anything you need, count on me.

[Kunkka0822]

@idncsk Could you check your email, please?

[BigHouse0405]

@idncsk thanks for invite me.

[BigHouse0405]

@idncsk Hello, sir , How are you?

[idncsk]

Removing App/Apps from scope Scope in terms of data abstractions will be

• File
• Note
• Tab
• Todo

[BigHouse0405]

@idncsk, i can do it, sir

[idncsk]

This task is already worked on by @Kunkka0822 , I am in the middle of a refactor for the canvas-server part (separating index and document storage which may have not been the brightest idea) but that note above was meant for him

[idncsk]

As agreed with @Kunkka0822, moving back to TODO, lets prioritize electron for now