State of Org-roam v2 support

jethrokuan commented 3 years ago

Before I release Org-roam v2, I'd like at least basic compatibility with org-roam-server. Ideally, both packages should upgrade at the same time so MELPA upgrades aren't catastrophic. When org-roam-server is ready, I can coordinate the upgrades of the "officially-supported" extensions to v2.

goktug97 commented 3 years ago

Hey,

I am not sure. It looks more complicated than I expected. Everything has been changed. It seems everything is done with node instead of file and a lot of functions I used are now depracted. I am really busy with masters, job, research, other projects etc. It is really awesome that you can still keep up!! Last year, I was devoting my free time to this project but this year I want to keep it to myself :)

Currently, I think org-roam-server works pretty well for org-roam v1 without any issues and most of the Issues are feature requests.

If anyone wants to migrate org-roam-server to v2 that knows how org-roam v2 works that would be awesome.

midas0441 commented 3 years ago

@goktug97 I just saw this message, and I just want to say that I empathize. I am amazed by what you, @jethrokuan and others do for a large community of users. As a person who values (and has benefited from) what org-roam-server and org-roam provide to my workflow, I want to say that I completely understand your situation and wish you best in your prioritization of your research and education. Good luck!

jethrokuan commented 3 years ago

Hey, I am not sure. It looks more complicated than I expected. Everything has been changed. It seems everything is done with node instead of file and a lot of functions I used are now depracted. I am really busy with masters, job, research, other projects etc. It is really awesome that you can still keep up!! Last year, I was devoting my free time to this project but this year I want to keep it to myself :)

@goktug97 Gotcha. I totally understand, thanks for all the effort you've put in on this project. Since this is one of the more popular extensions, I'll personally perform the migration of this project, but only after v2 is actually merged into master, so there will be a period of time where org-roam-server won't be functional. I took a brief look at what needs to be done, and there's really quite a bit. I might also have to strip things down, like I did with Org-roam itself.

All the best :)

midas0441 commented 3 years ago

Thank you, @jethrokuan. Truly, above and beyond!

goktug97 commented 3 years ago

Awesome! This was my first project with Javascript or web design in general and I had 0 experience with Javascript. I just had this idea and wanted to implement it, so the code is a little bit mess. There is a lot of room for improvement. Good luck!

kirillrogovoy commented 3 years ago

I've been using Org Roam v2 for some weeks, but I've never used v1 and, therefore, org-roam-server.

Even before discovering org-roam-server, I had an idea that "I should totally draw a graph of my nodes". That's how I ended up here, also having read the whole thread and studied the code.

I wanted to contribute, but apparently, I'm a very bad Elisp hacker. I can understand most of the code, but trying to write something on my own feels like when you try to draw on paper for the first time in your life: fun but ugly.

However, I have 7 years of experience with Web. I've built and scaled numerous apps, and I feel very confident at working with Next.js, Typescript, etc. I also work with HTTP servers and SQL queries every day.

I've been thinking about building my own version of org-roam-server that's Typescript-heavy and Elisp-light (due to my incompetence) and eventually sharing it with the folks. It would only support v2.

Do you guys have any thoughts? Are there reasons to not do it? Or maybe I could use my strong side elsewhere with greater leverage?

If anyone feels like it makes sense, I'd be happy to pair up with somebody who'd do the Emacs side of things.

And @jethrokuan @goktug97 thank you for every second spent working on org-roam and org-roam-server. :heart: :heart: :heart: These projects, along with Orgmode itself, are my best discoveries when it comes to personal tooling.

smallstepman commented 3 years ago

@kirillrogovoy some of us had same idea, you can follow discussions at https://github.com/smallstepman/org-roam-server/discussions to see where we at :)

smallstepman commented 3 years ago

@jethrokuan we'd like to move development and discussion here... can you please give my account permission to create new branches and push commits to this repo? would you also please enable github discussion in the repo settings?

jethrokuan commented 3 years ago

Sure, I'll add those involved into the Org-roam-server team.

kirillrogovoy commented 3 years ago

@kirillrogovoy some of us had same idea, you can follow discussions at https://github.com/smallstepman/org-roam-server/discussions to see where we at :)

Wow, that's awesome!

I've hopped through the discussions. Looks like you guys are looking for the same outcome as I do.

I'd love to join and be useful! I'll try chiming into the discussions and not flooding this issue.

smallstepman commented 3 years ago

Thanks, I'm in :) Can we also have GitHub discussions enabled for this repo?

jethrokuan commented 3 years ago

I created another (currently private) repo for your usage, feel free to do whatever you want with it.

tefkah commented 3 years ago

Thank you! I have been added to org-roam-server but not yet to this unknown private repo, could you add me there as well? Thank you so much!

smallstepman commented 3 years ago

I think you need to invite us there Jethro, cause I too can't see it edit: also add @kirillrogovoy

jethrokuan commented 3 years ago

Strange, I not sure where I added you guys before, but now you should've all received an invite to the team, which lets you see the repo. @smallstepman @ThomasFKJorna @kirillrogovoy

kirillrogovoy commented 3 years ago

I can confirm I can see and interact with the repo, thanks!! (after accepting the invitation in my Email)

kirillrogovoy commented 3 years ago

I'm going to cite our conversation here: https://github.com/smallstepman/org-roam-server/discussions/3#discussioncomment-1013416

TLDR: I, @ThomasFKJorna, and possibly @smallstepman are starting the work on a new app that's supposed to supersede https://github.com/org-roam/org-roam-server in the future.

We will only support org-roam v2 and there's no guarantee about having that app have everything that org-roam-server has.

We'll share it separately once we lay down the foundation.

pschorf commented 3 years ago

@kirillrogovoy I'd be interested to help once things get rolling.

agzam commented 3 years ago

I don't want to sound like a pompous prick since I'm not offering anything definable (sorry, at the current point of my life, I don't think I can open up some bandwidth for [yet another] project).

However, I guess it's not such a big crime to share my two cents on the matter of rebuilding org-roam-server.

I think it would be nice if you folks considered re-writing it in Clojurescript. I don't mean to start a colossal flamewar (and start arguing about cons and pros), but since the back-end (if we can consider Org-roam as the back-end to Org-roam-server) is written in Lisp, I think it simply would make sense to try to keep at least some consistency.

And the only "production-ready" Lisp dialect usable for front-end web apps today is Clojurescript.

If you ever consider my argument as, at the very least rational, I could maybe help with the basic skeleton for the project. Also, Clojurescript is not that difficult to learn, even though, I admit, it does require some learning (just like any other tool).

kirillrogovoy commented 3 years ago

I think it would be nice if you folks considered re-writing it in Clojurescript. I don't mean to start a colossal flamewar (and start arguing about cons and pros), but since the back-end (if we can consider Org-roam as the back-end to Org-roam-server) is written in Lisp, I think it simply would make sense to try to keep at least some consistency.

Hey Ag,

Thanks for chiming in! No worries, we all mean good here. :wink:

As much as we'd love to fiddle with a new programming language, the main risk right now is not finishing anything tangible even with the technology we know the best (because life 😅).

Also, whatever tech we end up using, we need at least one very active member who's an expert at that tech. (None of us are experts at Clojurescript, but I had a ton of experience with Typescript and React).

Last, we are making the project for the community and we want to enable as many people as possible to contribute or just tweak and hack the app for their own needs.

Hence, we are using arguably the most popular Web-dev stack of 2021 that we do have some expertise at.

kirillrogovoy commented 3 years ago

I'd be interested to help once things get rolling.

@pschorf Sure thing! I'll make sure to let people here know when we roll out the "v0" — Just watch this issue.

We are keeping it private on purpose to not have an uncontrollable inflow of PRs and/or issues without much direction, but once it's out, everybody's help is welcome!

tefkah commented 3 years ago

@agzam As @kirillrogovoy said, we don't really have any experience with ClojureScript and would like to get a working version out as soon as possible, both for people wanting an org-roam-server replacement and our own time :p

Another reason for using react is that we want to use a graphing library that is extremely easy to use, namely react-force-graph. Force-graph can also be implemented with ClojureScript of course (I got the idea from LogSeq, who do use ClojureScript) but without someone somewhat competent at ClojureScript it would eat up a lot more of our time trying to figure out how to make the two click (LogSeq's graph isn't the best example to work off of).

I do really like the idea though, it got brought up in our discussion at some point as well, but at the moment I don't think we will switch, sorry!

tefkah commented 3 years ago

Here's a little teaser to whet your appetite a little Note that we are in the process of simplifying things massively, not the final product yadayada

https://user-images.githubusercontent.com/21983833/126295409-bc5c5f11-1ddb-47de-be1f-28a1e3140054.mp4

jcguu95 commented 3 years ago

The teaser is perfect. I'm so much triggered. How many nodes do you think it will be able to handle smoothly? I have near 10k nodes, and it will only grow faster. I expect many other users would have similar issues too.

EDIT My machine had a hard time running react-force-graph's large graph example. It still works, but the latency is quite noticeable.

tefkah commented 3 years ago

The above graph is my personal knowledge base and has around 1k nodes atm.

2D version should be fine for larger graphs, see this example with 75k nodes (warning: s l o w ): https://vasturiano.github.io/force-graph/example/large-graph/

3D graph is more tricky, see here one with 4k nodes: https://vasturiano.github.io/3d-force-graph/example/large-graph/

In general however, graphs can be very performant with the right settings, and we to provide a good way to interface with them (see the sliders). With that, even very large networks are fine once they stabilize the experience is rather smooth regardless. Some options are inherently slow (highlighting the nodes on hover for instance), but again, should be mostly fine once the graph has been stabilized.

EDIT: to your edit: that graph also has one of the more perfomance sapping features enabled, namely particles travelling the links indicating their direction. That alone makes a huge difference, as without it nothing needs to be rerendered every frame, and panning should be smooth.

jcguu95 commented 3 years ago

(I love how your "large graph" expands from a ball of mess to a nicely structured sheet!)

Seems that this is the best tool we have to visualize our knowledge network. I'm sold! Please also let me know if helps are needed! Look forward to it. Thank you :smile: