Missing documentation on `$` prefix for parameters

chuanqisun commented 1 year ago

Hi, I encountered some examples of $ parameters on these pages:

$AIR_ROUTES_NODES_URL in https://docs.cozodb.org/en/latest/tutorial.html#Importing-dataset-the-hard-way
$q https://docs.cozodb.org/en/latest/vector.html

The problem is that there is no explicit documentation for such parameter syntax in CozoScript. The two occurrences above appear in advanced topics so new users might not find them easily.

Some ideas:

Add a section on Queries page about this syntax
Document parameter binding in one of the host languages (maybe Rust?) and point other languages to that location
- SQLite, for example, documents the parameter syntax in the C-language interface

chuanqisun commented 1 year ago

As a related issue, search $ on the documentation sidebar yield no results. It was probably escaped/sanitized.

zh217 commented 1 year ago

Yes, the docs need an update. There's quite a lot of "undocumented" features on the dev branch now, for example this is legal (on dev):

?[] <- [[$x, $y, $z]]
:put rel {}

I will try to find time to document all of them when the next release comes.

For the sphinx docs excaping issue, I'm not at all familiar with their search functionality. Any ideas?

chuanqisun commented 1 year ago

Quick investigation: Sphinx's indexer seems to strip $ from the terms. Here is the raw index: https://docs.cozodb.org/en/latest/searchindex.js

Looking for AIR_ROUTES_NODES_URL in that file, we can confirm that the $ is not part of the "term"

I wasn't able to dig further. I think it's either an nbsphinx parser issue, or an sphinx indexer issue. It was just a nitpick btw.

gisborne commented 1 year ago

I’d be delighted to help with that.

I’m an experienced technical writer and developer, with a strong knowledge of formal logic and the relational model. On Jun 15, 2023 at 10:38 -0700, Ziyang Hu @.***>, wrote:

Yes, the docs needs an update. There's quite a lot of "undocumented" features on the dev branch now, for example this is legal (on dev): ?[] <- [[$x, $y, $z]] :put rel {} I will try to find time to document all of them when the next release comes. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

zh217 commented 1 year ago

@gisborne That would be very much appreciated! I think it would be best to proceed in this way:

In the future, there will be alpha and beta releases
- These will be source code-only, as building all the artefacts is quite time- and resource-intensive
In the release notes, new features will be very briefly explained
Anyone at any time can submit pull requests to https://github.com/cozodb/cozo-docs/tree/dev for changes in the documentation, not only restricted to documenting new features
Before each final release, the changes will be consolidated so as to make sure that the documentation stay coherent and consistent with the features

gisborne commented 1 year ago

Great!

Is there anything you want to update now, or shall we wait on the new release (rough ETA?).

Some comments you’ve made lead me to believe you have a rough roadmap in mind. Even a rough one would be good to share, if I can help with that.

I love everything about this project. My personal interest is in getting the industry to switch from SQL to Datalog, and to shift most business logic into Datalog. I think there’s a potential revolution in engineering efficiency to be had there. I’m hoping before long that Cozo will be a serious industrial relational database, with failover and scalability. Various things you’ve said lead me to understand that you are moving in that direction.

FWIW.

On Jun 15, 2023, at 21:08, Ziyang Hu @.***> wrote:

@gisborne https://github.com/gisborne That would be very much appreciated! I think it would be best to proceed in this way:

In the future, there will be alpha and beta releases These will be source code-only, as building all the artefacts is quite time- and resource-intensive In the release notes, new features will be very briefly explained Anyone at any time can submit pull requests to https://github.com/cozodb/cozo-docs/tree/dev for changes in the documentation, not only restricted to documenting new features Before each final release, the changes will be consolidated so as to make sure that the documentation stay coherent and consistent with the features — Reply to this email directly, view it on GitHub https://github.com/cozodb/cozo/issues/135#issuecomment-1594062593, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAG7X22GBAFX6VRN33QI6DXLPL4BANCNFSM6AAAAAAZIFPB7Y. You are receiving this because you were mentioned.

zh217 commented 1 year ago

Our development is currently guided by the needs of our other projects, mainly focused on facilitating and automating scientific and social research. The recent features show how Language Learning Models (LLMs) have influenced these projects. Therefore, we're concentrating on improving text processing with complex logic. We're not heavily focused on big data or distributed processing at this time. While we don't prioritize speed, CozoDB, being built on Rust, is quite fast.

CozoDB is actively used in these projects in production, which means we might not have enough resources to meet all the demands of the open-source community. However, this also allows us to develop an authentically open-source CozoDB without relying on venture capital, which can sometimes harm open-source projects in the long run. As an open-source project, we invite anyone who wants to contribute new features or even fork the project.

gisborne commented 1 year ago

Are you saying by this that it isn’t on the roadmap to implement features that make Cozo suitable as a business database (probably failover is what’s needed)? On Jun 15, 2023 at 22:51 -0700, Ziyang Hu @.***>, wrote:

Our development is currently guided by the needs of our other projects, mainly focused on facilitating and automating scientific and social research. The recent features show how Language Learning Models (LLMs) have influenced these projects. Therefore, we're concentrating on improving text processing with complex logic. We're not heavily focused on big data or distributed processing at this time. While we don't prioritize speed, CozoDB, being built on Rust, is quite fast. CozoDB is actively used in these projects in production, which means we might not have enough resources to meet all the demands of the open-source community. However, this also allows us to develop an authentically open-source CozoDB without relying on venture capital, which can sometimes harm open-source projects in the long run. As an open-source project, we invite anyone who wants to contribute new features or even fork the project. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

zh217 commented 1 year ago

No I'm not saying that.

gisborne commented 1 year ago

Delighted to hear.

In the interim, I’m ready to help with documentation as I can. And I’m going to learn Rust to see if I can start contributing to the code.

On Jun 16, 2023, at 18:34, Ziyang Hu @.***> wrote:

No I'm not saying that.

— Reply to this email directly, view it on GitHub https://github.com/cozodb/cozo/issues/135#issuecomment-1595569485, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAG7X6676LIJPMINDNIYSDXLUCSJANCNFSM6AAAAAAZIFPB7Y. You are receiving this because you were mentioned.

zh217 commented 1 year ago

The next release will have to wait for a while, as I have my hands full at the moment.

cozodb / cozo

Missing documentation on `$` prefix for parameters #135