Datascript Explorer

[tangjeff0]

132 https://athensresearch.github.io/athens/cards.html#!/athens.devcards.db_boxes

91 https://athensresearch.github.io/athens/cards.html#!/athens.devcards.sci_boxes

Using sci, allow users to query their database in the browser. Datalog is a very powerful language and could one day be as common knowledge as SQL.

[ticean]

:heart: Is this in progress now? This would be a great feature to keep around in the final version!

The sci.web interface is :100:. Is there a reason that we need sci itself or could we use the datascript client in CLJS for the eval?

[tangjeff0]

Good question @ticean. We are interested in sci.web beyond pure datascript because we expect that there may be use cases where users want to import arbitrary npm packages. For instance, if they wanted to use other graph visualization libraries.

*Edit: Another reason is that sci provides a safe sandbox for code execution.

[jelmerderonde]

I'm working on something, not much code yet, but I do have an idea that might work. Current progress can be seen here: https://jelmerderonde.github.io/athens/cards.html#!/athens.devcards.db_boxes.

The code I have builds on the work done earlier by @tomisme in #91.

General idea:

• Create a block "type" that evaluates it's contents using sci. Provide bindings for datascript functions like q and pull. Also provide a binding to the datascript connection.
• Results will be displayed below the block in a simple visual browser, displaying single values, lists and tables.
• Values that indicate a "next step" like a unique id or a reference (we can identify these using the datascript schema) should be clickable. Clicking them generates a new query and a new set of results that can be displayed in the browser.
• Keep a history of previous queries/result-sets so users can navigate backwards.

Executing queries

• [x] Provide correct bindings to sci
• [x] Execute query and display results as plain text
• [x] Catch errors and display them

Browser

• [x] Scalar values
• [x] Sequences --> list
• [x] Sequences of maps --> table with headers

Navigation

• [x] Detect "navigatable" datoms
• [x] Generate queries
• [x] History with previous queries

Feedback to this idea is appreciated.

[tangjeff0]

@jelmerderonde great start!

---

Values that indicate a "next step" like a unique id or a reference ... should be clickable

Love this idea!

---

Keep a history of previous queries/result-sets so users can navigate backwards.

Keep in mind that we get history and time travel for free with DevCards:

• https://tangjeff0.github.io/athens/cards.html#!/athens.devcards.db
• http://rigsomelight.com/devcards/#!/devdemos.core

It would be nice to have this a developer tool outside of DevCards too. You get this with Elm, Redux DevTools, and re-frame-10x, but posh/re-posh doesn't have this unfortunately.

---

If you haven't already, try doing datascript queries in Roam. Things to note about Roam's queries:

• they eagerly evaluate, which is sometimes annoying
• if the query is too large, you get an error message. How should we handle large queries?

[jelmerderonde]

@tangjeff0 thanks!

---

Keep in mind that we get history and time travel for free with DevCards:

Good point, I will enable this next time I work on it.

---

they eagerly evaluate, which is sometimes annoying

I will have to figure out how to do this, I will also take a look at Roam queries.

[jelmerderonde]

Small update, I've managed to get some minimal browsing going. You can now click on cells that we know contains a :db/id and it will generate some new code for you.

Also, the code is a mess and in need of some cleaning up, I will do that later.

[jelmerderonde]

Another update, see the latest state at: https://jelmerderonde.github.io/athens/cards.html#!/athens.devcards.db_boxes

• Improved the table React no longer complains about missing :keys
• If the result is not a map, only the first 10 results are displayed initially. This improves responsiveness.
• Navigation is also possible on refs and unique attributes.
• Added an explanation and a button to reset to the initial state.
• Devcards history is now enabled, allowing you to go back to earlier queries and results.

I think I'm now at a stage where it would be helpful to have some discussion about which direction we should take this in. When is it "good enough" for developers? What questions should they be able to answer with it? How can it be further improved?

I'm looking forward to hearing more ideas and opinions!

[tomisme]

@jelmerderonde I'm definitely finding it useful for exploring our attributes. I added this to my devcards css:

.com-rigsomelight-devcards_rendered-card {
  display: flex;
  flex-direction: column-reverse;
}

I'm sure it breaks other cards somewhere but it moves the history control bar up above the text box.

[tomisme]

Some ideas for where we could go:

• explore using datafy and nav (https://corfield.org/blog/2018/12/03/datafy-nav/)
• a button that lets you go 'up' the graph to a block's parent

[tangjeff0]

I'm sure it breaks other cards somewhere but it moves the history control bar up above the text box.

@tomisme we can override CSS on each individual Devcard namespace. No worries!

---

Agree with datafy and nav. Let's get inspired by REBL and Datomic On-Prem Console for what other interactions and UI we could build!

---

Can't wait to have #31 so people can begin querying their own DBs to learn datascript!!

[jelmerderonde]

I've added some minor improvements in #138:

• [x] Reverse flex direction of browse db card as suggested by @tomisme
• [x] Adds the ability to browse "up" the graph, also a suggestion by @tomisme

For now I think this is as far as we should go with this. It should be suitable for experimenting with datascript and exploring the database. When we have a firmer idea on how we will work with datascript and how we want to implement similar functionality in Athens we can pick this up again.

Do you agree @tangjeff0 ?

[tangjeff0]

Looks great @jelmerderonde , approved and merged.

Eventually, when parser is further along, we can implement exploration for [[links]]s and ((block))s.

[jelmerderonde]

Eventually, when parser is further along, we can implement exploration for [[links]]s and ((block))s.

This shouldn't be hard to implement (I think). Let's revisit it when @roryokane deems it ready :-).

[tangjeff0]

Moving this to the "Done" column because doesn't require any immediate action

[tangjeff0]

This has mainly been superseded by #139. Closing!