Syntax

[cmontella]

RFC Link - https://github.com/witheve/rfcs/blob/master/proposed/syntax.md

[Eucalyptus2013]

In the chat app, you use freeze all.

What does it mean?

Thanks

[cmontella]

https://github.com/witheve/rfcs/blob/master/proposed/syntax.md#freeze-all

Good catch, I forgot to include a section on that. Right now, all is used to specify that the subsequent objects are available globally. By default, facts you add to Eve are visible only to your session. In the chat app, we need those messages to be available to other users, so we use the all keyword.

Honestly, we're not too thrilled with the keyword all (maybe global makes more sense?), so suggestions here are appreciated.

[RubenSandwich]

Considering the call for a new name for Eve's Objects:

(object is a pretty generic and overloaded term, so let us know if you have ideas for what to call these guys)

The name Object is problematic for a variety of reasons. You mention the obvious one of Objects in Software is usually referring to a unit in a OOP model. This is problematic not only for the person who is familiar with OOP but also for the person who has heard of OOP, and also for the learner of Eve who after Eve might want to dip their toes into other forms of programming. Furthermore the term Object is a vague and crude term in that anything can be an object.

So I suggest the name Records. You describe objects as:

key-value pairs attached to a unique ID

This falls in line with the concept of Records in Elm and Databases, and somewhat close to other functional programming languages concept of Records. But more importantly Records is a term already known well to mean a collection of data about a specific thing to the general public.

[bertrandrustle]

Given that "Eve" is a homophone of "EAV" (Entity-Attribute-Value) and that EAVs are Eve's core abstraction under the hood, it seems more appropriate to refer to Eve's objects as entities. entity.attribute = value is more conventional than object.attribute = value.

Record is also a sensible alternative.

[shamrin]

 guest = if p = [#friend busy-dates: not(party.date)] then p
         if [#friend spouse busy-dates: not(party.date)] then spouse

And later:

In the above example, we add guests to the list if they are a friend and not busy, or if they are a spouse of a friend and not busy.

It seems to be an inconsistency here. If I'm not mistaken, "if they are a spouse of a friend and not busy" should be written as if [#friend spouse: [busy-dates: not(party.date)]] then spouse. Currently you've implemented "if they are a spouse of a non busy friend".

[cmontella]

Pascal pointed out some deficiencies in the example on the mailing list. We wrote it a couple weeks ago when the runtime still couldn't run the code. I'm currently rewriting the program to make it correct.

[Eucalyptus2013]

How long does a session last?

After the session is closed, is the data stored in the Eve DB deleted?

[cmontella]

@Eucalyptus2013: The Eve DB is a distributed database, so it's a little nuanced, since there are actually multiple DBs (we call them "bags", a better word is appreciated here too).

When you start an Eve server, it starts with an empty global bag. When I start a session on the server (which lasts as long as I am connected to it i.e. the websocket is open), then I'm given my own instance bag, which is empty. But the facts available to me are the union of my instance bag and the global bag.

So every time I say freeze or maintain, the subsequent facts are added to my instance bag. When I leave, I take all my facts with me.

When I say freeze all, then I'm no longer talking about my instance bag, I'm talking about the global Eve bag. In that case, when I leave, my facts are still there.

In the context of a chat app, it uses freeze all to put the messages in a global bag. Thus if I'm in a conversation and my session disconnects, my messages don't suddenly disappear from the terminal. Similarly, if my friend is messaging me while I'm not disconnected, I will see the messages as soon as I connect, because those messages will be in the global bag, and the global bag is unioned with my instance bag.

Does that make sense?

[Eucalyptus2013]

If I want to store a fact in the global bag (by using freeze all), because I want to access this fact after the session is closed, but I don't want the other users to access this fact, how can I do this?

[cmontella]

Right now, our permissions story is incomplete. Eventually, we will have a notion of a user, and be able to stick facts into bags with restricted access.

Until then, you can create your own users, and parameterize the facts with a user attribute. The you can join the restricted facts against the current user. The chat example does this in a way.

[cmontella]

Okay, I've updated the RFC with a better example program. Right now, there are still some execution issues with it, so the result if you run it isn't correct. But we'll work on those more today.

I also removed a section on reusable code. We had a discussion about user defined functions yesterday, and that whole area will be going through a redesign. I'll add that new design to the proposal here as a pull request when I've gotten it together.

[bertrandrustle]

After further consideration, being as objects are employed not only as literal representations of Eve DB facts but also as patterns for matching against such facts, I don't believe records is a suitable replacement term for objects; and while entities still seems applicable, I now prefer the term forms.

[rainhead]

First off, how stable are you trying to get your syntax? You mention under "Risks" that you don't want to turn people off, which is reasonable, but Eve is not at risk of becoming hoon. I'd worry less about turning people on or off and more about gently introducing them to the semantics. Let's allow ourselves to learn how to write Eve programs before we get confident about syntax.

That said, the biggest thing I'd expect to cause people to "not get it" and be turned off is the fact that although a block looks like a procedure, you're actually building a single query. let … in … and similar constructs frame this well in e.g. Haskell, but I'm not sure how to give an imperative programmer the right expectations.

The second-biggest thing are the set semantics. This still throws me off sometimes after writing SQL for 10-15 years. I really don't know what to recommend, but I feel like the syntax has to communicate a programmer's expectations around cardinality. Perl's @ sigil is the only real thing that comes to mind.

On objects: I agree, records or entities would be fine. "Record" is maybe more friendly.

On phases: I think of "prepare" and "commit" phases, if we are talking about them. Consider "do" and "commit", but I'm not confident I understand my intention when using freeze and freeze all. I think there's a sense of making a commitment to others, of making a statement that is hard to retract, so: "publish"? Manipulating the state of bags is surely not what I'm thinking about.

if guest = [@Arthur] What's going on here? Is = testing for set membership? For set intersection?

guest = [#invited name]: The [#invited] syntax seems to reinforce the idea that you're dealing with a set, but then you're assigning that set to a singular guest. Again, how does the programmer express expectations around cardinality?

sum(burgers given burgers, guest) I read , as having lower precedence than given, and didn't understand what was going on here until I read the explanation.

What is [#session-connect] for? Does it mark the beginning of a serializable transaction? Is it necessary? Why does it share tags' syntax?

People who are 50 year old
  [tag: "person" age]
  age = 50

Regarding age: this auto-binding feels much too subtle. You're also discouraging people from using identifiers that communicate something about the intention of the block, or tie together multiple queries. This feels more confusing than destructuring does in ES6, but maybe my feelings would change with practice.

How do you fit predicates other than equality into that record query syntax?

Sorry, I'm at a disadvantage here: you've spent a couple years programming in this paradigm, and I haven't. I can tell you what's confusing to me, but without that experience of solving actual problems in Eve, it's hard for me to suggest concrete improvements.

[cmontella]

Peter,

Sorry, I'm at a disadvantage here: you've spent a couple years programming in this paradigm, and I haven't. I can tell you what's confusing to me, but without that experience of solving actual problems in Eve, it's hard for me to suggest concrete improvements.

Actually, you're exactly the kind of person we want commenting on this; we need the perspective of an outsider because ultimately we want more people using Eve than just us :)

First off, how stable are you trying to get your syntax?

I think we're happy with the overall direction of the syntax, so we wouldn't want to switch to s-expressions or a c-style curly brace syntax. That said, we still see this syntax evolving as we discover more about how to write good programs this way. Since we posted the RFC, we already have new ideas for how to do user defined functions.

That said, the biggest thing I'd expect to cause people to "not get it" and be turned off is the fact that although a block looks like a procedure, you're actually building a single query.

One of our decisions we hope will mitigate this perception is that the blocks are not really named, and there's no real concept of "calling" a block, so without those a procedure really can't exist. Once upon a time, we thought about wrapping the blocks in a query {} just to delineate them, but it seemed redundant and unnecessary.

Another way we hope that our code doesn't "look" procedural is that we're trying push the idea that eve files are actually prose with code interspersed, rather than code with comments interspersed. Thus, visually, eve code will look more like a document than code.

Finally, I think that while the statements could look procedural, the overall feel of the syntax stresses that there’s something different going on. If we had a c-style curly brace syntax, I think that it would be a much bigger concern.

The second-biggest thing are the set semantics.

Yes, we see this as a source of confusion too, but we hope to mitigate it in several ways. For the most part, you can actually code without thinking about cardinality, because our evaluation is row by row. Just consider everything as a scalar, and most of the time things just work when they’re not. You really only have to pay attention to cardinality when you’re using an aggregate, or mutating.

With a little tooling though, we can actually provide the cardinalities of your variables as you code, so this is one way we can help here. Another might be to try and enforce a convention, that plural variables should be used for sets and singular ones should be used for scalars. But that would be purely convention. Regardless, we'll see if this is an issue or not.

if guest = [@Arthur] What's going on here? Is = testing for set membership? For set intersection?

guest = [@arthur] is saying that guest is equivalent to an object that has a name @Arthur.

Again, how does the programmer express expectations around cardinality?

Any time the programmer uses a tag selector, then there's the potential for cardinality greater than one. Regarding the plurality of the variable, maybe this is something we try to encourage through convention. If the variable can be plural, write it as a plural.

sum(burgers given burgers, guest) I read , as having lower precedence than given , and didn't understand what was going on here until I read the explanation.

Commas for us are whitespace. They have no syntactic meaning. The use of the comma here was in the english sense to delineate a list. It should be read "sum the burgers given burgers, guest". However, we’re thinking about new ways to express this to avoid any confusion.

What is [#session-connect] for? Does it mark the beginning of a serializable transaction? Is it necessary? Why does it share tags' syntax?

It has a tag because it’s a normal object, like any other. When I connect to the server, Eve puts a #session-connect in the DB that exists for one tick of the executor. For that one tick, the object [#session-connect] exists, and I have the opportunity to do something to the DB. So this is a good place to initialize any data with a freeze. Using a maintain here would destroy the data on the next tick, because [#session-connect] no longer exists.

This is something we might sugar over in the future though.

Regarding age : this auto-binding feels much too subtle. You're also discouraging people from using identifiers that communicate something about the intention of the block

You can bind any variable to create an alias. For example: [#person age: age-of-a-person]

How do you fit predicates other than equality into that record query syntax?

The same way you do equality. For instance: [#person age > 50]

Thanks again for your comments; you've raised some important points that I need to think about some more. Let me know if you have more ideas/opinions

[bertrandrustle]

When an Eve program is run, are the blocks evaluated in top-down sequence as they appear in the source file?

When a block is evaluated, is the collect phase evaluated eagerly, or is it evaluated when its objects are first referenced in the mutate phase?

Blocks with a maintain clause appear to keep the queries in the collect phase alive. Is there syntax for terminating these open queries programmatically?

Can a single block contain a maintain clause and a freeze clause?

[cmontella]

Bertrand,

When an Eve program is run, are the blocks evaluated in top-down sequence as they appear in the source file?

No, block order should not matter. Typically, line order should not matter at all for Eve programs. However, there are two instances when order matters:

1. In every block, you can only query the Eve DB before a mutation fence (freeze or maintain). So any selects, aggregates, etc. need to come before the fence. After the fence, you can only really use mutation operators.
2. Arms in an if-else block are evaluated sequentially. I point out that in the party planning example, if a #friend is #vegetarian and #hungry, then she will actually get 2 hamburgers instead of 0, because she satisfies the condition of the hungry arm before the vegetarian arm.

When a block is evaluated, is the collect phase evaluated eagerly, or is it evaluated when its objects are first referenced in the mutate phase?

We eagerly evaluate it. For instance, consider this block:

a block
  x = [#tag1]
  y = [#tag2]
  maintain
     y := [#tag3]

If there are no objects with #tag1, then y will never get #tag3, even though we never talk about x in the mutate phase.

Blocks with a maintain clause appear to keep the queries in the collect phase alive. Is there syntax for terminating these open queries programmatically?

The way you could achieve this is by putting a #switch object in the collect clause, in the fashion that I demonstrated above. When the #switch is present, then the block is active. When the #switch is gone, then the block does nothing.

Can a single block contain a maintain clause and a freeze clause?

Yes, blocks can also contain multiple freeze clauses e.g. the chat example uses a freeze and freeze all clause in the same block.

[yazz]

I just found this comments thread. I think a tutorial would be great for Eve. I can write one myself, but would like to know, how long before the first version of the syntax will set set in stone as it were?

[bertrandrustle]

Thanks for nailing down those loose edges for me Corey. I've learned conceptually esoteric languages like Haskell without much difficulty, but in learning Eve I'm having to bury pretty much all of my assumptions and a good number of my intuitions.

There's one important aspect of Eve that remains opaque: I haven't seen public discussion of Eve's temporal logic underpinnings yet. Can you address this and how it might be exposed to programmers through the developer syntax?

[cmontella]

Zubair,

The syntax probably won't be set for a while, so a tutorial might be out of date as soon as you write it. I'm working on one myself (well a couple), which will help in learning the language. However, maybe writing about your experience in learning Eve would be a worthwhile exercise.

[ibdknox]

@zubairq the main thing is we don't want to set anything too into stone until we see more usage from folks outside of us. That being said, there's only one biggish change that I can think of at the moment in terms of the syntax, which is a slightly different syntax for function calls. Beyond that, unless we get some drastic feedback, it's likely just wording changes and such.

@bertrandrustle That's interesting, our language is easily an order of magnitude simpler than something like Haskell. What intuitions and assumptions are being challenged? Do you think there's something we could do to help facilitate that understanding?

At the moment, we don't expose general time traveling in queries, but that's probably the simplest and least semantically important part of the temporal logic. Our semantics are based on dedalus, which has the notion of talking about "now" and "next". This shows up for us as part of the difference between maintain and freeze - maintain specifies that the mutations are bound to their query and that they happen instantly, whereas freeze freezes the mutations at a point in time and asserts them in the next "tick". The reason for this is that to be order-independent our rules are run to fixed-point and there are some things that need to be stratified in order for them to terminate. For example, let's say I wanted to update a counter every time a button is clicked:

increment my counter
  [#click element]
  element.name = "my-cool-button"
  freeze
    element.current-count := element.current-count + 1

If freeze asserted that change "now", it'd never reach a fixed-point because each loop would increment the counter and you'd run again. This query is cyclic - it relies on a thing it changes. By asserting the change in the next tick though, the first loop of the evaluation would generate current-count + 1 and the second loop would regenerate that same value, meaning it's done. For the most part, the only place this comes up is when reacting to events - you want to freeze the value at a point in time and make sure you don't subsequently create an infinite number of other values by looping back on yourself.

[cmontella]

@RubenSandwich suggested changing "freeze" to "commit". The thought here was that "freeze" implies that the object is frozen in time, forever unchanging, which seems wrong since a frozen object can be mutated.

Thoughts?

[bertrandrustle]

@ibdknox

That's interesting, our language is easily an order of magnitude simpler than something like Haskell. What intuitions and assumptions are being challenged? Do you think there's something we could do to help facilitate that understanding?

The choices Haskell makes on what to expose and what to abstract away closely complement my natural cognition, whereas Eve brings one of my greatest challenges to the fore, temporal reasoning.

I'm more fluent in structure than process. My brain models systems as a complex of visual-spatial impressions where each part can be shifted within a concrete-abstract spectrum to facilitate relations with other parts or systems. This is pretty effective for structural pattern recognition, but a major weakness is the lack of a temporal sense, and so I often resort to modeling temporal things as spatial/structural things, somewhat like the Bret Victor demos in which he models time using geometry. With Eve a visual interface may prove more natural for me than a textual syntax.

In the Haskell world almost everything is lazy and immutable by default and time rarely makes an explicit appearance. Programs are fundamentally structured around timeless, unchanging functions which are devoid of side effects. These properties make it easy for me to reason about things. Haskell is a largely frozen world.

Superficially the Eve world is declarative as with Haskell. But Eve programs are structured around movement of state in temporal dimensions. A block essentially describes a discrete interaction of state in time, the duration of which is determined by a two-phase process over state: observations of state (queries) matching certain criteria in the first phase determine whether or not state will be changed in the second phase, and the nature of change in the second phase can be snapshotting (freezing in time) of state or manipulation and continued observation of state (maintenance of state in an open-ended "now"). Also in contrast to Haskell, side effects appear to be Eve's primary method of computation.

I'm not sure any of that makes my perspective relatable or even understandable. As for improving the textual syntax, I don't have anything useful to propose yet; I need more experience with it. But in light of how my brain works I concede I might be an outlier the textual syntax probably shouldn't try too hard to accommodate. ;)

[bertrandrustle]

@cmontella "freeze" is a good choice from a time-oriented viewpoint, and "commit" is a good choice from a database-oriented viewpoint. If Eve is a temporal query language on top of a relational database, which viewpoint should prevail?

The time viewpoint is consistent with the abstraction presented by Eve's temporal semantics. (Although the way I conceive of it, the term "snapshot" captures the meaning even better than "freeze.")

The database viewpoint is closer to the underlying implementation, where it's just a standard database transaction, something already very familiar to developers. The term "commit" itself is also common to source control tools and has much the same meaning there.

[benjyhirsch]

Some thoughts:

• As an alternative to “objects”, have you considered calling them “patterns”? It seems like it would more clearly that there might be multiple matching entities in the Eve DB. I also like “forms” as @bertrandrustle suggested for the same reason.
• Is there any advantage to allowing the syntax identifier.attribute += value in addition to identifier += [attribute: value]? It seems like the former could cause confusion given its similarity to syntax in other languages with very different semantics (i.e. someone writing order.total-price += item-price probably doesn’t expect total-price to be a set containing all the distinct item-prices, and probably meant instead to aggregate something in the collect phase). Also, the following minimal pair might be confusing:

Assume that there's an entity matching [b: [c: 1]] in the DB (and therefore an
entity matching [c: 1]), but no entity matching [c: 2].

The following results in there being an entity with [b: {[c: 1], [c: 2]}] in the DB
(i.e. it creates a new entity matching [c: 2], distinct from the one matching [c: 1]):
  a = [b: [c: 1]]
  freeze
    a.b += [c: 2]

Whereas the following results in there being an entity with [b: [c: {1,2}]] in the DB
(i.e. it adds 2 as another value of the attribute c on the same entity that matches [c: 1]):
  a = [b: [c: 1]]
  d = a.b
  freeze
    d += [c: 2]

If the syntax in the first block weren't allowed, here's how you would write it:
  a = [b: [c: 1]]
  freeze
    a += [b: [c: 2]]
which at least somewhat more clearly might mean something different from the second
block.

• If I understand correctly, the syntax [attribute: value] means substantially different things in different contexts:

  • In the collect phase, it queries for entities of the given shape in the DB, and binds the retrieved values to the specified identifiers so as to make the asserted equivalences true.
  • In the mutate phase, on the top level of the right hand side of +=, -=, or :=, it specifies that these are the attributes that we are going to modify on the entities that are bound to the identifier on the left (and these are the values we will add/remove/set on those attributes).
  • In the mutate phase, either on its own or nested within an attribute on the right hand side of += or :=, it is a literal entity that gets created in the DB.

  One possible alternative to at least distinguish between second and third within the mutate phase and explicitize when we create new entities in the DB:

++[attribute: value]
++[attribute: value, other-attribute: ++[nested-attribute: nested-value]]
identifier := [attribute: ++[nested-attribute: nested-value]]

This clears up the confusion of the earlier minimal pair even more clearly:

This clearly creates a new entity in the DB:
  a = [b: [c: 1]]
  freeze
    a += [b: ++[c: 2]]

Whereas this clearly doesn't (assuming the above syntax for creating new DB entities):
  a = [b: [c: 1]]
  d = a.b
  freeze
    d += [c: 2]

• Some possible alternatives to “collect”: “lookup”, “bind”, “bindings”, “find”, “query”, “get”, “define”, “definitions”, “match”, “matches”, “observe”, “observations”, “declare”, “declarations”, “ask”, “questions”, “learn”.
• Some possible alternatives to “mutate”: “change”, “changes”, “modify”, “modifications”, “update”, “updates”, “edit”, “edits”, “set”, “mutations”, “delta”, “deltas”, “effects”, “do”, “act”, “tell”, “say”, “statements”.
• The notion of absolute versus relative might be useful in naming and/or explaining "freeze" versus "maintain"
• I agree with @bertrandrustle that "snapshot" is potentially more intuitive than "freeze".

[bertrandrustle]

It seems likely the keywords maintain and freeze will continue to be confused with notions of mutability. Both terms connote keeping something as it is by preventing or undoing change. Compounding the problem, the keywords serve a secondary purpose in Eve's syntax: signifying the beginning of a block's so-called mutate phase. Terms which are more explicitly timelike and storage-related could increase their conceptual distance from mutability.

• For maintain I propose refresh, which connotes keeping up to date something that may also change, just as a DRAM chip keeps live data by refreshing its cells, or a computer display keeps a live image by refreshing its pixels. Another alternative could be sync, if that isn't deemed too overloaded.
• For freeze I again propose snapshot, a concept many developers employ on a daily basis in dealing with things such as filesystems, Docker containers, and virtual machine images. A snapshot captures the state of a dynamic system at a particular instant, and can be used as a stable branching point from which to explore future states.

In addition to renaming maintain and freeze, the "mutate" phase could be renamed. Indeed, the names of both block phases may benefit by reflecting underlying database semantics. (See also the suggestions by @benjyhirsch above.) Borrowing from high-level SQL terminology (not necessarily actual syntax):

• collect phase becomes query phase
• mutate phase becomes manipulate phase. SQL's data manipulation language operators mirror Eve's mutate operators: INSERT (+=), UPDATE (:=), DELETE (-=).

I'm aware that Eve's relational database doesn't use SQL, but SQL is widely familiar to people with database experience, and I couldn't identify terminology for a comparable two-phase dichotomy in the domain of Prolog/Datalog/Dedalus.

[bertrandrustle]

I thought of another perspective on replacement candidates for maintain and freeze.

maintain and freeze are imperative verbs which feel out of place in a declarative syntax. Nouns and adjectives are more orthodox in declarative programming and may improve conceptualization in this case.

Objects fenced with maintain represent dependent data that's responsive to changes; the data continually reacts to its supporting queries. Objects fenced with freeze represent standalone data that's passive; after the data is asserted it is at rest with respect to its supporting queries.

With those considerations, I propose renaming:

• maintain to reactive
• freeze to passive

[ibdknox]

First off, thanks for the awesome and thoughtful feedback @benjyhirsch and @bertrandrustle ! I spent a bunch of time last week and this weekend thinking about the comments here and have a few proposals that I think help address a lot of the feedback.

Most fundamentally, we should probably adjust the way we talk about Eve's paradigm in general. Instead of evoking logic or relational languages, it seems like the most straightforward way of describing Eve is as a pattern matching language. "When you see this pattern, do this." Amusingly enough, that's where we very first started with the madlib version of Eve. To that end, I propose the addition of the match keyword to mark the beginning of a block:

invite friends who are not busy
  match
    [@"my party" date]
    friend = [#friend busy-dates != date]
  maintain
    friend += #invited

I tested a few variants of maintain/freeze with people and have come to bind/commit as being the most familiar while accurately representing the semantics of each. They also fit the pattern matching description pretty well; "match this and bind it to...", "match this and commit these..." So that would become:

invite friends who are not busy
  match
    [@"my party" date]
    friend = [#friend busy-dates != date]
  bind
    friend += #invited

I also suggest we change commit all to commit global. To your point @bertrandrustle, in this scheme we could call the parts match and action, which seems pretty straightforward. This is still using verbs here, but I've found that trying to reinforce that the semantics are declarative hasn't really been beneficial in getting people to understand the model. At some level you can think of these as instructions to Eve - e.g. "match these things and do this stuff when you do" - so verbs fit nicely. Fixpointing we could describe as matching the rules until no new actions are fired.

@benjyhirsch you brought up some really good points about the confusing semantics of the mutation operators and I spent the most time trying to figure out something consistent for them. It took a bit, but I think we've got something that's a good deal more sensical than what's there now. The biggest change is that any object in the action part of a rule is a new object. Also, instead of set (:=) behaving like a merge in some cases, there would be an explicit merge operator (<-). :=, +=, and -= would all behave much more like standard assignment in an imperative language, except you have to specify an attribute (which #foo and @joe do implicitly):

// compile error, no attribute is specified and setting an object to 
// another object doesn't make sense
foo := [bar: 3]
// set the bar attribute of foo to 3
foo.bar := 3
// add 4 as another bar attribute of foo (foo.bar is now 3 and 4)
foo.bar += 4
// remove 5 as an attribute of bar on foo (if there was a 5, it would be gone)
foo.bar -= 5
// merge the new object into foo
foo <- [bar: 3]
// add/remove a tag or name to foo
foo += @joe
foo -= #cool
// remove foo
foo := none
// create a brand new object
[#cool bar: 3]
// create an object and get a reference to it
foo = [#cool bar: 3]
[#some-other-object baz: foo]

match
  foo = [#foo]
  baz = foo.bar
bind
  // this would be a compile error as no attribute is referenced
  baz := [a: 3]
  // set a to 3 on the objects of foo.bar, if baz is not an object, that would be a runtime error
  baz.a := 3
  // this would merge the new object into baz, setting a to 3
  baz <- [a: 3]

One thing I'm not certain about is what baz := none should mean. It would remove all the attributes attached to baz (thus removing baz), but what about the references to baz? In this case, foo.bar has a value equal to that object's uuid, should it also be removed? what about other references?

Finally, unrelated to this part, we want to change the way "functions" are called to reinforce that they are themselves really just objects that are created on demand. We also want to make the code a lot more readable by using keyword arguments:

// old syntax
x = sin(foo)
// new syntax
x = sin[angle: foo]

One big advantage besides being self documenting is that optional/alternative parameters are much more straight forward in this scheme:

x = sin[radians: π/2]
x = sin[angle: 90]

Putting the name outside is really just sugar for:

[#sin #function angle: 90, return: x]

Which means that defining your own functions would be something like so, where ? at the end of an identifier means it's an input:

My crazy mathematical formula
  match
    return = (value? + 30) / π
  bind
    crazy-math[value?, return]

In reality you need functions in our language very rarely, but this is a very powerful system. For example if I wanted to support an entirely different coordinate system for sin, I could do this:

coolest coordinate system ever
  match
    return = sin[angle: value? / 2 + 30]
  bind
    sin[chris-degrees: value?, return]

use my sin
  match
    answer = sin[chris-degrees: 40]
  bind
    [#div text: answer]

[ibdknox]

All of the above proposals have been implemented in the match-syntax branch.

[bertrandrustle]

@ibdknox Your changes seem reasonable, but I'm fuzzy on a few things regarding the changes to function syntax.

Now that the match (formerly collect) phase is explicitly fenced, is anything actually being matched in the match clause of your example code here?

My crazy mathematical formula
  match
    return = (value? + 30) / π
  bind
    crazy-math[value?, return]

That looks like a simple equality statement involving primitives and attribute variables but no objects. Conceptually, I believe simple equalities and other statements which don't query the database, filter a query, or affect any statements which do those things, should go before any match/bind/commit clauses, at the first code indentation level.

From a practical standpoint, because a non-querying, non-filtering equality rule such as this is, I assume, always true, its presence in the match clause seems to rule out the possibility of constructing a subsequent bind clause that conditionally terminates the block via dependency on other rules in the match clause.

Finally, would the availability or application of the crazy-math function behave differently if bind were changed to commit in that example?

[skybrian]

Instead of "object" I suggest "selection". We could say that each query defines selection variables in the "collect" phase. This makes it clear that a selection is not immutable - the value of each selection variable (its "current selection") actually changes over time.

Then it would be natural to rename the "collect" phase to the "select" phase.

[skybrian]

"freeze" suggests to me that it creates something frozen. I suggest "save" instead. When the query fires, it saves data to the database. But this data is saved, not frozen.

[bertrandrustle]

Since the syntax appears to be nearing general stability I'm trawling Eve's source to piece together a fully comprehensive cheatsheet. Almost everything is pretty straightforward but there are a few things I need some clarification on.

1. What is the exact purpose of the is expression?
2. What is the exact purpose of the filter keyword?
3. There are logical operators for or and not, but I can't find and. Will that be added at some point, or do implicit joins cover all use cases?
4. Are there other primitive data types exposed through the developer syntax besides Booleans, strings, and numbers? Are UUIDs (using the special bracket notation ⦑x⦒) meant for end use, and if so, where/how can they be used?
5. Are there any fully realized collection types besides sets, and do they have a literal syntax? The tuple-like constructs in statements such as (foo, bar) = if baz then (1, 2) seem to be more syntactic sugar for binding operations than fully realized collection types.

[cmontella]

I've gone ahead and updated the RFC with the new syntax proposals.

The updated sections include:

• The example program
• Program structure
• Block structure
• The match phase
• Functions
• The action phase
• Mutation operators
• Commit vs Bind
• commit global

Of note, and not discussed by Chris above, is the proposal to fully support Markdown in Eve programs by fencing code with backticks. This means that indention is no longer syntactically meaningful, but we will still be encouraging it stylistically for readability purposes. This brings us much closer to Literate Programming as described by Knuth.

[ibdknox]

@bertrandrustle interestingly, because of how we think about functions, everything you do is a match. We treat functions like infinite relations in mathematics. So in the given example, what if value? was a string? In our case, there's no match in our + relation for a string and a number so it fails the match and nothing happens. We can extend that semantic a bit to get a very nice error model. Your program doesn't crash when an expression would normally throw an exception in another language (like adding a string and a number), instead it behaves like other failed selects and also inserts an object about the failure. You can then choose to react to those error objects however you want. As an example, imagine you wrote some software to fly a remote control plane. If you get a bad read off the altimeter you might just ignore it, but you could write a query that says if you get 3 error objects in 10ms it's time to land. By turning errors into just normal information, we get to bring the full power of the language to deal with them.

Answers to your list:

1. is( ... ) takes an expression that would normally filter down the results and give it to you as a boolean. For example if you wrote 4 = 3 in a match, that match would never succeed. If you wrap that in an is, you'll get false: false = is(4 = 3).
2. Filter isn't a keyword, it's just a name we happened to use in todomvc
3. We may find a need for it at some point, but everything is inherently and'd in our language. Each pair of expressions in a match can be read as having an and in between.
4. Aside from tooling reasons, there's no reason to use UUID literals. Those are the only types we have for now.
5. Nope, we only have relations and as you say, sugar for a talking about a group of things instead of having to write them all out individually.

[ibdknox]

@skybrian bind/commit have tested really well so far. FWIW, we had save and moved it to freeze because the way we ended up explaining it to people was as "freezing the values in time." In any case, bind and commit seem much better, bind especially.

[bertrandrustle]

Markdown has no official syntax specification and the reference implementation is ambiguous in places. If Markdown is to be supported in Eve source files, there's an unofficial standard, CommonMark, being developed by some well-known and respected developers. It's currently in draft but probably near enough to a stable 1.0 release to base an Eve parser on.

[Eucalyptus2013]

You can 'commit global', can you 'bind global'?

Can you restrict a match to a specific bag? For example, 'match session' to match with the facts stored in the session bag. I ask this question for performance reason. If you know that a fact is stored in a specific bag, is it slower to query all the bags? I know there are two bags currently (session and global) but maybe in the future the user could create its own bags.

[yazz]

Having to have ``` after the initial comment is kind of weird as most people will find back tick almost impossible to find on their keyboard. On my work computer it is not even listed as a key so I have to copy and paste it from the web

[bertrandrustle]

@zubairq Remember, this syntax is targeted at developers who already know other programming languages. I assure you that most developers do know where the backtick key is, or if their trusty development keyboard doesn't have one, they certainly know some quick input method to produce the backtick/backquote/grave accent. ;)

[ibdknox]

@zubairq huh, I didn't know that. CommonMark allows both ``` and ~~~ to be delimeters. Is tilde (~) easy to type?

[yazz]

~ is even worse as most people have never even heard of it (except when you say Tilde and people think of Tilda Basmati Rice) let alone know where it is on the keyboard. Similar to tilde, is almost impossible to find on many European keyboards due to the fact that we have so many different regional keyboards due to the languages spoken here (German is the main language, not English in Europe). Also another problem with on many European keyboards is that to make it appear you have to press it twice, and then it shows up twice as well. What would be great would be if there could be some sequence of commonly used characters used in an uncommon way to do this, like ****\ or something like that

[bertrandrustle]

@zubairq German keyboard layouts are insane, but I haven't met a German programmer who couldn't type backticks. They had to learn because the backtick is widely used in programming and markup languages:

https://en.wikipedia.org/wiki/Grave_accent#Use_in_programming

In hindsight it was probably a bad move to abuse the grave accent for programming purposes unrelated to accenting. (Knuth and Stallman share a lot of the initial blame, but they were just making the most of the primitive, non-standardized keyboards they had to work with at the time.) But it's not going away anytime soon, so it's one of those obscure things programmers just need to learn how to use.

That said, I don't expect Eve's graphical interface, in whatever form it takes, will have any use for backticks. If it did I'd probably consider that a design smell.

[yazz]

@bertrandrustle To type three backticks they need to press ` four times, then delete one of them (as you have to press backtick twice to get it to showon many European keyboards). Even Github has a problem with backticks in comments with a backslash first, so that is another reason (although minor not to use them). You are right in that any programmer can find the backtick, but we need there to be as little friction as possible when learning Eve.

I teach a lot of HTML, javascript and other programming to people on a daily basis, and far more "non programmers" are programming than you could possibly imagine, and these people do not know the difference between \' and ` .... these sort of things can make them hate a language, even though for hard core developers this is a non issue ....

And I have met lots of German programmers who have problems with backticks and other strange characters since I lived and worked in Germany for 2 years and I work with many Germans... maybe because my experience is that most developers are "junior" developers these days...

It may make sense to figure out when we say Eve language is for developers we should decide "which" developers is it for.

[yazz]

@bertrandrustle You do make a good point when you say "this syntax is targeted at developers who already know other programming languages". In this case the hundreds of Data Scientists I work with would definitely have trouble with Eve as a language, so lets be clear that Eve Language is for "Senior Developers".

[ibdknox]

Let me chat with the team on Monday. FWIW, the reason we arrived at just using markdown was because of its relative ubiquity. I don't understand how it ended up with this design given what you said. It's pretty ridiculous to discount entire groups of people because of syntax character choices.

There are a few ways to mitigate this that I can think of:

1. depart from markdown syntax
2. accept the somewhat ambiguous indented code case
3. paper over this in our editor, e.g. a simple key combination or click to create a code block. (The intent is for our editor to be closer to word than emacs)

Realistically, 3 is likely the most ideal case for novices in general since it means they don't have to deal with the disaster that is most programming oriented text editors. I'm not sure that's enough though as it doesn't make sense to alienate people who do still want to use emacs/vim/sublime with different keyboards.

[yazz]

@ibdknox Ah, that makes sense now, why you chose the 3 backticks. I had no idea that it was used to denote code with in Markdown. I checked the Markdown documentation and it seems that there is no other way to denote code, except for indentation, which I did like by the way. But I guess markdown is a well understood format so maybe when in Emacs or Vim mode could still be used since only programmers use those editors, and for everyone else your suggestion of 3) sounds good!

[yazz]

@ibdknox And maybe I made too much fuss ovr the anyway, as I ended up just copy and pasting the lines with in it anyway!

[ibdknox]

ooo, or option 4:

1. Don't hardcode any of the keywords/delimeters and allow them to be localized...

[cmontella]

@zubairq: No, you're not making a fuss, this is a perfectly legitimate concern. Copy and pasting ticks isn't a good user experience.

[RubenSandwich]

@ibdknox I like the new match and bind blocks, they definitely help explain what is happening. It is also very interesting to me that your bringing up Literate Programming as one of the dreams of it is writing code in the order that makes sense to the programmer and not the order that makes sense to the compiler, a requirement I haven't seen met yet, but from my understanding the code block order does not matter in Eve.

One thing I think should be clearer defined in Eve is collections. Here is a great example, is the following variable holding a single or multiple values?

bob = [#people #bob]

Currently in Eve their is no way to know. Might I suggest the compiler warning the user if the variable assigned to plurality doesn't match the results plurality. Something like:

[#people #bob] returns multiple records but you are assigning it to a result that is not plural. Might we suggest naming your results: 'bobs' so other people know multiple records are stored there?

Now I know that figuring out the plurality of words is one of those problems that the last 10% might be near impossible, but I do think that this would help improve Eve code by enforcing certain naming for results containing multiple records.