The path to a new system programming language!

[daokoder]

Just some ideas, still very preliminary :)

https://gist.github.com/daokoder/1621c0c106a30f0ee8af

[Night-walker]

I'll try to be straightforward and sincere. Don't take my attitude as neither condemning or approving, it's just considerations which I deem important for you to reckon with.

That's utterly unexpected. Honestly, the first thought I had was a concern about the quality and pace of development of Dao, as it still requires a lot of polishing and approbation. And documentation, and testing, and etc. Sharing some parts of the new language with Dao should of course greatly simplify the task, but... It's just that there is saying about chasing two hares which results in loosing both, and I often find that it is true.

Anyway, I certainly cannot stay indifferent at a sight of a new language :) Since C++, there weren't any noticeable languages equally suitable for low-level and high-level programming. Well, there is D, but few really use that pile of everything dumped indiscriminately on a dusty C++ basis. There is Go, a conservative C remake which, even though it's not bad, doesn't really solve anything while lacking both abstraction and performance to compete with C++ on par.

But now there is Rust. It's almost here, the 1.0 debut is already in the vicinity. Truth be told, Rust also lacks the abstraction of C++; notably, it's too pointer-ownership-lifetime-oriented to a point that it looks like a logical game for paranoidally-inclined C fans. I don't see way to implement implicit sharing and other stuff which allows to forget about technical details and concentrate on high-level task you're actually solving. Nevertheless, Rust is currently the most promising systems programming language which already solves all problems related to memory management at compile time. And it is the only language which has potential to replace not only C++ but C as well, which hasn't happened for you-know-how-many decades.

From the ideas you laid out I can guess they are born under impression of Rust. But making a systems programming language is much more ambitious task comparing to a scripting language. It's a very tight, demanding niche, a blood-stained arena on which seasons gladiators make short work of feeble newcomers. A language like Dao can be used even without guarantees of safety, stability and performance, but a systems language may not. You wouldn't take some virtually unknown, practically unproven and generally unsteady language developed on an "occasional basis" to write something of DaoVM scale, would you?

I often caught myself on the thought that it would be great to use something instead, or at least beside, C for Dao-related development. However, now I think that Dao itself should already be sufficiently convenient and maintainable to use it for tasks where performance or lower-level tricks are not of high importance. And solving really complex tasks is easily accomplished by the use of known and proven third-party libraries.

Now, descending to concrete theses about the supposed NeoDao.

All memory errors such as dereferencing null pointer, double-free, freeing objects still in use, leaking must all be caught a runtime.

Rust resolves all of that at compile time: null pointers simply cannot be created using standard language facilities, freeing is carried out without any intervention of the programmer and is proven to be correct by the compiler.

It will use ownership to greatly simplify memory deallocations. Primitive variables cannot be shared, as they do not support reference couting.

So, ownership or reference counting? One is not compatible with the other, as they both solve the same task.

Assignment := to transfer ownership?

Actually, any reference assignment or passing of it to a function must result in ownership transfer, as otherwise it's simply not ownership but "commonship" which cannot give any guarantees in concurrent environment :)

Overall, it's not clear what such language is meant for and what it can offer to those who've seen Rust. I can imagine it can be simpler and higher-level at some points, but that is not enough for it to be adopted for serious use. If a systems language does not imbue its users with convincingly magical capabilities without any sacrifices and is not backed by major IT players and there is no (planned) enterprise-quality IDE with auto-completion and integrated help and etc. -- the prospects aren't bright.

Only a narrow specialization could keep such language afloat -- if you can find a niche where NeoDao would perfectly fit in, only then its development won't be in vain. That is my opinion.

[daokoder]

That's utterly unexpected. Honestly, the first thought I had was a concern about the quality and pace of development of Dao, as it still requires a lot of polishing and approbation. And documentation, and testing, and etc. Sharing some parts of the new language with Dao should of course greatly simplify the task, but... It's just that there is saying about chasing two hares which results in loosing both, and I often find that it is true.

I haven't finished reading you comment. But I feel I should clarify a few points first: no need to worry about its impact on the development of Dao, I am not going to develop that language before Dao has reached production quality. In fact, I won't start before I am sure I will have sufficient time to work it through. I know how much time it might take. I would not start it if I have no time to finish it. Currently I am just thinking and recording my ideas, and trying to think things through.

I brought it up mainly because I'd like to have some feedbacks :)

[daokoder]

Truth be told, Rust also lacks the abstraction of C++; notably, it's too pointer-ownership-lifetime-oriented to a point that it looks like a logical game for paranoidally-inclined C fans. I don't see way to implement implicit sharing and other stuff which allows to forget about technical details and concentrate on high-level task you're actually solving.

This sounds quite right about Rust.

Nevertheless, Rust is currently the most promising systems programming language which already solves all problems related to memory management at compile time.

Resolving all memory issues at compiling time sounds great, but in practice, all those things that make this work seem to have added too many extra details that the developers must master or pay attention to. I only went through the Rust documentation briefly, this is the impression I get, maybe I am wrong.

And it is the only language which has potential to replace not only C++ but C as well, which hasn't happened for you-know-how-many decades.

I am a bit skeptical about this based on my impression.

From the ideas you laid out I can guess they are born under impression of Rust.

Not exactly, however I did had a look into it after I have got some preliminary idea (which was motivated by some discussion I saw on reddit and/or hacker news).

But making a systems programming language is much more ambitious task comparing to a scripting language. It's a very tight, demanding niche, a blood-stained arena on which seasons gladiators make short work of feeble newcomers.

Maybe more ambitious, but not necessary much harder, especially when there is LLVM. Also the reasons to develop a new language do not have to be competing or replacing existing languages. It can be develop purely out of curiosity and fun:)

Rust resolves all of that at compile time: null pointers simply cannot be created using standard language facilities, freeing is carried out without any intervention of the programmer and is proven to be correct by the compiler.

That sounds great, but as I mentioned, to support such compiling time memory security, Rust has added extra details (and complexities). You mentioned this problem too: "it's too pointer-ownership-lifetime-oriented to a point that it looks like a logical game for paranoidally-inclined C fans". Ada is also a more secure language than C and C++, but you know what happened.

I think programmer intervention of memory management is acceptable, as long as, it does not get too tedious and the errors are detected as early as possible at runtime. For example, dereferencing null error happens when the object referenced by a reference is already deleted when it is dereferenced. The error should have been reported early on when the object was deleted while it is still being referenced, not when it is accessed later on.

So, ownership or reference counting? One is not compatible with the other, as they both solve the same task.

Well, they both determine when an object should be freed. Here the ownership, I meant it for a running time thing. It is compatible with reference counting, together they could check for anomalous memory deallocations.

Actually, any reference assignment or passing of it to a function must result in ownership transfer,

The ownership should tell who is responsible to free the object represented by the reference. At least, this is what I mean by ownership. So it cannot be transfer simply by passing it to function, because clearly the function or an variable in the function may not be responsible for the objects's deallocation.

Overall, it's not clear what such language is meant for and what it can offer to those who've seen Rust. I can imagine it can be simpler and higher-level at some points, but that is not enough for it to be adopted for serious use. If a systems language does not imbue its users with convincingly magical capabilities without any sacrifices and is not backed by major IT players and there is no (planned) enterprise-quality IDE with auto-completion and integrated help and etc. -- the prospects aren't bright.

I wouldn't worry for that.

Only a narrow specialization could keep such language afloat -- if you can find a niche where NeoDao would perfectly fit in, only then its development won't be in vain. That is my opinion.

Not sure what niche it might be. It will not depend on the language, but depend on who might use it. In any case, the development may not be in vain, since the development will mainly be filling the gap between Dao and LLVM, it could be useful for DaoJIT in some way. Also the most important thing to think about this new language is about memory management and concurrency, such thinking could be helpful to Dao one way or another:)

[Night-walker]

Resolving all memory issues at compiling time sounds great, but in practice, all those things that make this work seem to have added too many extra details that the developers must master or pay attention to. I only went through the Rust documentation briefly, this is the impression I get, maybe I am wrong.

That's true, but Rust has become simpler over the last months. It's much simpler (in design) then C++. It's possible that it wouldn't require much efforts to add some implicitness to facilitate high-level programming -- just as you don't need to worry about pointers, references and memory sharing with QString in C++.

And it is the only language which has potential to replace not only C++ but C as well, which hasn't happened for you-know-how-many decades. I am a bit skeptical about this based on my impression.

I don't foretell Rust to replace C and C++. I just state the fact that it is the only well-known language which is capable of that because it combines modern, high-level concepts with zero-overhead mechanics, just like C++ in its time.

Maybe more ambitious, but not necessary much harder, especially when there is LLVM. Also the reasons to develop a new language do not have to be competing or replacing existing languages. It can be develop purely out of curiosity and fun:)

Well, it's still good when you can put your creation to actual use. If I didn't think that Dao has competitive potential, I wouldn't be as interested to work with and on it.

That sounds great, but as I mentioned, to support such compiling time memory security, Rust has added extra details (and complexities). You mentioned this problem too: "it's too pointer-ownership-lifetime-oriented to a point that it looks like a logical game for paranoidally-inclined C fans".

The thing is, complexity is acceptable when the compiler itself does not permit you to write code in a wrong way. That's far more time- and nerve-saving in the long run then spending nights in the debugger trying to understand why your mulrithreaded code which already went into production behaves differently depending on the moon phase. And Rust solves that. Even though coarsely and obtrusively, it gives the programmer assurances which are invaluable when writing large-scale systems meant to be both highly efficient and highly reliable.

I have seen kilometer-long C++ compiler messages when auto, &, &&, and templates are used together in a wrong way. I've read large articles with insights on poorly known or misunderstood aspects of C++ code behavior. Comparing to C++ (especially C++11), Rust is much more simple, clean and predictable language. It just lacks the ability to abstract away from the details, but the language is evolving and may eventually approach this problem.

The ownership should tell who is responsible to free the object represented by the reference. At least, this is what I mean by ownership. So it cannot be transfer simply by passing it to function, because clearly the function or an variable in the function may not be responsible for the objects's deallocation.

Then it's just plain reference counting (ARC), as I can hardly imagine how else what you're describing can be done in concurrent environment. Properly freeing heap-allocated data is not as important as preventing undesirable sharing of that data, which cannot be ensured without something like ownership transfer. If sharing is not supervised, ownership simply looses its very meaning.

Also the most important thing to think about this new language is about memory management and concurrency, such thinking could be helpful to Dao one way or another:)

Thinking may certainly be helpful :) But it will require something more then few simple checks to devise a systems programming language which could look competitive when compared to Rust. And anything less then that would simply not be interesting :)

[dumblob]

That's true, but Rust has become simpler over the last months. It's much simpler (in design) then C++.

Well, can I find some brief list of new/removed things (in the last 12 months)? I'm quite curious as the overall complexity was the main reason why I abandoned Rust after few school-exercise-sized programs. It felt good, but really too obtrusive that I couldn't remember every small corner of the language (I admit, I didn't learn it by hard, but on the other hand I also didn't see anywhere any full conceptual description of the language allowing me to derive those corners of the language - just few random parts - maybe because of it's incompleteness at that time).

[dumblob]

It just lacks the ability to abstract away from the details, but the language is evolving and may eventually approach this problem.

I'm not so sure about the may wording. I have a feeling that authors of Rust are not so into such high-level view as e.g. in Dao.

[Night-walker]

There is The Rust Guide and Road to Rust 1.0 if you want to know the current state of the language. In short, they removed all pointer types excluding &, which made things noticeably simpler then before. Nevertheless, the language still has maniacal bias toward handling each piece of data as if it was the last drop of water in a desert.

I'm not so sure about the may wording. I have a feeling that authors of Rust are not so into such high-level view as e.g. in Dao.

Rust is quite high-level when it comes to functional programming, in certain cases even higher-level then Dao (but that's not very significant). But right now Rust is indeed too pointer-oriented to be convenient for applied programming, and there is no evidence that its developers intend to do anything to change that.

Overall, I don't think Rust is a perfect systems programming language. Currently it's not the language I would want to use. But it's cleverly designed and worth being taken into account.

[daokoder]

Then it's just plain reference counting (ARC), as I can hardly imagine how else what you're describing can be done in concurrent environment.

They are actually different, ownership is owned by one, reference count is counted by many. Anyway, we are probably talking about different things.

But it will require something more then few simple checks to devise a systems programming language which could look competitive when compared to Rust. And anything less then that would simply not be interesting :)

I don't think Rust should the examplar that new system languages should look to. I think any language that is simple, expressive, efficient and safe would be competitive (and interesting). It does not have to be complete compiling time safety as in Rust. A combination of compiling time safety and running time safety would be much better if such combination can make the language much simpler. Also simplicity should be not just about simplicity of using the language, but also be about the design and implementation of a language. I think such simplicity is rather important for a system language.

Rust is still too complicated for the masses, I doubt it can get any simpler because of its emphasis on its current features. Its complexity in its design and implementation could be a huge obstacle for an independent implementation of the language. Though the need for an independent implementation is very rare, such need will eventually arise for a system language. Of course, I cannot prove this possibility since I cannot predict the future:)

[Night-walker]

Basing it on Dao means a large portion of the Dao implementation could be reused, in particular, the parser, the type system, the bytecode format and the virtual machine.

Since new language would share large part of Dao both in syntax and internal implementation, I wonder if it's possible to implement it as a sibling language alongside Dao on top of existing code base. DaoVM could incorporate changes allowing it to support processing of the modified syntax and external code generation. Then, as with DaoJIT, a separate library would allow to diverge from the normal code processing sequence used for Dao, instead employing LLVM facilities to produce the bitcode.

If that is somehow possible, it would allow to seamlessly use the sibling languages together as parts of a single platform similarly to .NET. For instance, you could easily load LLVM-compiled (NeoDao) libraries within Dao code -- DaoVM would automatically recognize that, load the external code generator, process the code meant to be compiled, build the libraries using LLVM and link them to Dao code by automatically adding bindings to bridge these two worlds.

[daokoder]

Since new language would share large part of Dao both in syntax and internal implementation, I wonder if it's possible to implement it as a sibling language alongside Dao on top of existing code base.

It can start from the existing code base. But if you mean sharing the code base, it may be possible, if we don't mind DaoVM becoming bloated. However, it is better to keep them separate so that the development of Dao will not be interfered.

DaoVM could incorporate changes allowing it to support processing of the modified syntax and external code generation. Then, as with DaoJIT, a separate library would allow to diverge from the normal code processing sequence used for Dao, instead employing LLVM facilities to produce the bitcode.

This is definitely possible.

If that is somehow possible, it would allow to seamlessly use the sibling languages together as parts of a single platform similarly to .NET. For instance, you could easily load LLVM-compiled (NeoDao) libraries within Dao code -- DaoVM would automatically recognize that, load the external code generator, process the code meant to be compiled, build the libraries using LLVM and link them to Dao code by automatically adding bindings to bridge these two worlds.

This was more or less what's in my mind when I mentioned "excellent interpolation with Dao":).

But I am not sure if they can be called as a single platform, as the involved components will be quite loosely coupled.

[Night-walker]

But I am not sure if they can be called as a single platform, as the involved components will be quite loosely coupled.

It would be a huge advantage to be able to share not only parts of the implementations of the languages, but also the interface and the libraries (at least partially), so that there is no need to have two sets of standard components tailored for each language independently. Then it would be much easier to learn, use, develop and maintain both languages under a single unified platform, no matter how different their handling may be under the hood.

[daokoder]

but also the interface and the libraries (at least partially), so that there is no need to have two sets of standard components tailored for each language independently.

NeoDao should be able to import Dao modules seamlessly. And the standard NeoDao libraries (for string, list and map etc.) would be designed as close to the corresponding Dao types as possible.

Then it would be much easier to learn, use, develop and maintain both languages under a single unified platform, no matter how different their handling may be under the hood.

Yes, knowing one of the two should be easy to learn the other. But there will be some significant syntax differences, even though the majority part will be essentially the same.

[Night-walker]

But there will be some significant syntax differences, even though the majority part will be essentially the same.

Learning the syntax of a language which is based on familiar concepts is a matter of hours. Learning and keeping in mind the whole array of necessary library-based functionality requires much more than that.

If languages share standard modules, and language-level (user-written) code can be seamlessly interconnected, then it's apparently a single platform. It provides tremendous advantages both for the development and usage of such languages, so it makes sense to tight them together as much as possible from the very beginning.

[daokoder]

Learning the syntax of a language which is based on familiar concepts is a matter of hours. Learning and keeping in mind the whole array of necessary library-based functionality requires much more than that.

Yes, that's why I intend that the standard types and libraries of NeoDao should be designed as close to those of Dao as possible.

so it makes sense to tight them together as much as possible from the very beginning.

I agree with everything you said except this sentence. I think it is a bad idea to tight them together too soon, because NeoDao is still hypothetical, and there are still many things unclear, they should not be tightened together prematurely. However the idea of one platform should be kept in mind during the design and implementation of both languages, so that when it becomes possible and necessary, they can be tightened together.

[Night-walker]

However the idea of one platform should be kept in mind during the design and implementation of both languages, so that when it becomes possible and necessary, they can be tightened together.

Software projects are rarely known to be joined together, they usually just fork. "When it becomes necessary" often appears too late to do anything about it, so I think it is reasonable to keep in mind the possibility of multi-language platform whenever decisions are made.

[daokoder]

There are probably a couple of things we should think about now. For a system language, list, map and numeric array cannot be supported as builtin types, so their enumeration syntaxes may be removed or change to something like C++ initializer list using {}, this means [] can be freed for other use. I wonder if [] would be a better choice than () for tuples, because () has issues with single item tuples.

Also I intend to support tuple as a builtin type just like in other languages such as Rust. So its type name would be better in the form of (int,string) or [int,string], since tuple type wouldn't be a typical generic type that will be supported in NeoDao, and tuple<int,string> no longer looks appropriate. Also I wonder if <> should be avoided in non-generic type names, this means we may also need to change enum<a,b> to something like enum{a,b} and routine<x:int=>string> to routine(x:int)=>string etc. These changes would actually look better, aren't they?

Here is the thing, do we make these changes Dao too for better consistency with the hypothetical NeoDao? These changes seems quite reasonable per se. It might even be possible to remove the keyword tuple, but I have been considering a new syntax such as tuple name { type-list }, after all, we have enum name { sym-list } and class name { ... } etc.

But changing the enumeration syntaxes seems too big a change. In particular, I wonder if we should remove [] for numeric array, and use something like array{ ... }, and switch from () to [] for tuples. Or we keep [] for numeric and () for tuple in Dao, and use () for tuples in NeoDao. Then I just feel it is a bit pity that [] is a bit wasted, and we will suffer a bit the one item issue of ().

[daokoder]

On the other hand, we may not be able to avoid <> completely for non-generic types. For example, builtin array type is likely to be denotated with TypeName[], then it would be problematic to declare an array of variant types or routine types, because there would be ambiguity with type names such as int | string []. A natural solution could be <int | string>[], of course, {} could also be used instead.

[daokoder]

OK, I just realized that, using [] for tuple type names such as [int,string] might not be as clean as I thought, when it appears in builtin array types such as [int,string][5]. So maybe better to keep () for tuples, then we can keep the enumeration syntax unchanged for list, map and array:)

[dumblob]

Nice elaboration. I'm actually quite satisfied with the current syntax for tuples. The single-item tuple might seem as an issue, but in practice, a single-item tuple is not so much useful (the cases I've come up with are more easily solvable by disjoint types - often with none).

[Night-walker]

A notation like int[], string {} may create ambiguity both for the parser and the user when the language has such a rich syntax as in the case of Dao. list<...>, map<...>, etc. may be verbose, but also very clear and readable, particularly when you have maps of lists of tuples etc.

I suppose there isn't really any premise to use type [] syntax for arrays, as they won't be just plain memory chunks as in C (unless you mean static LLVM vectors).

As for the enumeration syntax, perhaps we could use [] for all containers (not counting tuples) in Dao? It would make the overall syntax a bit more clean and clear ({} for code only), with the unavoidable problem of distinguishing arrays (and, to lesser extent, code section argument blocks).

Array enumerations are quite nice for matrices, I wouldn't like to see them removed. The most natural alternative I can think of is (1 2 3) -- easily distinguished from tuple by the absence of comma, but may appear exceedingly problematic to parse.

I am still in doubt whether it's a good idea then.

[daokoder]

list<...>, map<...>, etc. may be verbose, but also very clear and readable, particularly when you have maps of lists of tuples etc.

I suppose there isn't really any premise to use type [] syntax for arrays, as they won't be just plain memory chunks as in C (unless you mean static LLVM vectors).

list<...> and map<...> will be used for the standard list and map types. int[] (and int[5] too) is for builtin types (plain memory chunks with sizes), such types should have no methods, otherwise there will be need for a runtime which I will try to avoid. So this will be very plain type unlike the typical container type, it would be inappropriate to use list<...> type name.

I just found out that Rust uses [int] and [int,..5] for such types, I had considered [int] before, but just felt not convenient to put size in it. I don't know why Rust uses [int,..5], it looks quite awkward.

As for the enumeration syntax, perhaps we could use [] for all containers (not counting tuples) in Dao?

It's better to keep them unchanged.

[Night-walker]

I just found out that Rust uses [int] and [int,..5] for such types, I had considered [int] before, but just felt not convenient to put size in it. I don't know why Rust uses [int,..5], it looks quite awkward.

LLVM has vector type, which is a constant-sized array. Rust reflects this with specific [type, ..size] syntax.

[daokoder]

LLVM has vector type, which is a constant-sized array. Rust reflects this with specific [type, ..size] syntax.

I understood that. I was merely surprised by their choice of this awkward syntax [type, ..size], I would simply use [type, size].

[Night-walker]

Now, what I would want more then anything else in a system language is default passing of non-scalar types by reference as in C#. I don't see any justification for using explicit & other then for the reason of syntax consistency.

[daokoder]

Now, what I would want more then anything else in a system language is default passing of non-scalar types by reference as in C#.

This will bring more problems in the language. Suppose the following function will pass its parameter by reference:

routine Test( obj: SomeType )
{
...
}

So here SomeType would be supposed to represent a reference type. Then you will need something else to represent the value itself, but any name other than SomeType would be a confusion, since it is clearly more logical that SomeType should represent SomeType's value. Using SomeType to represent reference is especially confusing in class:

class Klass
{
    var  index : int;  // reference?
    var  obj: SomeType;  // reference?
    var  index2 : %int;  // value? Assuming % is used for value type;
    var  obj2: %SomeType; // object?
}

[Night-walker]

There should be no confusion.

var index : int; // reference?

No, int is a primitive type.

var obj: SomeType; // reference?

Yes.

var index2 : %int; // value? Assuming % is used for value type;

Inapplicable to int.

var obj2: %SomeType; // object?

To formulate if more precise, an inlined object.

[Night-walker]

If the hypothetical language cannot abstract from pointers, it will not be any better then Rust. I don't even see a reason to develop it if it will boil down to the same pointer swarming.

[daokoder]

To formulate if more precise, an inlined object.

That does not look natural.

If the hypothetical language cannot abstract from pointers, it will not be any better then Rust. I don't even see a reason to develop it if it will boil down to the same pointer swarming.

There will be no pointers, just references. Every non-scalar objects will be accessed through references, one type of reference for users, but multiple internal types of references under the hood.

[Night-walker]

There will be no pointers, just references. Every non-scalar objects will be accessed through references, one type of reference for users, but multiple internal types of references under the hood.

Rust also has no pointers, just references -- and what? As I said, it is in my opinion absolutely pointless to make a simplified Rust clone. And using implicit references as in higher-level languages would make difference.

[daokoder]

As I said, it is in my opinion absolutely pointless to make a simplified Rust clone.

Where did you get the impression that I want a Rust clone? Rust is not even close to the kind of system programming language I like:)

And using implicit references as in higher-level languages would make difference.

Implicit reference has problems, its use with non-scalars is not consistent with the use of scalars. It must have other caveats when getting into details, because distinguishing object (value) and reference is crucial for a system language (for memory layout, assignment and copying etc.).

[Night-walker]

Where did you get the impression that I want a Rust clone? Rust is not even close to the kind of system programming language I like:)

For now, I don't see any advantages over Rust in memory handling. And I actually would like them to exist.

distinguishing object (value) and reference is crucial for a system language

It can always be determined by the type. I don't understand what's the reason to force the usage of & everywhere. Just because it's how C/C++ handle things? I don't see any other real reason for that. Making references to primitive types? Arguably useless. Assignment and copying? Can be solved regardless of the default passing method.

[daokoder]

For now, I don't see any advantages over Rust in memory handling. And I actually would like them to exist.

Your mean complete compiling time safety, right? As I said before, I prefer a combination of compiling time and running time safety with much simpler syntax. Yeah, I never pretend this language will have better memory handling than Rust, I wouldn't trade simplicity with a better but much more complicated memory handling.

It can always be determined by the type. I don't understand what's the reason to force the usage of & everywhere.

They would be use mostly in fields and parameters. Why everywhere? What's the difference if you have to use something else to represent values/objects? Your implicit reference would be just like flipping a coin.

Just because it's how C/C++ handle things?

Speculation.

I don't see any other real reason for that. Making references to primitive types? Arguably useless. Assignment and copying? Can be solved regardless of the default passing method.

Well, if you don't consider avoiding the inconsistency (where var a: int would be a value and var a: SomeClassType would be a reference) between scalars and non-scalars is a real reason, nothing else I could say would be real reasons. I must say, this inconsistency alone is a sufficient reason to reject such implicit references.

[Night-walker]

Your mean complete compiling time safety, right? As I said before, I prefer a combination of compiling time and running time safety with much simpler syntax. Yeah, I never pretend this language will have better memory handling than Rust, I wouldn't trade simplicity with a better but much more complicated memory handling.

No, I actually meant simplicity and abstractness which are hampered by the constant need to use explicit references a-la C.

They would be use mostly in fields and parameters. Why everywhere? What's the difference if you have to use something else to represent values/objects? Your implicit reference would be just like flipping a coin.

Not at all. The need to use non-scalar data by value mostly makes sense only for structure fields, where you may want to avoid extra memory allocation and extra pointer dereferencing, and it's not very important even for a systems programming language. Other then that, the use of & is just a nuisance. A ritual stretching its roots to C. Making referencing implicit is one of the largest differences between e.g. C++ and Java/C#.

Well, if you don't consider avoiding the inconsistency (where var a: int would be a value and var a: SomeClassType would be a reference) between scalars and non-scalars is a real reason, nothing else I could say would be real reasons. I must say, this inconsistency alone is a sufficient reason to reject such implicit references.

It's apparently not an inconsistency for Dao, as well as for many other languages, including compiled D. I don't see how it suddenly becomes critical for a systems programming language.

[daokoder]

The need to use non-scalar data by value mostly makes sense only for structure fields, where you may want to avoid extra memory allocation and extra pointer dereferencing, and it's not very important even for a systems programming language.

You forgot to mention array of objects. And it is not for avoiding a few extra memory allocations.

Making referencing implicit is one of the largest differences between e.g. C++ and Java/C#.

And one of them is a system language, the others are not.

It's apparently not an inconsistency for Dao, as well as for many other languages, including compiled D.

In Dao, that's because scalars are also objects.

For D, I am not familiar, please give some examples.

By the way, your implicit reference requires distinguishing scalar and non-scalar types, which could be problematic. Using C for an example, do you treat struct myint64 { int high, low; } a scalar or non-scalar?

[Night-walker]

The need to use non-scalar data by value mostly makes sense only for structure fields, where you may want to avoid extra memory allocation and extra pointer dereferencing, and it's not very important even for a systems programming language. You forgot to mention array of objects. And it is not for avoiding a few extra memory allocations.

I don't quite remember arrays of objects laid continuously in memory to be used in C interfaces I've worked with. That is inconvenient most of the time, I suppose. For instance, getting a reference to an object inside such array is quite unsafe and may be problematic to control. Setting values in it always results in their full copying. Adding or inserting to such array may result in copying large chunks of memory. I hereby consider this a marginal case :)

It's apparently not an inconsistency for Dao, as well as for many other languages, including compiled D. In Dao, that's because scalars are also objects.

Sophistic :) What I meant is that non-primitive data cannot be shared between several contexts, while non-primitive one is always shared. That's as if the former was passed by value and the latter by reference.

For D, I am not familiar, please give some examples.

I also don't know much about D. Brief investigation revealed just that "Class objects are instantiated by reference only".

By the way, your implicit reference requires distinguishing scalar and non-scalar types, which could be problematic. Using C for an example, do you treat struct myint64 { int high, low; } a scalar or non-scalar?

This can be trivially resolved by distinguishing tuples from structures. Tuples are considered primitive data and are passed by value. Structures are non-primitive and are passed by reference. Other then that, the structures and tuples may be largely similar, if not identical. Thus various objects which are scalar by meaning (numbers, datetime values, etc.) can be represented as tuples, and everything else obviously falls into the category of structures.

[dumblob]

This can be trivially resolved by distinguishing tuples from structures.

If we generalize this, we could probably narrow down the problem to having a "reference" variant and "primitive" variant for each existing type (let's call it type dualism :)). For many of them it's useless to have both variants, but for some it's not (e.g. for tuples - both variants will definitely find their use in a system language). I'm quite sure, that if we carefully choose a proper subset of reference variants and primitive variants, we'll end up with a decent number of types (definitely higher than the number of types in Dao, but not that much I think) without the need of (de)referencing them.

[Night-walker]

If we generalize this, we could probably narrow down the problem to having a "reference" variant and "primitive" variant for each existing type (let's call it type dualism :)).

There is no need for that excluding the hypothetical tuple-struct pair. For the reasons of simplicity and safety, scalar types need not be passed by reference. And some other non-scalar types like arrays need not be passed by value (only copied on manual request). Simple and safe.

[daokoder]

This can be trivially resolved by distinguishing tuples from structures. Tuples are considered primitive data and are passed by value. Structures are non-primitive and are passed by reference. Other then that, the structures and tuples may be largely similar, if not identical. Thus various objects which are scalar by meaning (numbers, datetime values, etc.) can be represented as tuples, and everything else obviously falls into the category of structures.

The problem of tuples for this is that they don't have methods, so they will not be sufficient for types such as datetime etc.

I just checked D with a few lines of code. It turns out that it distinguish struct and class this way: a variable of struct holds a value, while a variable of class holds a reference. Maybe we can introduce struct and make such distinction too. This way, there will be no need for special syntax to represent a value of a class type. Such special syntax introduces inconsistency, which was what I strongly against. Without such syntax, it should be acceptable and seems even better.

[Night-walker]

The problem of tuples for this is that they don't have methods, so they will not be sufficient for types such as datetime etc.

You a bit misunderstood me. I wasn't talking about Dao tuples, "tuple" was just the first and most appropriate term I came with, I could just as well call it "value class" or something like that. For instance, Rust have tuples, structs and tuple structs, and all of them may have methods introduced via traits, there are no principle limitations here.

I just checked D with a few lines of code. It turns out that it distinguish struct and class this way: a variable of struct holds a value, while a variable of class holds a reference. Maybe we can introduce struct and make such distinction too. This way, there will be no need for special syntax to represent a value of a class type. Such special syntax introduces inconsistency, which was what I strongly against. Without such syntax, it should be acceptable and seems even better.

That's more or less what I've been talking about.

[daokoder]

You a bit misunderstood me. I wasn't talking about Dao tuples, "tuple" was just the first and most appropriate term I came with, I could just as well call it "value class" or something like that. For instance, Rust have tuples, structs and tuple structs, and all of them may have methods introduced via traits, there are no principle limitations here.

I knew you meant tuple type in the new system language. But I prefer builtin types to have no methods, it would be much simpler.

[Night-walker]

By the way, null pointers can be completely avoided by the use of variant typing similarly to Dao and Rust. That allows to avoid needless checks both for the user and the VM.

[Night-walker]

There will, however, be need to distinguish heap- and stack-allocated string/array literals. And the need to create uninitialized arrays of specified size. That may require certain syntax derivations.

[Night-walker]

Automatic return will be disabled.

That's a step back. As their will be some closures or code sections, as well as numerous small functions, there will be need for short and expressive notation without the archaism of mandatory return.

[daokoder]

By the way, null pointers can be completely avoided by the use of variant typing similarly to Dao and Rust. That allows to avoid needless checks both for the user and the VM.

Sure.

There will, however, be need to distinguish heap- and stack-allocated string/array literals. And the need to create uninitialized arrays of specified size. That may require certain syntax derivations.

I am considering to support the new keyword for allocation on heap.

var object = SomeClass()  # On stack;
var object = new SomeClass()  # On heap;

var a : [int] = { 1, 2, 3 }  # On stack;
var a : [int] = new { 1, 2, 3 }  # On heap;

var a : [int, 100] = new { 1, 2, 3, 4, 5 }  # Initializing with the repetition of 5 elements;
var a : [int, 100] = new {}  # Uninitialized;

That's a step back. As their will be some closures or code sections, as well as numerous small functions, there will be need for short and expressive notation without the archaism of mandatory return .

I probably wrote that down under a bit confusion when I forgot the auto return in Dao is not the exactly same as those in other languages such as Rust.

You should not consider anything there as final, especially you should not take anything below the Radom ideas below section seriously:)

[zaxebo1]

@daokoder : i replied to you on your NeoDao idea at https://gist.github.com/daokoder/1621c0c106a30f0ee8af :-)

[dumblob]

@zaxebo1, it seems we're talking at cross purposes. NeoDao is not as much about implementation, but rather about the language itself. NeoDao is not about being a messiah, but rather a decently small and uncomplicated system programming language offering very high concepts and nearly 100% memory safety.

Of course, implementation matters, but it's not an issue nowadays when we have bunch of languages (compiled or interpreted), bytecodes and intermediate representations etc. running almost everywhere.

[Night-walker]

@zaxebo1, you're missing several important points here:

1. NeoDao is obviously targeted at LLVM, which is the only easy way to implement a compiled language. LLVM is bound to C/C++ API.
2. Haxe is not a systems programming programming language and no converter or backend will be able to change that. It cannot possibly have the efficiency and flexibility to implement a compiler or VM.
3. Since C++ (1984?), there weren't any new systems programming languages I am aware of, until Rust. Go does not belong to that category. Maybe D, but it's not positioned as such.

But you are right that the language should be visibly original to be recognized.

[zaxebo1]

From my point of view, (A)haxe generates C++ source code. (and also "Haxe to pure C" source code generation work is also underway) (B)Now, clang compiles C++ to llvm in extremely optimal way. see ( http://clang.llvm.org/ )

==>So, we can use haxe as systems programming language.

Infact i have already bound a large number of systems API of C to haxe, which runs obviously via "LLVM" through clang.

Tomorrow, if Microsoft makes a derivative of C# as basis of its system programing for its OS on Desktop/Xbox/Windows Mobile/Embedded Platforms/.... http://en.wikipedia.org/wiki/Singularity_(operating_system)
Singularity OS a research OS of Microsoft , a highly-dependable OS in which the kernel, device drivers, and applications were all written in managed code.

Similar .NET like managed code OS, Framework from Microsoft http://en.wikipedia.org/wiki/Midori_%28operating_system%29 See library OS of drawbridge from Microsoft http://research.microsoft.com/en-us/projects/drawbridge/ The systems programming here is done totally in managed code.

Now, in that case the same Haxe can generate C# (or derivative ) source easily and we can easily bound the managed code API to haxe. Its will not be mix of unmanaged and managed code. It will be "a pure managed code for systems programming" in this new OS too.

for example2:
http://en.wikipedia.org/wiki/.NET_Micro_Framework this platform only allows systems programming in .NET/CLR . and Not in C/C++. Here, haxe can easily generate PURE managed code in C# for SYSTEMS PROGRAMMING LANGUAGE

We now only need another layer on top of haxe , ie, a full programming language implemented in pure haxe to run everywhere , on everyVM , or via llvm ; and in scripting and in systems programming language. And all this does not require separate implementations. It only requires a single implementation of Dao in just one language Haxe.

It will solve systems programming language problem in one more new way, adding an important new thing. If you implement just one more programming language in C compiling to llvm, then after few years it may again get lost among crowd, in absence of what new it adds.

"We can't solve problems by using the same kind of thinking we used when we created them." -- Albert Einstein

we need to tread again a new systems programming language not with the same old path. Whats new major leap we offer, over the older ones. And one major new dimension is simple: "implementation in haxe". IMHO.

[dumblob]

@Night-walker, just to clarify "Go does not belong to that category.". Go is capable of being used as a system programming language. The only bottleneck is, that you'd have to write the allocator yourself using the unsafe package (which is actually not so far away from what os kernels do :)).

@zaxebo1, keep in mind, that Haxe is one of the slowest languages due to it's design. Disregarding Haxe, each level of translation significantly lessens the possibility of good optimizations and also inevitably makes the result unnecessarily bloated (i.e. slow per se).

[dumblob]

Btw I've recently discovered an interesting experiment to create a system-programming capable language while retaining the overall safety and simplicity of solely garbage collected languages. The project is quite young with first commit from 13 May 2012, but it's worth looking at as it may impose some new ideas.

https://github.com/zdevito/terra

Actually, from my point of view, the only thing it does is compilation of designated statically-typed parts of code (these are written in the Terra language) into native code using clang according to some type-inference rules allowing seamless interlay with non-designated parts (these are written in standard Lua). It's similar to embedding ASM code into C except that the designated parts can be constructed and compiled also in runtime (sort of programmer-guided JIT compilation). The performance results are then of course very good (indeed comparable to C).