Open tueda opened 6 years ago
Hi Takahiro,
I have one problem with the proposal. I would rather see
Actually with your #global the #namespace might even have been superfluous if it were not for backward compatibility. We just prepend the name of the procedure. (and the _). The reason is that I want the namespace inside is that the procedure files have to start with #procedure
Maybe be can define a localprocedure that has this feature. That way the local procedure starts with
and can reside in the file procx.prc just like a normal procedure. For the existing programs nothing changes and also not for the calling of the procedures. A local variable a of procx can be referred to as procx_a and all should be well.
It leaves the question of calls inside calls and whether we want to nest the namespaces. It looks like a flat namespace (just one level) would already do the job.
What do you think? Did I miss something in your setup?
Cheers
Jos
On 14 Nov 2017, at 17:58, Takahiro Ueda notifications@github.com wrote:
Namespaces, or scopes of name bindings in general, are essential to construct robust programs/libraries in a readable way. FORM lacks for supports of them except preprocessor variables and procedures on the call-stack. (.global may be hard to use for this purpose). Without some mechanism to hide internal code of a library, users unintentionally interfere with the internals and may easily get wrong results. Then, the library should contain a message like this https://github.com/benruijl/forcer/blob/fab41c858bd9df936310c25ec3c85ea0bbd91e99/forcer.h#L5-L6. In principle, library developers can put some unique prefix for all local objects, e.g., MyLibraryPrivateXYZ, but this certainly obfuscates the code and worsens the readability and maintainability.
Probably you have your preference about how FORM should implement namespace. Here I just record what I thought. It is compatible with the current version of the FORM langauge; if one doesn't use namespaces, the code just works. For now, the implementation difficulty is not considered.
namespace
... #endnamespace hides all internal objects, except objects explicitly specified by #global . Similar to Mathematica's BeginPackage["Package
"] ... exported symbols ... Begin["
Private`"] ... End[], EndPackage[], in a sense. This should be applied for all names: symbols, $-variables, procedure names, preprocessor variables, etc.namespace MyLibrary
global globalx;
global $globalx;
global GlobalProc;
ifndef `Setting1'
define Setting1 "0"
endif
S globalx,localx;
procedure LocalProc()
id localx = 0;
endprocedure
procedure GlobalProc()
id globalx = 0; $globalx = 0;
endprocedure
endnamespace
In the above code, globalx, $globalx and GlobalProc are at the global scope, but the others are in the MyLibrary namespace. Access to the local objects from outside requires the fullname, by using _ for the namespace separator:
define MyLibrary_Setting "1"
call MyLibrary_LocalProc()
call GlobalProc()
message `$globalx'
Internally, procedures should remember in which namespace they are defined, to resolve object names when expanded.
proclocal
to make local objects accessible only in a procedure. Similar to Mathematica's Module[{...}, ...], in a sense, and shadows the specified names. Sometimes one want to use local objects only in a procedure:
procedure Proc(x)
proclocal n
S n; id
x'^n? = n *
x'^n / x;endprocedure
Maybe #local as the syntax instead of #proclocal would be just fine. Note that symbols could be declared (or Moduleoption local for $-variables) anywhere once the patch in #188 https://github.com/vermaseren/form/issues/188 would is merged.
I would like to apply #proclocal also for the procedure arguments, in such a way that the following example works:
procedure Foo(out)
proclocal out
redefine `out' "OK"
endprocedure
define out
call Foo(out)
message `out'
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/vermaseren/form/issues/236, or mute the thread https://github.com/notifications/unsubscribe-auth/AFLxEhPP-HbLwLYoBG2vR9oTzOCfpD8nks5s2cadgaJpZM4Qdqay.
#procedure myproc
#namespace myproc
#endnamespace
#endprocedure
should also work: in this case the procedure myproc
is not in the namespace myproc
. #localprocedure
would be OK for this purpose, though the name "localprocedure" is a bit obscure: the procedure is local? or objects in the procedure are local?
Actually I have something like C++ namespace in mind and a library in .h
file would look like:
#namespace SomeLibrary
#namespace detail
* Really private stuffs.
#endnamespace
* Many procedures sharing objects in the same private namespace.
#endnamespace
Actually with your #global the #namespace might even have been superfluous if it were not for backward compatibility. We just prepend the name of the procedure. (and the _).
This may be true. But a benefit of namespace is that in a limited scope long names can be written by shorter names. #global
would be generalized as #using
, making an alias as using
in C++11:
#namespace MyLib
* Applied only in this namespace.
#using a = _Foo_Bar_a
#endnamespace
(And, well, maybe it is just my feeling: names starting with _
look somewhat private, (1) in C/C++, identifiers starting with _
are reserved (2) in Python, identifiers starting with _
are recommended to have a protected scope.)
Hi takahiro
I am not familiar with the namespace and using of C++11.
The idea is to use #namespace and #endnamespace in a ‘free’ way like you describe below. I’ll forget about the #localprocedure. This #using you will have to explain a bit better.
There remain a few little problems. If I use
#namespace …
#endnamespace
and I call the procedure recursively things can become quite a mess. Another is names with []. My name scanning procedures have to become a lot more sophisticated. I am still unsure what to do about
Symbol y1; * generates x1_y1
Symbol y2; * generates x2_x1_y2
id y = y1+y2; * This should still be unambiguous
The problem starts with autodeclare. This can become very confusing when namespaces and autodeclare interfere. Maybe in the end there is no problem, but for now I cannot oversee it all. For the rest, I think we start having the concept under control.
Cheers
Jos
On 14 Nov 2017, at 22:46, Takahiro Ueda notifications@github.com wrote:
procedure myproc
namespace myproc
endnamespace
endprocedure
should also work: in this case the procedure myproc is not in the namespace myproc. #localprocedure would be OK for this purpose, though the name "localprocedure" is a bit obscure: the procedure is local? or objects in the procedure are local?
Actually I have something like C++ namespace in mind and a library in .h file would look like:
namespace SomeLibrary
namespace detail
Really private stuffs.
endnamespace
Many procedures sharing objects in the same private namespace.
endnamespace
Actually with your #global the #namespace might even have been superfluous if it were not for backward compatibility. We just prepend the name of the procedure. (and the _).
This may be true. But a benefit of namespace is that in a limited scope long names can be written by shorter names. #global would be generalized as #using, making an alias as using in C++11:
namespace MyLib
- Applied only in this namespace.
using a = _Foo_Bar_a
endnamespace
(And, well, maybe it is just my feeling: names starting with look somewhat private, (1) in C/C++, identifiers starting with are reserved (2) in Python, identifiers starting with _ are recommended to have a protected scope.)
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/vermaseren/form/issues/236#issuecomment-344409745, or mute the thread https://github.com/notifications/unsubscribe-auth/AFLxEvsty4PXqH4WuoMAXsui3Tm_SNo8ks5s2goxgaJpZM4Qdqay.
To me, it is clear that
#namespace x1
Symbol y1;
#namespace x2
Symbol y2;
id y = y1 + y2;
#endnamespace
#endnamespace
should be treated as
Symbol x1_y1;
Symbol x1_x2_y2;
id x1_x2_y = x1_x2_y1 + x1_x2_y2;
and there is no ambiguity.
When one wants AutoDeclare local objects, one needs to autodeclare it in the namespace.
Probably, what I expect for []
would be
#namespace x
S [y];
#endnamespace
should be translated into
S [x_y];
A tricky thing is what happens for
#namespace lib
#procedure polyderiv(x)
S n;
id `x'^n? = n * `x'^n / `x';
#endprocedure
#endnamespace
#namespace my
#using polyderiv = _lib_polyderiv
#call polyderiv(z)
#endnamespace
One would expect it should be expanded as
S lib_n;
id my_z^lib_n? = lib_n * my_z^lib_n / my_z;
To realize this behaviour, the preprocessor has to know the namespace for `x'
at the expansion of it. I mentioned that procedures should remember in which namespace they are defined. The same for preprocessor variables. Namely, FORM needs to know where the expanded text (possibly at intermediate stages for resolving preprocessor variable names) comes from. #call polyderiv(z)
is effectively defines #define x "z"
, so this character z
is considered to come from the namespace my
.
There would be a situation that every character in an identifier comes from different namespaces. For example abc
with a
coming from namespace my1
, b
from my2
, c
from my3
. A reasonable rule would be to use the namespace of from the first character in the identifier. In this case my1_abc
.
One reason why people dislike preprocessors in other languages is because they are always "global" and ignores namespace/modular structures. An example is found in FORM: a header file included from form3.h. This file defines a lot of macros. So when one wants to add some features in FORM by using some library and writes code like
#include "form3.h"
#include <some_3rd_party_library.h>
one has to check if some_3rd_party_library.h
doesn't contain any identifiers replaced by these macros (like chartype
, functions
, vectors
, cbuf
, AC
etc.), even for its private code, deeply hidden by struct/class and namespaces. Because the library header may include other headers, maybe system headers depending on each system, it is virtually impossible to guarantee the robustness. FORM preprocessor needs to know namespace structure a bit and must avoid such catastrophic scenarios.
Hi Takahiro,
I think you got this wrong. Since x2_x1_y does not exist, it should start looking for one namespace higher, ie. x1_y which also does not exist and hence for y. The staement then becomes id y = x1_y1+x2_x1_y2; If you do not do it this way, you may get in lots of trouble. In your ‘solution’ you would get undefined variables and it is hard to access global variables. This all as far as I can see it.
Cheers
Jos
On 15 Nov 2017, at 11:35, Takahiro Ueda notifications@github.com wrote:
To me, it is clear that
namespace x1
Symbol y1;
namespace x2
Symbol y2; id y = y1+y2;
endnamespace
endnamespace
should be treated as
Symbol x1_y1; Symbol x1_x2_y2; id x1_x2_y = x1_x2_y1 + x1_x2_y2; and there is no ambiguity.
When one wants AutoDeclare local objects, one needs to autodeclare it in the namespace.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/vermaseren/form/issues/236#issuecomment-344552559, or mute the thread https://github.com/notifications/unsubscribe-auth/AFLxEsf1Q2TdcXaDph5v_K0FodRFesZwks5s2r5vgaJpZM4Qdqay.
Hi Takahiro
And what about [5/3+(3-1)] ? Thusfar I see only one solution and that is x_[5/3+(3-1)], but that may get into trouble sooner or later with set elements for built in sets. At the moment the built in sets should not give problems, if the compiler knows about them and hence has special code for them.
Jos
On 15 Nov 2017, at 11:40, Takahiro Ueda notifications@github.com wrote:
Probably, what I expect for [] would be
namespace x
S [y];
endnamespace
should be translated into
S [x_y]; — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/vermaseren/form/issues/236#issuecomment-344553788, or mute the thread https://github.com/notifications/unsubscribe-auth/AFLxElTrZspy5h303iiMHY16GsBnmGjAks5s2r-FgaJpZM4Qdqay.
Hi Takahiro,
I agree that we need to do something, but in your examples you only use variables. If I call a procedure with argument 1+a things are not quite this simple. The #using would have to be used for each individual variable. I do not know how practical that is. Maybe the same as for preprocessor variables which also have to be defined individually.
It looks like we still have to think a bit……
Jos
On 15 Nov 2017, at 13:19, Takahiro Ueda notifications@github.com wrote:
A tricky thing is what happens for
namespace lib
procedure polyderiv(x)
S n;
id
x'^n? = n *
x'^n / `x';endprocedure
endnamespace
namespace my
using polyderiv = _lib_polyderiv
call polyderiv(z)
endnamespace
One would expect it should be expanded as
S lib_n; id my_z^lib_n? = lib_n * my_z^lib_n / my_z; To realize this behaviour, the preprocessor has to know the namespace for `x' at the expansion of it. I mentioned that procedures should remember in which namespace they are defined. The same for preprocessor variables. Namely, FORM needs to know where the expanded text (possibly at intermediate stages for resolving preprocessor variable names) comes from. #call polyderiv(z) is effectively defines #define x "z", so this character z is considered to come from the namespace my.
There would be a situation that every character in an identifier comes from different namespaces. For example abc with a coming from namespace my1, b from my2, c from my3. A reasonable rule would be to use the namespace of from the first character in the identifier.
One reason why people dislike preprocessors in other languages is because they are always "global" and ignores namespace/modular structures. An example is found in FORM: a header file included from form3.h https://github.com/vermaseren/form/blob/954024f163e8d702e1f28c1dd99846c08b0333f8/sources/variable.h. This file defines a lot of macros. So when one wants to add some features in FORM by using some library and writes code like
include "form3.h"
include
one has to check if some_3rd_party_library.h doesn't contain any identifiers replaced by these macros (like chartype, functions, vectors, cbuf, AC etc.), even for its private code, deeply hidden by struct/class and namespaces. Because the library header may include other headers, maybe system headers depending on each system, it is virtually impossible to guarantee the robustness. FORM preprocessor needs to know namespace structure a bit and must avoid such catastrophic scenarios.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/vermaseren/form/issues/236#issuecomment-344575922, or mute the thread https://github.com/notifications/unsubscribe-auth/AFLxEgnk-2dQE7cZqLxv2CeAjGzPtwL2ks5s2ta_gaJpZM4Qdqay.
OK. The definitions of my #namespace
and your #namespace
are different. I expect just abbreviations of long names, while you expect a complete cascade loop-up. In my case
#namespace x1
Symbol y1;
#namespace x2
Symbol y2;
id y = y1 + y2;
#endnamespace
#endnamespace
should definitely give a compile error and this is the desired behaviour to forbid any possible programmers' confusions with objects in different levels. But maybe your way would work...
If I call a procedure with argument 1+a things are not quite this simple.
In my suggestion FORM knows which namespace the text 1+a
is originated from. So a
will belong to that namespace.
I would like to mention again about #proclocal
for preprocessor variables. Currently, a practical way to introduce a private namespace (with an ugly hack) is putting a prefix by defining preorcessor variables. For example,
#procedure Foo(a,b)
#define Private "FooPrivate"
#define x "`Private'x"
#define y "`Private'y"
* Use `x' and `y' as local objects.
#endprocedure
The namespace can be easily changed by the Private
variable later. Up to now it looks good. But when one wants to return a result from a procedure by #redefine
, the nightmare begins.
#procedure Foo(a,b,dest)
#define Private "FooPrivate"
#define x "`Private'x"
#define y "`Private'y"
* Use `x' and `y' as local objects.
#redefine `dest' "..."
#endprocedure
* Doesn't work!
#define x
#call Foo(1,2,x)
So, all preprocessor variables (including dummy arguments) inside procedures can conflict with the outside world.
Now, I'm wondering: if preprocessor variables inside procedures could get some implicit namespace, then the problem would be avoided. Maybe this solution can be relatively easily implemented, because it is all in the preprocessor(?) Because preprocessor variables are anyway put in the stack, this change doesn't affect existing programs(?), except #redefine
for a global (or top-level) preprocessor variables (for this some other solution is required).
Actually, what I need may be the #redefine
command that redefines a variable in higher-levels, outside the procedure:
#procedure foo(x)
#redefine,nonlocal `x' "2"
#endprocedure
#define x
#call foo(x)
(The choice of the optional keyword nonlocal
comes from Python, not necessarily.)
Hi Takahiro,
Thanks! I just read it and probably will read it a few times more the coming days, just to think of the implications and potential implementations. I am still not very enthusiastic about ::. Also the fact that super would be a reserved name is a bit against the Form naming conventions. Maybe we can talk about this on Friday/Saturday. Some routines, like GetName, would need quite some changes. Also the recognition of preprocessor variables becomes much more complicated, while the question is whether that is needed, because the preprocessor variables do go on a stack.
The fact that calculational languages (I could C and Rust amoung those) might be able to do away with a preprocessor, should be disconnected from whether Form can. In calculational languages I can make real subroutines, while in computer algebra that is next to impossible, but we can avoid this anyway with the procedures which are giant macro’s and hence a preprocessor feature. Effectively it is inline coding.
Just a few considerations that pop to mind…..
See you Friday.
Jos
On 14 Nov 2018, at 21:11, Takahiro Ueda notifications@github.com wrote:
Here is an RFC https://gist.github.com/tueda/3b2c601a19c93835ae544915cebbc9e3. @vermaseren https://github.com/vermaseren How about this?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vermaseren/form/issues/236#issuecomment-438641027, or mute the thread https://github.com/notifications/unsubscribe-auth/AFLxElS6CNcmwgHi0bZEtEttPkE9TJnVks5uvAhagaJpZM4Qdqay.
Maybe it would be OK to use the namespace separator and "super" as super_foo_bar
though it quite looks like a "word".
I think preprocessor variables also need the namespace system. The RFC contains an example using the fact that procedures' arguments are prefixed by a namespace (#procedure get
). Another example is that it's gameover when a Forcer user defines POS1
, for which I must say it is relatively easy to conflict.
Hi Takahiro,
I have been thinking about the trouble I get into if I put the preprocessor variables into the namespace as well. Considering that nearly always you can also use $-variables instead of #define, I will make life much easier by excluding the preprocessor variables. And apropos that super_foo_bar looks like a word: that is just a matter of what you are used to. But again, it makes the implementation a lot easier. I am still thinking about how to do the #use which will need to make temporary lists of variable names. And I will have to search through that list first. But if we are in nested namespaces this may get wild.
OK, I’ll keep thinking.
Jos
On 15 Nov 2018, at 11:17, Takahiro Ueda notifications@github.com wrote:
Maybe it would be OK to use the namespace separator and "super" as super_foo_bar though it quite looks like a "word".
I think preprocessor variables also need the namespace system. The RFC contains an example using the fact that procedures' arguments are prefixed by a namespace (#procedure get). Another example is that it's gameover when a Forcer user defines POS1, for which I must say it is relatively easy to conflict.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vermaseren/form/issues/236#issuecomment-438891478, or mute the thread https://github.com/notifications/unsubscribe-auth/AFLxEkfGJZsIxsgK1vlEOKDOX4bc7zpuks5uvM7SgaJpZM4Qdqay.
Hi Takahiro,
I have been thinking a bit more about the preprocessor variables. I still do not see how I can do that without messing up an awful lot. But indeed, the $ variables cannot do all the things a preprocessor variable can do, like manipulate strings. Could you think of a way by which we can make the dollar variables also manipulate strings without messing up at the executable level. Maybe some function nameof(x) and another string(this is a string) and one that can concatenate or clip? This may open completely new control possibilities.
Cheers
Jos
On 24 Nov 2018, at 11:04, Jos Vermaseren t68@nikhef.nl wrote:
Hi Takahiro,
I have been thinking about the trouble I get into if I put the preprocessor variables into the namespace as well. Considering that nearly always you can also use $-variables instead of #define, I will make life much easier by excluding the preprocessor variables. And apropos that super_foo_bar looks like a word: that is just a matter of what you are used to. But again, it makes the implementation a lot easier. I am still thinking about how to do the #use which will need to make temporary lists of variable names. And I will have to search through that list first. But if we are in nested namespaces this may get wild.
OK, I’ll keep thinking.
Jos
On 15 Nov 2018, at 11:17, Takahiro Ueda <notifications@github.com mailto:notifications@github.com> wrote:
Maybe it would be OK to use the namespace separator and "super" as super_foo_bar though it quite looks like a "word".
I think preprocessor variables also need the namespace system. The RFC contains an example using the fact that procedures' arguments are prefixed by a namespace (#procedure get). Another example is that it's gameover when a Forcer user defines POS1, for which I must say it is relatively easy to conflict.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vermaseren/form/issues/236#issuecomment-438891478, or mute the thread https://github.com/notifications/unsubscribe-auth/AFLxEkfGJZsIxsgK1vlEOKDOX4bc7zpuks5uvM7SgaJpZM4Qdqay.
What would be a problem if we make a syntax for $-variables assigning a string?
$x = "abc";
$y = "def";
$z = $x + $y; * "abcdef"
Perhaps it may require internally string_(...)
function or we may make a "string" data type in $-variables.
It is not so much a problem. It opens up many new possibilities and it is best to think about those before we do any definitive implementation.
And then there is: $x = “abc”; $y = 2; $z = $x+$y; ?????? Whether the last makes sense, depends on what we really want. It merits some thinking…..
On 25 Nov 2018, at 17:03, Takahiro Ueda notifications@github.com wrote:
What would be a problem if we make a syntax for $-variables assigning a string?
$x = "abc"; $y = "def"; $z = $x + $y; * "abcdef" Perhaps it may require internally string_(...) function or we may make a "string" data type in $-variables.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vermaseren/form/issues/236#issuecomment-441422655, or mute the thread https://github.com/notifications/unsubscribe-auth/AFLxEvou9aiTZNFpWHiesMwcF-6lNPdrks5uyk7pgaJpZM4Qdqay.
I would suggests 2 + "abc"
must give a runtime error as in Python, and the user has to explicitly convert an integer (or expression) to a string string_(2) + "abc"
, thought other languages like Java implicitly apply toString()
.
Here you run into a problem already. In Form + is commutative. In your strings it is not. In that sense multiplication is better prepared for it.
Imagine that in addition to symbols, vectors functions etc we also make string variables. What could we do with it? I am not thinking about imitating other languages. I think about new things. Maybe a noncommuting function whose argument is a string? String strfun; And then: …strfun(abc)strfun(def)…. Chainin strfun; gives strfun(abcdef) or alternatively strfun(abc,def) and id strfun(str1?,str2?) = strfun(str1str2); would give strfun(abcdef) and factarg would then give strfun(a,b,c,d,e,f) or things like that. The next question then would be what strfun(a)+strfun(b) would mean? No further meaning? Just an attachment to a term, each term with its private string information? There is a whole world of possibilities here.
On 25 Nov 2018, at 17:16, Takahiro Ueda notifications@github.com wrote:
I would suggests 2 + "abc" must give a runtime error as in Python, and the user has to explicitly convert an integer (or expression) to a string string_(2) + "abc", thought other languages like Java implicitly apply toString().
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vermaseren/form/issues/236#issuecomment-441423266, or mute the thread https://github.com/notifications/unsubscribe-auth/AFLxEpXyOgJ8FE1gWyfuI2FUx-cdk5nrks5uylHpgaJpZM4Qdqay.
Namespaces, or scopes of name bindings in general, are essential to construct robust programs/libraries in a readable way. FORM lacks for supports of them except preprocessor variables and procedures on the call-stack. (
.global
may be hard to use for this purpose). Without some mechanism to hide internal code of a library, users unintentionally interfere with the internals and may easily get wrong results. Then, the library should contain a message like this. In principle, library developers can put some unique prefix for all local objects, e.g.,MyLibraryPrivateXYZ
, but this certainly obfuscates the code and worsens the readability and maintainability.Probably you have your preference about how FORM should implement namespace. Here I just record what I thought. It is compatible with the current version of the FORM langauge; if one doesn't use namespaces, the code just works. For now, the implementation difficulty is not considered.
#namespace <name>
...#endnamespace
hides all internal objects, except objects explicitly specified by#global <names...>
.Similar to Mathematica's
BeginPackage["Package`"]
... exported symbols ...Begin["`Private`"]
...End[]
,EndPackage[]
, in a sense. This should be applied for all names: symbols, $-variables, procedure names, preprocessor variables, etc.In the above code,
globalx
,$globalx
andGlobalProc
are at the global scope, but the others are in theMyLibrary
namespace. Access to the local objects from outside requires the fullname, by using_
for the namespace separator:Internally, procedures should remember in which namespace they are defined, to resolve object names when expanded.
#proclocal <names...>
to make local objects accessible only in a procedure.Similar to Mathematica's
Module[{...}, ...]
, in a sense, and shadows the specified names. Sometimes one want to use local objects only in a procedure:Maybe
#local
as the syntax instead of#proclocal
would be just fine. Note that symbols could be declared (orModuleoption local
for $-variables) anywhere once the patch in #188 would is merged.I would like to apply
#proclocal
also for the procedure arguments, in such a way that the following example works: