libphorward documentation

I forked the libphorward project. Then I made (a lot) of changes to the documentation files (.t2t). And... that turned out to be a waste of time. Documentation is created using the comments as found in the C source code. And those will be rewritten every time you recreate the t2t files.

I figured it would be better to correct the typos and then issue a pull request. Less work for you (otherwise I'd post an issue with all the typos and you'd have to correct all the files yourself). It seemed like a good idea to correct the typos and then issue a pull request. Of course I should have corrected the typos in the right place (eg not in the .t2t files).

Am I correct in thinking that in order to correct typos in the documentation of libphorward I have to change the comments as found in the C source code? And when I am satisfied with the changes I made I can issue a pull request. Leaving it up to you to decide whether or not to accept the pull request.

I looked at the way parsing (lalr1) is performed in libphorward. And I have something of a feature request.

Would it be possible to have an external scanner call the parser ('push parsing')? The interface to the libphorward generated parser would consist of a single function that should get called by the scanner for every token that the parser should parse.

Having an external scanner call the parser could make it a lot easier to parse input from anything besides a static string (eg stream-ish style parsing). Also the parser could be part of a 'larger' parser that would decide what subparser would get fed a certain token. It would be great to use x subparsers to parse some 'exotic' language (for example C++ :)).

Parsing one token using push parsing could look something like this.

token = get_token(scanner)
parse(parse_state,token)
if (parse_state->error_) then
  ''error message
end if

I am assuming the parser needs a place (parse_state) to keep track of parsing state. The numbering scheme for tokens could be straightforward: first token to appear in the grammar would get assigned a possible user defined number (or 1 or something else). Subsequent tokens appearing in the grammar would get numbered upward (or downward) from that first , possible user defined number.

parse_state could be used to store other, user-defined information (pppar could be extended to contain some user defined data in the shape of an any ptr or a plist or...?).

It would also be nice if the user could specify a routine that gets executed at given points during the parse. With some restrictions (not every position in the grammar might be appropriate).

Specifying where the routine should get called could be done by inserting special symbols in the grammar.

Whenever the parser reaches a point during the parse that corresponds with the position of the symbol the user defined routine gets executed.

An example.

struct_type: typedef struct s_identifier '{' struct_declaration_list'}' init_suffix;
init_suffix : init_declarator_list | ';';
s_identifier : identifier ;#

The user defined routine would need some type of info (context) so it can figure out what to do. A unique name (or number) should be added to every #

struct_type: typedef struct s_identifier '{' struct_declaration_list'}' init_suffix;
init_suffix : init_declarator_list | ';';
s_identifier : identifier ;#s_identifier

It's up to the user to come up with the names or the numbers after the #. The parser will simply call the user defined routine passing the name or number as an argument.

That's it for now. It's a shame I misunderstood the way documentation for libphorward gets generated. Otherwise I could have contributed something to libphorward.

Hello @FreeBASIC-programmer,

I forked the libphorward project. Then I made (a lot) of changes to the documentation files (.t2t). And... that turned out to be a waste of time. Documentation is created using the comments as found in the C source code. And those will be rewritten every time you recreate the t2t files. ... Am I correct in thinking that in order to correct typos in the documentation of libphorward I have to change the comments as found in the C source code? And when I am satisfied with the changes I made I can issue a pull request. Leaving it up to you to decide whether or not to accept the pull request.

Well, first I'm very happy about your help, so you're very welcome in doing any improvements to libphorward, may it documentation, code or ideas.

I'm a little sad that you recognized that parts of the documentation are generated after doing changes. This only relates to the file doc/ref.t2t, which is generated from all .c source files where /** this kind of comment */ is used in front of functions. The magic behind the documentation generator lays in the script run/pdoc.

Anyway, changes you did to the other t2t files, like list.t2t or parse.t2t are done in the right place, these files aren't generated and stay on their own.

I think it's a good idea to put a README.md file into the doc/ directory and tell about the structure and its usage, which files are generated and things which are important.

I'm looking forward to get a pull request from you soon, and I'm really willing to accept this! :smiley:

And I have something of a feature request.

Your ideas and feature requests are very welcome. I hope you agree that I split them into separate issues #7 and #8 to easier distinguish between different issues. Feel free to do so if you have more than one topic you want to discuss or propose :grin:

That's it for now. It's a shame I misunderstood the way documentation for libphorward gets generated. Otherwise I could have contributed something to libphorward.

Yep, and you already did and I also hope you will do so,with a pull request, coming soon!

Thank you & so long,

Jan

There is a bit of problem with pull - request. Let me paint the picture for you using an example. I change text A to text A' in my fork of a project. Then the project owner changes text A to B. I issue a pull request after the project owner has changed the text from A to B.

Assuming A' is compatible with A (going from A to A' poses no problem) and B is not compatible with A (A and B have very little in common). Then a change from B to A' makes no sense. And the pull request gets denied.

The example is not that far fetched. I think the above is what is likely to happen when you issue a pull request. It looks a bit like the kind of thing that happens when a central database gets updated. With the difference that people updating the central database work on one and the same database. And data gets updated directly (or not depending on locking etc...).

But a fork of a github project and the project are two distinct databases. Changes I make are not directly uploaded to the main projects (and rightly so). github does track the differences (it tells me that my branch is 9 commits ahead, 5 commits behind phorward:develop). But all of that tracking does not change a thing to the whole A A' B issue.

Issued pull request regardless of the problems with pull - requests.

Hello @FreeBASIC-programmer,

thank you for your answer and pull request. Yep, the problem you describe is exactly the way it is meant to be when using git: a distributed version control, with independently managed trees and branches. In the case you describe, if there comes up a conflict with two lines changed by both developers, a decision has to be taken manually which change is the correct one to be chosen, or a general refactoring, that correctly merges both changes, has to be done. So it wouldn't be a problem if you make a pull request which cannot directly be merged. Then, a manual intervention is required and senseful, so it is correctly merged into the main branches.

I'm currently a little busy with several things, so that I cannot fully review your pull request right now, but as soon as I have more time, I will both merge your pull request and answer the other issues #7 and #8. Your proposal to implement push parsing gave me many new, great ideas to improve the entire software and make it much more flexible, thanks for this!

So long, and keep up your great contributions! :+1: Jan

Hello @FreeBASIC-programmer, I finally merged your pull-request and added you changes into the appropriate positions in the source code, so ref.t2t now is generated including the changes you did to it.Thanks a lot!

phorward / libphorward

libphorward documentation #6