Building Forth (Book)

[Immortalin]

Hi All,

I had been planning to write a book on building Forth for quite a well but never quite got around to it. That was back in 2016 before I knew Forthhub existed. Since communities like Clojure have produced books like the famous O'Reilly Cookbook series and the Rust Programming Language Book, I am going to open up the repo so all can contribute. I will be writing a fair chunk of it, editing contributions and also finding a publisher for dead tree form. Here's the link, please feel free to bookmark it and I will set up a ebook build system soon.

https://github.com/Immortalin/Building_Forth

[Immortalin]

The main content will cover bootstrapping Forth from a scripting language (Python, Ruby or perhaps Go) and then finally to full Forth bootstrapping. Perhaps a implementation in C too. Along the way concepts such as indirect threading will be covered. War stories, famous usages of Forth and any other Forth related content are also welcomed. For code, I am particularly interested in easy, simple implementations of the does and create macros and also a full Forth x86 assembler. The simpler the better. Remember our audiences are not embedded programmers, this is a book for general programmers.

[Immortalin]

Of particular interest is the relation of Forth to modern stacks, i.e. what role can Forth play in facilitating unikernels and Internet of Things

[Immortalin]

For bare metal Forth OS, QEMU + this can be used to skip the whole modern PC initialization mess.

[Immortalin]

WebAssembly would also be another great target option

[alexshpilkin]

There is also Pure64 to handle initialization, but the initialization is only tedious, given some architectural knowledge, whereas any more or less modern x86-64 assembly is pure madness. (I think I have two half-functional Forth assemblers for x86 where the instruction descriptions turned out to not be expressive enough in the middle of the table; 80186 is easy, though.)

Frankly, I don't think getting more targets up is that difficult, it's the first target that's the problem. For a book, you might want to choose something clean — ARMv4, maybe, or MSP430, or possibly even the JVM. I looked at WebAssembly, and it looks rather convoluted to me, but it might be the sorry state of the documentation.

[massung]

Obviously anything someone would like to do to further Forth or get new people understanding it is great; I commend - and would endeavor to assist with - any project like this where I could.

That said, I personally feel that if there was Forth "book" that needed written, it would be to take Brad Rodriguez's very good, and yet very dated, Moving Forth web pages and update them both in terms of focus (the documents do bounce around a bit) and their targets; while the Z80 and 6502 are still quite popular for emulating and hobbyists, can you even get a Z8 or PDP-11 any more? Certainly teaching with code that targets C or JavaScript would help many newcomers.

His pages are very old, and I'm constantly in a minor state of panic thinking the domain could be lost at any time.

Of course, as I type this, all I can think to myself is, "if you want this, why not just do it instead of hoping someone else does?" ;-)

[scotws]

@massung You'll be happy to hear that the indefatigable Juergen Pintaske has reformatted Rodriguez' texts and published them in e-book format on Amazon (https://www.amazon.de/Moving-Forth-Internals-Processor-English-ebook/dp/B07BTV7QVR/) as part of his Forth Bookshelf.

@alex-shpilkin I agree with you that x86 assembler is, ah, daunting. One possible target, especially for a book such as this, would be the free RISC-V ISA (https://riscv.org/). It seems to be gaining real-world traction with Nvidia and Western Digital getting behind it for some of their projects, and there is now actual silicon out in the world that you can buy (https://www.sifive.com/products/hifive1/). This would allow you to create a bare-metal Forth with a more modern slant.

[scotws]

Sorry, I just remembered that Mecrisp-Forth has already been (experimentally) ported to RISC-V assembler, see https://www.forth-ev.de/article.php/20180306213428204 for the announcement (in German) and https://sourceforge.net/projects/mecrisp/files/ for the actual files.

[pebhidecs]

On 11/04/2018 at 2:00 PM, "Jeffrey Massung" notifications@github.com wrote:

His pages are very old, and I'm constantly in a minor state of panic thinking the domain could be lost at any time.

Panic not. I have a peel of the content in pdf format for safe keeping and personal reference. I would have to ask Brad about letting it go elsewhere though.

Regards

Paul E. Bennett IEng MIET Systems Engineer Lunar Mission One Ambassador --

---

Paul E. Bennett IEng MIET..... Forth based HIDECS Consultancy............. Mob: +44 (0)7811-639972 Tel: +44 (0)1392-426688 Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..

---

[massung]

@scotws Wow, thanks for that link! I'm so happy that is the case. While Kindle is fine, if there was a physical copy I would have clicked "buy" already. ;-)

[Brad-R]

Paul, in principle I'm not averse to that content being archived and redistributed either in the original HTML or in PDF format. But now that Juergen has gone to all the trouble of producing it as a Kindle book, I'd really prefer if people supported his effort and acquired it that way.

I've been talking for a long time about updating that series to use the MSP430 as the example. Maybe it's time I did so.

Cheers, Brad

On Wed, 11 Apr 2018 06:20:17 -0700 "Paul E. Bennett IEng MIET" notifications@github.com wrote:

Panic not. I have a peel of the content in pdf format for safe keeping and personal reference. I would have to ask Brad about letting it go elsewhere though.

-- brad@bradrodriguez.com

[AntonErtl]

On Wed, Apr 11, 2018 at 03:37:25AM +0000, Lin wrote:

WebAssembly would also be another great target option

Only for an interpreter. A student of mine looked into whether Forth->WebAssembly compilation would be possible, and his answer was "no", because WebAssembly does not support generating new WebAssembly code at run-time, and then running it.

• anton

[monsonite]

Brad, List,

The MSP430 is a great little microcontroller, easy to learn ISA, and a good fit to the requirements of Forth - being 16-bit, a good set of registers and a useful, truly orthogonal instruction set.

However, IMHO it's never really been promoted - always the Bridesmaid and never the Bride, struggling to find its niche - and in this age of $1 ARM processors, I suspect that it's missed what opportunity it may have had 15 years.

What interests me is that it is now available with unified non volatile memory as FRAM, and some devices have 24bit ADCs (MSP430I2041). Its core is also available as an FPGA soft core.

I know of at least half a dozen Forths * for this mcu - from this list and elsewhere - so it has found favour in the Forth community.

An update to include the MSP430 - and a different take on building a Forth would be appreciated.

Ken

* Fast Forth Mecrisp Camel Forth eForth 4th 4 Education noForth plus others

On 11 April 2018 at 14:45, Brad-R notifications@github.com wrote:

Paul, in principle I'm not averse to that content being archived and redistributed either in the original HTML or in PDF format. But now that Juergen has gone to all the trouble of producing it as a Kindle book, I'd really prefer if people supported his effort and acquired it that way.

I've been talking for a long time about updating that series to use the MSP430 as the example. Maybe it's time I did so.

Cheers, Brad

On Wed, 11 Apr 2018 06:20:17 -0700 "Paul E. Bennett IEng MIET" notifications@github.com wrote:

Panic not. I have a peel of the content in pdf format for safe keeping and personal reference. I would have to ask Brad about letting it go elsewhere though.

-- brad@bradrodriguez.com

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ForthHub/discussion/issues/68#issuecomment-380457187, or mute the thread https://github.com/notifications/unsubscribe-auth/AAuUPxkbM3UDcU66I9BAc2jofkmPaqnVks5tngj3gaJpZM4TPTb3 .

[AntonErtl]

On Wed, Apr 11, 2018 at 06:45:32AM -0700, Brad-R wrote:

I've been talking for a long time about updating that series to use the MSP430 as the example. Maybe it's time I did so.

For a long-lived update, ARM may be better; from what I heard, TI is not doing new MSP430 microcontrollers, and is using ARM in new microcontrollers.

• anton

[pebhidecs]

On 11/04/2018 at 2:45 PM, "Brad-R" notifications@github.com wrote:

Paul, in principle I'm not averse to that content being archived and redistributed either in the original HTML or in PDF format. But now that Juergen has gone to all the trouble of producing it as a Kindle book, I'd really prefer if people supported his effort and acquired it that way.

I've been talking for a long time about updating that series to use the MSP430 as the example. Maybe it's time I did so.

Cheers, Brad

My main use is as a personal reference and I had no intention of letting it go elsewhere without your consent. As it is also in Juergen's bookshelf, I agree that others should get it from there.

EuroForth 14th Sept 2018 to 17th Sept 2018 at the DoubleTree Hilton Queensferry Hotel, Edinburgh, Scotland. You are welcome if you can get there. More details coming soon.

Regards

Paul E. Bennett IEng MIET Systems Engineer Lunar Mission One Ambassador --

---

Paul E. Bennett IEng MIET..... Forth based HIDECS Consultancy............. Mob: +44 (0)7811-639972 Tel: +44 (0)1392-426688 Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..

---

[Immortalin]

@Brad-R would you be interested in donating content or helping out? Moving Forth would be a great guide and template for an updated book in the same vein

[yunfan]

x86 is boring, but still we all use it. i guess there's arm and mips could be considered too, also the riscv, since the author of qemu has wrote a online emulator for it, its easy to try for us

for arm, there's rpi and many other clones, for mips just check our wifi aps. and for the internet of things, please give esp8266 a chance, i knew someone has already done(punnyforth)

also i am very glad you mentioned unikernel. its a worth to try. finally there's someone focus on cloud instead of his outdated microcontroller

[quozl]

give esp8266 a chance, i knew someone has already done(punnyforth)

CForth too.

[yunfan]

@quozl thanks for completed me. but i prefer asm based implementation, although c implmented one might be more portable(which made me strugle :[ )

[MitchBradley]

I am the author of an ASM-based Forth system (Forthmacs) that has been ported to, at least, 680x0, SPARC, MIPS, x86, PA-RISC, PowerPC, Alpha, SaxPY, ARM32, and ARM64. That system has served as the base of an application (Open Firmware) that shipped on several million computers from several different companies. I'm also the author of CForth, a C-based Forth system that runs on pretty much anything. As such, I know quite a bit about the tradeoffs between ASM and C based Forths. Personally, I rarely use the ASM based system anymore. All the tools that I care about, with the sole exception of the interactive assembly language debugger, run the same on both, sharing most of the same source code. There was a time when an ASM Forth was quite useful to me - when my job involved debugging "first silicon" of new microprocessors. At that time, one often could not trust C compilers to generate correct code, and even when they did, it was helpful to dissect problems down to the assembly language level when you could not trust the silicon. Those days are over. It has been nearly two decades since I have had my hands on first silicon of a new micro. I'd venture to guess that few people on this list have ever had that privilege/responsibility. Furthermore, C compilers are extremely good now, often generating better code than most humans can. At the beginning of my career, there was a time when I had found at least one code generation bug in every compiler I had ever used. Now, I can't remember the last time I found such a bug. The turning point for me came when I needed to convert a hand-code ASM program for AVR into C. That program, which I had written, had a tightly coded inner loop that did bit timing for a pulse-position infrared modulator. It was becoming difficult to maintain because of an ever-increasing feature set. It needed to be rewritten in C for ease of maintenance, but I didn't have time to go through a full testing and qualification cycle for the new version. Realizing that the AVR C compiler generated good code, I was able to write the C code so that the compiler generated exactly the same - byte for byte - machine code as the old ASM version. So we were able to switch over to the new code base with no additional testing. From then on, new mods could be tested as time and schedule permitted. With CForth, I have been able to target new instruction sets essentially instantly. That happened with at least ARM-THUMB (a different instruction encoding than ARM32), x86-64, xtensa (esp8266 and esp32). At this point I have just stopped caring about instruction sets. For a lot of my career they were extremely important; now they are uninteresting.

[quozl]

I agree with @MitchBradley. Unless you have a good reason to code in assembly, it's not worth the bother or the slow down you get by avoiding compiler optimisers. If it is worth it, then at least only in Forth words that are critical to an application, and only if the speed improves. Hand assembly just doesn't always help.

[yunfan]

no i am not prefer asm for showoff, its just for me, using asm could understand much of the details.

also i'd saw some embed project's c code(maybe 8051), which's code organization is really ugly, i think that might be the source of bad impression of embeding c code

also i prefer tiny things, not only the executable , but also the tools, this is also the reason i like forth dont worry, i dont hate cforth, its just if i need to gave my choice for only 1 forth, then i prefer asm implementation

[Immortalin]

Using C in a step kind of defeats the point of Forth - a self contained system that is NOT yet another variation on the Bell Labs computing vision

[Immortalin]

The only other system that has cultural similarity with Forth is Lisp and that's on another totally different abstraction level altogether. Beyond the islands of Forth and Lisp, everything else is just a sea of curly braces. I have nothing against C per se just that the whole point of Forth is that it brings something new to the table, a table that's currently only occupied by C/C++ and perhaps now Rust.

[Immortalin]

Though we could always just bring in LLVM..

[MitchBradley]

C has been described as a "universal assembler". It works reasonably well for that purpose. You can use as few runtime library routines, or as many, as you wish. I have done pretty much every variant, as needs demanded, from bare-metal where my code was running starting at the reset vector, with zero dependence on any code that I didn't write, to fully-hosted systems where Forth sits on top of a huge stack of external OS code, to Forth inside the OS kernel acting as a resident debugger, to tethered systems where Forth runs on a host and takes control of an app on another processor via a tiny stub of a few hundred bytes of code. Your idea of "the purpose of Forth" is obviously quite different from mine. For me, Forth is a way to do whatever I need to do at the moment. Sometimes it's bare metal, sometimes it's a small part of an enormous system.

[Brad-R]

Lin, if I'm going to create content, I think that effort would be better directed to my own project. But do feel free to adapt "Moving Forth" for your needs.

I'm going to stay with the MSP430 as a target system for the update. First, I think it has greater educational value for the topics I'll be exploring. Second, I already have an MSP430 Forth implementation.

Regards, Brad

On Wed, 11 Apr 2018 15:21:34 +0000 (UTC) Lin notifications@github.com wrote:

@Brad-R would you be interested in donating content or helping out? Moving Forth would be a great guide and template for an updated book in the same vein

-- brad@bradrodriguez.com

[alexshpilkin]

@Brad-R MSP430 is great, actually—even a full commented listing of the whole kernel (a la TeXbook appendix B) should be short enough, although that’s not probably not what you’re going for. It also has very real implementations and development tools, unlike the RISC-V.

One request: if the ebook prepared by Pintaske is going to be the definitive version, could it be released in a DRM-free way? I’m fine with paying for it, but don’t like restrictions that prevent proper archiving.

[Immortalin]

@MitchBradley good point! What would be your recommendations for the book?

[Mecrisp]

Dear Brad,

I am a great fan of your old "Moving Forth" articles, and I am perfectly fluent in MSP430 assembly, my favourite instruction set. If you are going to continue or relaunch your series, I am there for proofreading and discussion. If you wish, we can address issues of flash memory handling and how to implement constant folding together.

Matthias (the one behind Mecrisp)

Lin, if I'm going to create content, I think that effort would be better directed to my own project. But do feel free to adapt "Moving Forth" for your needs.

I'm going to stay with the MSP430 as a target system for the update. First, I think it has greater educational value for the topics I'll be exploring. Second, I already have an MSP430 Forth implementation.

Regards, Brad

[MitchBradley]

I think there are enough Forth implementations already. I would like to see more activity around applications.

[MitchBradley]

I think there are enough Forth implementations already. I would like to see more activity around applications.

[Brad-R]

Probably the "definitive" version is the original articles, which are published DRM-free on my personal web page http://www.bradrodriguez.com/papers/. I have not yet seen Juergen's edited version -- I don't have a Kindle -- so I don't know how many changes he made. I think he mainly fixed the formatting so it would read well on Kindle, and removed or updated some dead hyperlinks.

• Brad

On Thu, 12 Apr 2018 18:10:34 +0000 (UTC) Alexander Shpilkin notifications@github.com wrote:

One request: if the ebook prepared by Pintaske is going to be the definitive version, could it be released in a DRM-free way? I’m fine with paying for it, but don’t like restrictions that prevent proper archiving.

-- brad@bradrodriguez.com

[monsonite]

Brad, List,

A very timely email popped up today from Amazon to remind me that Moving Forth is now available for download. Thanks for making this material available and thanks to Juergen for publishing it as part of the "Forth Bookshelf".

I'm pleased to hear that you are revisiting some of the concepts from "Moving Forth" and updating them to use the MSP430 as a target.

I have been using the MSP430FR2433 Launchpad as my target - and this seems to have taken over from the old G2 Launchpad as a low cost dev-board. The '2433 offers 16K of FRAM and 4K of RAM - so it offers sufficient resources to host a fairly comprehensive Forth.

Unfortunately the '2443 is a tiny SMT part, and not exactly breadboard friendly - but I have overcome this by mounting it on a 20 pin DIL module.

2018 is the 50th year of Forth, so maybe with the MSP430 we can have a 50th Anniversary revival.

I have been following the resurrection of Forth on the IBM 1130 machine - seems like a worthy project.

https://rescue1130.blogspot.co.uk/2018/03/original-forth-from-ibm-1130-under.html

Ken

On 13 April 2018 at 21:20, Brad-R notifications@github.com wrote:

Probably the "definitive" version is the original articles, which are published DRM-free on my personal web page http://www.bradrodriguez.com/papers/. I have not yet seen Juergen's edited version -- I don't have a Kindle -- so I don't know how many changes he made. I think he mainly fixed the formatting so it would read well on Kindle, and removed or updated some dead hyperlinks.

• Brad

On Thu, 12 Apr 2018 18:10:34 +0000 (UTC) Alexander Shpilkin notifications@github.com wrote:

One request: if the ebook prepared by Pintaske is going to be the definitive version, could it be released in a DRM-free way? I’m fine with paying for it, but don’t like restrictions that prevent proper archiving.

-- brad@bradrodriguez.com

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ForthHub/discussion/issues/68#issuecomment-381250696, or mute the thread https://github.com/notifications/unsubscribe-auth/AAuUP-XAQ5nSroCaSoOYm2vu4vuQAZhkks5toQingaJpZM4TPTb3 .

[larsbrinkhoff]

There's one more Forth option for the ESP8266:
https://github.com/niclash/forthright

[niclash]

I don't recommend forthright. It was more of an attempt for me to really learn the ESP8266 and pick up my long lost Forth interest again. And I don't have time to maintain it.

What I never managed was to get the dictionary into the codebase, and the standard dict ended up using 12-15k (can't remember exactly) of "precious" RAM...

But feel free to build on it.

Cheers Niclas

On Mon, Apr 16, 2018 at 1:34 PM, Lars Brinkhoff notifications@github.com wrote:

There's one more Forth option for the ESP8266: https://github.com/niclash/forthright

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ForthHub/discussion/issues/68#issuecomment-381484525, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGgxUCCH0v3qggVgSfjq2BtEU1lX_JSks5tpC1dgaJpZM4TPTb3 .

-- Niclas Hedhman, Software Developer http://polygene.apache.org - New Energy for Java

[cwpjr]

Not having a kindle I request a pdf version.

On Wed, Apr 11, 2018 at 8:45 AM, Brad-R notifications@github.com wrote:

Paul, in principle I'm not averse to that content being archived and redistributed either in the original HTML or in PDF format. But now that Juergen has gone to all the trouble of producing it as a Kindle book, I'd really prefer if people supported his effort and acquired it that way.

I've been talking for a long time about updating that series to use the MSP430 as the example. Maybe it's time I did so.

Cheers, Brad

On Wed, 11 Apr 2018 06:20:17 -0700 "Paul E. Bennett IEng MIET" notifications@github.com wrote:

Panic not. I have a peel of the content in pdf format for safe keeping and personal reference. I would have to ask Brad about letting it go elsewhere though.

-- brad@bradrodriguez.com

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ForthHub/discussion/issues/68#issuecomment-380457187, or mute the thread https://github.com/notifications/unsubscribe-auth/AFC6xUqOpFdXzk4ZskAfsns26lRNKw87ks5tngj6gaJpZM4TPTb3 .

[yunfan]

@cwpjr can i have a look of that pdf version?

[pebhidecs]

On 24/04/2018 at 3:28 AM, "Clyde Willis Phillips Jr." notifications@github.com wrote:

Not having a kindle I request a pdf version.

On Wed, Apr 11, 2018 at 8:45 AM, Brad-R notifications@github.com wrote:

Paul, in principle I'm not averse to that content being archived and redistributed either in the original HTML or in PDF format. But now that Juergen has gone to all the trouble of producing it as a Kindle book, I'd really prefer if people supported his effort and acquired it that way.

I've been talking for a long time about updating that series to use the MSP430 as the example. Maybe it's time I did so.

Cheers, Brad

Considering how inexpensive some of these e-readers, like Kindle and small tablets are these days, or how easy it is to download a free e-reader, I am surprised that tech savvy guys do not have the means to read Juergen's offering.

I have had my pdf peel for quite a while, predating the one on Juergen's bookshelf.

Regards

Paul E. Bennett IEng MIET Systems Engineer Lunar Mission One Ambassador --

---

Paul E. Bennett IEng MIET..... Forth based HIDECS Consultancy............. Mob: +44 (0)7811-639972 Tel: +44 (0)1392-426688 Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..

---

[cwpjr]

I haven't been given any new PDF's. vis the thread. I have many of the classics. Cheers~!

On Mon, Apr 23, 2018 at 9:45 PM, yunfan notifications@github.com wrote:

@cwpjr https://github.com/cwpjr can i have a look of that pdf version?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ForthHub/discussion/issues/68#issuecomment-383785611, or mute the thread https://github.com/notifications/unsubscribe-auth/AFC6xW2hd6E9rjC-L2AzJ_1FPnP1AIUkks5trpGygaJpZM4TPTb3 .