WIP, FIXME, TODO

[Lecrapouille]

dev-refacto branch:

API:

• forth::Forth forth; is not possible => forth::Forth simforth; should be simforth::Forth forth;

Issues:

• CRITICAL: Token = Dictionary rooms = 2 bytes. Stack cells: = 4 bytes. ANSI word ! store a cell but ANSI cells are 2 bytes and ANSI C! is not possible here (except in HERE = m_here() / 2). 0BRANCH needs T! because we just need 1 token. How to implement C! because m_here needs to be moved of 0.5 (HERE shall return * 2 but no longer hold uint16_t).
• CRITICAL: How to dump dictionaries when using CLib ?
• MINOR: word INCLUDE: Forth when include the current path is not changed
• MINOR: Dictionary display: We can store data after EXIT (when displaying dico: they are displayed as literals) but words can use EXIT. Not possible to check if its the last word (need memorize the number of tokens inside the definition). Example: ./build/SimForth -f core/SelfTests/Tester.fth -e "42 , 43 ," -d For the moment I use RETURN

KO ANSI-Forth:

• MINOR: T{ : GI5 BEGIN DUP 2 > WHILE DUP 5 < WHILE DUP 1+ REPEAT 123 ELSE 345 THEN ; WORDS -> }T KO
• MINOR: S" 1 2 3 + + ABORT" qqqqq"" EVALUATE 1 . Finding the first " char seems not correct. Also manage \".
• FIND is not in the standard

ANSI-Forth suxx:

• s" 1 One, 2 Two, 3 Three is good." s" \s+" type type the second string smashs the first and type shows the last string. Better idea would be to use C pointer and different space memory.

Travis CI / Unit tests:

• valgrind finds 2 bugs when doing unit tests => ABORT"
• MAJOR: Big and little-endian architecture.
• IMPORTANT: Add clang++
• IMPORTANT: Add OS X
• IMPORTANT: Add Windows
• MINOR: Why CI fails linking libsimforth against a binary only with g++-4.8 while working nicely on a real distrib
• Add more unit tests in tester-core.fth: Example ' . EXECUTE and : FOO . ; ' FOO EXECUTE.

Endian:

• Dump dictionaries: Big endian surely not working.
• Travis CI: launch unit tests on big endian architecture.
• read/write dico: manage endianess (lscpu) in append() and operator[] => replace m_memory[m_here++] by append. Idea: private Cell: public Endian with methods such as swap, set, get(char*). Make dico.append(X) delegate this operation to Endian.

TODO/improvements:

• Remove IForth.hpp ?
• Cell: struct Eq::exec(Real) use ULPS instead of epsilon
• Add a words glossary (doc)
• Replace Result by std::error_code and add interpreter.error() interpeter.result() nameStreamInFault()
• IMPROV: dico display: add a \n after EXIT when displaying literals.
• TODO: https://github.com/uho/forth-notebooks
• TODO: https://github.com/bradn123/literateforth
• TODO: Manage better the creation of ~/.SimForth folder: 1/ Create it before needed by the interactive prompt loads its history. 2/ Create a sub-folder . 3/ Adapt the env path $SIMFORTHPATH to it.
• IMPROV In the standalone: forth.boot() is useless if a dictionary dump is loaded (options -a or -l)
• MINOR FIX: Dico check bounds (especially for multi-tasking).
• MINOR FIX: Implement the column position in the stream (Error reported) always set to 0
• MINOR FIX: Improve comments nesting do not taken into account spaces => '( ' count ')' and '(' but not ' ) ' or ' ( '
• MINOR FIX: RAII Stream avec popStream(): fd pas fermé
• MINOR FIX: Compile SimForth for 32-bits with options Float=float and Int=int32t => Float vs double / Int32 vs Int64: add macros in Utils.hpp for hiding strtof sinf expf ... vs strto sin exp. Idem pour strtoll (std::sin ... looks buggy).

Minimal Forth system:

• VARIABLE VALUE not made for switching between float and integer.
• Use computed gotos.
• Security: Make the dictionary in read-only to avoid crash with code 0 0 ! ... FILL, ERASE, CMOVE
• LOCALS
• OO: http://www.forth.org/fd/FD-V21N1,2.pdf
• Create a special private stack LS for Loop iterators. This will avoid dirty the auxiliary stack
• Missing MAX_INT MIN_INT, NAN, +INF, -INF
• Multi-tasking
• FILE? <word> returns the file path and the line where the word is defined.
• USING? <word> returns files using this word.
• THROW CATCH
• files (open, read, write)
• improvements FETCH, STORE http://www.forth200x.org/memory-2010-06-26.txt

Dictionary:

• Save(). Do not replace by default the file if it already exists.
• Save(). Add parameters begin and end for skipping primitives => Save libraries.
• area displacement: modify relative addresses for BRANCH, 0BRANCH, (does)

CLib improvements:

• structure and struct pointer and function pointers management.
• \C not working with interpretString().
• Accepter le format: C-FUNCTION FOO foo n -- void or C-FUNCTION FOO foo n --
• CLib static next_token is risky (utilisé comme index dans std::vector)
• Manage several CLib
• How to dump dictionaries when using CLib ?
• Options: choose the compiler when compiling CLIB. Add in Makefile -DCC=$(CC) to pass it to core/CLib/Makefile.

Improvements:

• Like Julia word "HELP " to display the help. Example HELP + returns ( n n -- n + n ). Idea: : HELP( ')' WORD DROP >H ; IMMEDIATE : + .HELP( n -- n + n )"
• Autocompletion des noms de fichiers.
• InterpretToken: use C goto-labels like done in reforth or gforth.
• dictionary::Backup missinfg dictionary.m_backup.set = false; in CODE(SEMI_COLON)
• Console/readline/completion add trigger onSpace to colorize the previous word
• Catch the signal Ctrl-C in the mode interactive ?
• SimForth -f "f1.fth" -f "f2.fth" gives ok ok but could be ok f1.fth ok f2.fth
• Variable environement $SIMFORTHPATH for searching the image file and Forth source code files instead of -p
• Compile a Simforth-fast: no stack underflow detections => remove DDEEP(x)

Debugger:

• Plusieurs modes d'affichage: affichage actuel et affichage sans le byte code (= uniquement la traduction des tokens).
• When a word crash could be nice to trace all its calling parents. For example "stack underflow caused by DUP called by FOO called by BAR from file xx.fth"
• The debugger pause before executing the word. Not better to execute it first then pause it ?
• Redo it in Forth ? The prompt for skipping or entering in a definition may be Forth words (but in this case how to deal interpreting commands while the interpreter is in a frozen mode. Maybe create a Forth task ?
• If commands are allowed can we manipulate data stacks (push, pop, swap ...) ?
• Add commands such as breakpoints, run, go back, ...
• Add commands such? to show help on the given word

Benchmark:

• [DONE] Implement a benchmark for SimForth vs SimTaDyn vs pforth. For the moment on Fibonacci: gforth 0.9s pforth 1s. SimForth: 3s. SimTaDyn: 7s.
• Optimize list order of primitive words: count their occurence. cat *.fs from gforth|pforth > all.fs Then: sort -k2 -n res.txt
• Profiler code to search why SimForth is x2 slower than pforth.

Done:

• [DONE] Primitives reforth: regexp, execve, fork
• [DONE] DEFER
• [DONE] MAJOR: to be fixed VALUE is not equivalent of VARIABLE
• [DONE] MAJOR: COMPILE inside a definition : KO. => Failed on https://rosettacode.org/wiki/Tree_traversal#Forth
• [DONE] : FOO 4 + DUP 4 - DUP ; broken display: (TOKEN) 4 + DUP (TOKEN) 4 - DUP the 2nd (TOKEN) when executed maked the word - underlined.
• [DONE] MINOR: Optimize Cell integer operations (for the moment using float computation). TBD: Ne plus utiliser f+ f/ f- mais + / - des qu'il y a un float retourne un float. Si on veut un int on utilise >INT ou CEIL, FLOOR, ROUND
• [WONT DO] Do not show the whole dictionary but the last N words. (SEE word is ok)
• [DONE] Autocompletion of the NOP (last word inside the dictionary: infinite loop).
• [DONE] CI: download LinkAgainstMyLibs and compile Forth example
• [DONE] DOES> ABORT" ROLL MODULES, static String, structure, array
• [DONE] Dico display regressions: (DOES) 1681 EXIT 10 NOP NOP instead of (DOES) 1681 EXIT 10 0 0
• [DONE] regression: literals after EXIT are displayed as tokens
• [DONE] Dico display regressions after (TOKEN) bad underlined tokens: 002f 0004 0069 \0082\ (TOKEN) 4 + \DUP\ with Word + is a primitive instead of underlining 0069 and +.
• [DONE] gcov not working with g++-8 in Travis CI Problem was lcov 1.13 not compatible with gcc >= 8. Install lcov 1.14
• [DONE] Makefile: Create a shared library of SimForth (problem: makefile multi-target). See Makefile-lib.
• [DONE] Makefile: Generate a config.hpp instead of version.h
• [DONE] : FOOBAR [ 123 ] LITERAL ; should store 123 on data-satck
• [DONE] T{ : GDX 123 ; : GDX GDX 234 ; GDX -> 123 234 }T is not working on SimForth because of prefered recursivity. To be fixed because of : / dup 0= if -10 throw endif / ; is not working / is not refering to the older / operator. If we want to refer to defining word we have to use : factorial recursive ... factorial else : factorial ... recurse
• [DONE] Autocompletion: >FL does not complete >FLOAT
• [DONE] Word autocompletion with lower cases
• [DONE] 2432902008176640000 =>warning "too large" + convert it to float.
• [WONTDO] 'r' marche mais pas '\n'. (I could not find a 'fromascii' fonction for control chars~~
• [DONE] Stream cannot parse MSDOS files
• [DONE] Show unprintable char on unknown words
• [DONE] Unknown word '\ VALUE:' because an non visible data is store. We should show non printable char in the word.
• [DONE] Copier/Coller d'une fonction de plusieurs lignes dans la console interactive ne fonctionne pas: "Error EOF reached"
• [DONE] Compilation of unit-tests seems to have more warnings activated than standalone
• [DONE] make check does not make Travis failed
• [DONE] 0 VALUE TOTO => VARIABLE TOTO 0 TOTO @
• [DONE] Append dumped dictionaries.
• [DONE] Implement a debuger.
• [DONE] Debugger: show IP
• [DONE] Proper dico display and debuger definition display: create separate functions
• [DONE] \EOF for ending a streaming
• [DONE] findWord(std::string const& word, Token& xt, bool& immediate). Returned xt is not really the XT (code field) but the CFA (== dictionary index). executeToken(xt) needs the CFA. TICK returns the CFA but should return the CF (1 ' DUP EXECUTE) Rename xt, tokens and give better names.
• [DONE] Dictionary display COMPILE 0BRANCH 02ab instead of COMPILE 0BRANCH >MARK
• [WONTDO] CLib: include ?? Data stack + typedef Cell/Token ?? (For the moment the generated C code is fine).~~
• [WONTDO] Upgrade: Push/pop Stream is error prone. Stack Stream has no RAI: no automatic file descriptor closed
• [DONE] Bug C Lib several 0 parameter + 1 return code: dsp[0] instead of dsp[-1]
• [DONE] Tracer: difficult to know the "depth" of secondary words currently executed. For example error of DUP inside a DO WHILE makes difficil to know which secondary words the DUP comes from.
• [DONE] Float sqrt, ln, log, exp, sin, cos (+ tester si on passe un int)
• [DONE] Recurse
• [DONE] FLITERAL (float literals)
• [DONE] LITERAL (with int > int16_t)
• [DONE] Regression: dictionary display: literals are displayed as unsigned.
• [DONE] Dictionary: display float and manage int and float literal > 65535
• [DONE] Config::expand("makefile") CLib::end() and also boot() /usr/share/SimForth/0.1/data
• [DONE] Conflits between TRACES.ON et WORDS ou -p: broken display
• [DONE] BUG MAJEUR: token des secondary words. TODO ecrire tests unitaires
• [DONE] Dictionary::createEntry() strlen("FOO") + 1. The +1 is error prone
• Replace Glib::Module by my own lib to break the dependence with GTK+
• [DONE] BUG MAJEUR Code field peut ne pas etre aligné correctement
• [DONE] BUG stream EOL
• [DONE] BUG "\ foo" a la fin d'un fichier => Failed opening 'f4.fs'. Reason 'Success'
• [DONE] Colorful logs
• [DONE] Regression avec autocompletion.
• [DONE] C Lib
• [DONE] Smudge some primitives (EXIT ...)
• [DONE] Display dico 4 par 4 mots + display literals in the current base
• [DONE] broken dico display: smudge words, additional columns with literals, show memory after EXIT
• [DONE] Display stack toujours en hexa
• [DONE] Faire trace word (pour le moment verbose)
• [DONE] Mettre les primitives dans le bon ordre (s'inspirer de 4th ?)
• [DONE] : FOO 1 2 et ; manquant => pas d'erreur
• [DONE] Data Stack est toujours affichee en hexa
• [DONE] Better stack underflow with words . DUP OVER. Check data stack underflow and display which words produced the error
• [WONT DO] Clear to 0 stack canaries zone for Stack::reset() ? Because of stack overflow checks
• [DONE] ./build/SimForth -f core/System/Core.fth -i + faire une erreur ca part en boucle infinie [ERROR] from core/System/Core.fth:83:0: Failed opening 'core/System/Core.fth'. Reason 'Success'

[Lecrapouille]

Thinking: Qt QScriptEngine:

QScriptEngine engine = new QScriptEngine(this);
MyObject* obj = ... ;
QScriptValue scriptObj = engine->NewQObject(obj);
QScprtValue global = engine->globalObject();
global.setProperty("obj", scriptObj);

QString script = ;
QScriptValue result = engine->evaluate(script);
if (result.isError())
  ... result.toString()

[Lecrapouille]

Upgrade words primitive managing the input stream:

Forth primitives such as SOURCE, WORD KEY, >IN, TIB, follow outdated specifications and made my Stream API more ugly. Their idea to allow the Forth interpreter to self-evolve by reading the Forth script is greater but their usage is very limited. For example, my Stream API was mainly made of feed(stream), hasWord() and getWord() and private methods such split() as but these words made me add methods such as split(delimiter) (delimiter is a string but only the 1st is used to follow the standard), skip(offset) (negative offset is badly managed)

We can make think a more modern way primitives words.