search

MitchBradley / cforth

Mitch Bradley's CForth implementation
Other
155 stars 41 forks source link

Tried to execute a null token #130

Closed jonsmirl closed 9 months ago

jonsmirl commented 9 months ago

I have cforth embedded in my esp32 app and I've somehow broken extension functions. Can you please give me some hints on what to look for? I've probably done something dumb but I can't see what it is.

I am failing on the first call to build-date

CForth built Tried to execute a null token
: .built  ( -- )  'build-date cscount type  ;

: banner  ( -- )
   cr ." CForth built " .built
   ."  from " .commit
   cr
;

in extend.c

    C(build_date_adr)       //c 'build-date     { -- a.value }
    C(version_adr)          //c 'version        { -- a.value }

my debug doesn't print

cell build_date_adr(void)
{
   ESP_LOGI(TAG, "JDS build addr");
    extern char build_date[];
    return (cell)build_date;
}

tccalls.fth

#21 ccall: 'build-date     { -- a.value }
#22 ccall: 'version        { -- a.value }

I am calling init_forth()

    init_io(0, (char **)0, (cell *)callback_up);   // Perform platform-specific initialization
    callback_up = (void *)init_forth();

dict.h and dicthdr.h are there

-rw-rw-r-- 1 jonsmirl jonsmirl 395136 Feb  8 01:30 dict.h
-rw-rw-r-- 1 jonsmirl jonsmirl     51 Feb  8 01:30 dicthdr.h
# Forth settings
idf_build_set_property(COMPILE_OPTIONS "-DBITS32" APPEND)
idf_build_set_property(COMPILE_OPTIONS "-DFLOATING" APPEND)
idf_build_set_property(COMPILE_OPTIONS "-DMORE_FP" APPEND)
idf_build_set_property(COMPILE_OPTIONS "-DNOSYSCALL" APPEND)
idf_build_set_property(COMPILE_OPTIONS "-DTARGET" APPEND)
idf_build_set_property(COMPILE_OPTIONS "-DMAXDICT=0x4000" APPEND)
quozl commented 9 months ago

If the debug doesn't print, my first guess is that some other source file is being used where you think extend.c is being used. In those situations, I use a link map to find where the symbol was resolved.

MitchBradley commented 9 months ago

Are you trying to execute 'build-date before tccsls.fth is loaded?

jonsmirl commented 9 months ago

Likely, since tccsls.fth is not part of my system.

I will try to add it.

jonsmirl commented 9 months ago

I don't see tccsls.fth anywhere, where is it?

From top of my app.fth

fl misc.fth fl dl.fth fl random.fth fl ilog2.fth fl tek.fth fl stringar.fth

jonsmirl commented 9 months ago

Isn't tccalls.fth getting turned into rodict.c via dict.h and dicthdr.h? My dict.h file is 390KB. Note, I don't have a file system.

I tried to start interactive mode like this: execute_word("app", callback_up);

I am calling prepare_builtin_dictionary(), I checked in the debugger.

is there a tool to dump this as something readable? dict.zip

MitchBradley commented 9 months ago

tccalls.fth is generated by makeccalls via a build rule, then compiled into app.dic

quozl commented 9 months ago

I would remove banner and use the ok prompt to introspect, with see and dump, to find out what the state of the built system is. On the other hand, if you don't have an ok prompt, debugger is the next thing to try.

jonsmirl commented 9 months ago

I replaced the call to "app" with this, and it works.

So something is wrong with my "app" word.

    forth_init();
    //execute_word("app", callback_up);

    char *cmd =
R"FORTH(
: .commit1  ( -- )  'version cscount type  ;

: .built1  ( -- )  'build-date cscount type  ;

: banner1  ( -- )
   cr ." CForth built " .built1
   ."  from " .commit1
   cr
;

banner1

)FORTH";

    spush((int)cmd, getCallback_up());
    spush(strlen(cmd), getCallback_up());
    execute_word("evaluate", getCallback_up());
jonsmirl commented 9 months ago

Is this a valid app.fth to make my 'app' word for esp32? app.zip

quozl commented 9 months ago

Nothing surprises me in your app.fth. It looks just like src/app/esp32/app.fth ... with a few changes.

jonsmirl commented 9 months ago

This works....

    forth_init();
    //execute_word("app", callback_up);

    char *cmd =
R"FORTH(
: .commit1  ( -- )  'version cscount type  ;

: .built1  ( -- )  'build-date cscount type  ;

: banner1  ( -- )
   cr ." CForth built " .built1
   ."  from " .commit1
   cr
;

banner1

)FORTH";

    spush((int)cmd, getCallback_up());
    spush(strlen(cmd), getCallback_up());
    execute_word("evaluate", getCallback_up());

and this gives null token. I don't understand why. "CForth built Tried to execute a null token"

void forth_main()
{
    forth_init();
    //execute_word("app", callback_up);

    char *cmd =
R"FORTH(

banner

)FORTH";

    spush((int)cmd, getCallback_up());
    spush(strlen(cmd), getCallback_up());
    execute_word("evaluate", getCallback_up());

    /* A task should NEVER return */
    vTaskDelete(NULL);
}
quozl commented 9 months ago

Use see to decompile banner and compare with banner1.

jonsmirl commented 9 months ago 
void forth_interact(void)
{
    ESP_LOGI(TAG, "forth_interact enter");
    execute_word("app", callback_up);
    ESP_LOGI(TAG, "forth_interact exit");
}

void forth_main()
{
    forth_init();
    //execute_word("app", callback_up);

    char *cmd =
R"FORTH(

: .commit1  ( -- )  'version cscount type  ;

: .built1  ( -- )  'build-date cscount type  ;

: banner1  ( -- )
   cr ." CForth built " .built1
   ."  from " .commit1
   cr
;

banner1

see banner
see banner1

)FORTH";

    spush((int)cmd, getCallback_up());
    spush(strlen(cmd), getCallback_up());
    execute_word("evaluate", getCallback_up());

    /* A task should NEVER return */
    vTaskDelete(NULL);
}

prints this

CForth built I (699) extend: JDS build addr
2023-03-18 17:31 from I (700) extend: JDS version addr
1a6805a
: banner 
   Tried to execute a null token
: banner1 
   Tried to execute a null token
I (6202) main_task: Returned from app_main()
jonsmirl commented 9 months ago

Maybe I am not including everything I need into rodict.c

./makeccalls <../extend.c >tccalls.fth
./forth forth.dic tccalls.fth app.fth
./makebi app.dic
quozl commented 9 months ago

Using fix-build-warnings branch and PlatformIO, I just built for an Adafruit Feather ESP32, and this is what I get for output;

CForth built 2024-02-09 05:11 from ad1819e
ok see banner
: banner
   ." CForth built " .built ."  from " .commit
   cr
;
ok see .built
: .built
   'build-date cscount type
;
ok see 'build-date
ccall: 'build-date    (Body: $0   ....-a.. )
ok 'build-date 20 dump
           0  1  2  3  4  5  6  7  \/  9  a  b  c  d  e  f  01234567v9abcdef
3f40c2f0  bc 00 00 00 4f 00 00 00  32 30 32 34 2d 30 32 2d  <...O...2024-02-
3f40c300  30 39 20 30 35 3a 31 31  00 61 64 31 38 31 39 65  09 05:11.ad1819e
3f40c310  00 00 00 00 10 45 0d 40  18 45 0d 40 f4 51 0d 40  .....E.@.E.@tQ.@
ok █

So yes, it may be something to do with your build.

jonsmirl commented 9 months ago

It is certainly something to do with my build. I have embedded cforth into a larger ESP32 app as a scripting language. The app uses 'evaluate' to run snippets of forth. What I'm trying to do now is enable interactive mode and failing in my attempts.

quozl commented 9 months ago

Have your app call forth() rather than evaluate?

jonsmirl commented 9 months ago

forth() is doing the same thing:

void forth(void)
{
    init_io(0, (char **)0, (cell *)callback_up);   // Perform platform-specific initialization
    callback_up = (void *)init_forth();
    execute_word("app", callback_up);  // Call the top-level application word
}

My target hardware has more flash to work with now. Tomorrow I will try turning on a file system and running ESP32 cforth the normal way. But first I will have to figure out how to get the cforth git tree off from IDF-3 and onto IDF-5 since my hardware is ESP32S3 and IDF-3 doesn't support the S3.

quozl commented 9 months ago

Oh, that won't work, you've probably got a forth() that calls execute_word on your app. Hmm. Hard to know without seeing your entire build, and that would take a while.

jonsmirl commented 9 months ago

I suspect the problem lies here:

./makeccalls <../extend.c >tccalls.fth
./forth forth.dic tccalls.fth app.fth
./makebi app.dic

The read only dictionary is not including everything it needs, but I don't know how to fix it.

quozl commented 9 months ago

I can't remember if PlatformIO supports S3 yet. If so, branch of our master, use PlatformIO to build for S3?

quozl commented 9 months ago

Can you check that makeccalls, forth, and makebi run without error? I've sometimes made mistakes in app.fth in a way that prevents app.dic from being written. Check the file dates and times?

quozl commented 9 months ago

Does tccalls.fth contain what you expect?

quozl commented 9 months ago

Also consider a way to test forth.dic ... if it has a problem injected by earlier build, app.dic will almost certainly inherit it.

jonsmirl commented 9 months ago

tccalls.fth looks fine. I am not getting any errors on generation. That is the LVGL API, the scripts draw screens.

#0 ccall: lv_scr_act        { -- i.screen }
#1 ccall: lv_scr_load       { i.screen -- }
#2 ccall: lv_obj_create { i.parent -- i.object }
#3 ccall: lv_obj_set_pos    { i.y i.x i.object -- }
#4 ccall: lv_obj_set_size   { i.h i.w i.object -- }
#5 ccall: lv_obj_set_width { i.w i.object -- }
#6 ccall: lv_obj_set_user_data { i.user_data i.object -- }
#7 ccall: lv_obj_align  { i.y i.x i.align i.object -- }
#8 ccall: lv_obj_center { i.object -- }
#9 ccall: lv_obj_add_style { i.selector i.style i.object -- }
#10 ccall: lv_obj_add_event_cb { i.user_data i.filter i.event_cb i.object -- i.desc }
#11 ccall: lv_btn_create    { i.parent -- i.button }
#12 ccall: lv_label_create  { i.parent -- i.label }
#13 ccall: lv_label_set_text { $text i.label -- }
#14 ccall: lv_label_set_long_mode { i.mode i.label -- }
#15 ccall: get_style        { i.index i.page -- i.style }
#16 ccall: lv_style_init    { i.style -- }
#17 ccall: lv_style_set_bg_opa { i.opa i.style -- }
#18 ccall: lv_style_set_bg_color { i.color i.style -- }
#19 ccall: lv_style_set_border_width { i.width i.style -- }
#20 ccall: lv_style_set_border_color { i.color i.style -- }
#21 ccall: 'build-date     { -- a.value }
#22 ccall: 'version        { -- a.value }
#23 ccall: ms              { i.ms -- }
#24 ccall: get-msecs       { -- i.ms }
#25 ccall: restart         { -- }
#26 ccall: i2c-open  { i.scl i.sda -- i.error? }
#27 ccall: i2c-close  { -- }
#28 ccall: i2c-write-read { a.wbuf i.wsize a.rbuf i.rsize i.slave i.stop -- i.err? }
#29 ccall: i2c-b@     { i.reg i.slave i.stop -- i.b }
#30 ccall: i2c-b!     { i.value i.reg i.slave -- i.error? }
#31 ccall: i2c-be-w@  { i.reg i.slave i.stop -- i.w }
#32 ccall: i2c-le-w@  { i.reg i.slave i.stop -- i.w }
#33 ccall: i2c-be-w!  { i.value i.reg i.slave -- i.error? }
#34 ccall: i2c-le-w!  { i.value i.reg i.slave -- i.error? }
#35 ccall: gpio-pin@  { i.gpio# -- i.flag }
#36 ccall: gpio-pin!  { i.level i.gpio# -- }
#37 ccall: gpio-toggle { i.gpio# -- }
#38 ccall: gpio-is-output { i.gpio# -- }
#39 ccall: gpio-is-output-open-drain { i.gpio# -- }
#40 ccall: gpio-is-input { i.gpio# -- }
#41 ccall: gpio-is-input-pullup { i.gpio# -- }
#42 ccall: gpio-is-input-pulldown { i.gpio# -- }
#43 ccall: gpio-mode { i.pullup? i.direction i.gpio# -- }
#44 ccall: wifi-mode@ { -- i.mode }
#45 ccall: log-level! { i.level $component -- }
#46 ccall: socket         { i.proto i.type i.family -- i.handle }
#47 ccall: bind           { i.len a.addr i.handle -- i.error }
#48 ccall: setsockopt     { i.len a.addr i.optname i.level i.handle -- i.error }
#49 ccall: getsockopt     { i.len a.addr i.optname i.level i.handle -- i.error }
#50 ccall: connect        { i.len a.adr i.handle -- i.error }
#51 ccall: stream-connect { i.timeout $.portname $.hostname -- i.handle }
#52 ccall: lwip-write     { a.buf i.size i.handle -- i.count }
#53 ccall: lwip-read      { a.buf i.size i.handle -- i.count }
#54 ccall: lwip-close     { i.handle -- }
#55 ccall: lwip-listen    { i.backlog i.handle -- i.handle }
#56 ccall: lwip-accept    { a.addrlen a.addr i.handle -- i.error }
#57 ccall: start-server   { i.port -- i.error }
#58 ccall: lwip-select    { i.sec a.exc a.wr a.rd i.n -- i.cnt }
#59 ccall: open-dir       { -- a.dir }
#60 ccall: close-dir      { a.dir -- }
#61 ccall: next-file      { a.dir -- a.dirent }
#62 ccall: file-bytes     { a.dir -- i.size }
#63 ccall: file-name      { a.dir -- a.name }
#64 ccall: rename-file    { $.old $.new -- }
#65 ccall: delete-file    { $.name -- }
#66 ccall: fs-avail       { -- i.bytes }
#67 ccall: m-emit         { i.char -- }
#68 ccall: errno          { -- i.errno }
#69 ccall: strerror       { i.errno -- $.msg }
#70 ccall: gpio-matrix-out { i.inven i.invout i.fun i.pin -- }
#71 ccall: gpio-matrix-in  { i.invert i.fun i.pin -- }
#72 ccall: mcpwm_gpio_init  { i.gpio# i.io_signal i.pwm# -- e.err? }
#73 ccall: mcpwm_init  { a.conf i.timer# i.pwm# -- e.err? }
#74 ccall: mcpwm_set_frequency  { i.freq i.timer# i.pwm# -- e.err? }
#75 ccall: mcpwm_set_duty_in_us  { i.duty i.op# i.timer# i.pwm# -- e.err? }
#76 ccall: mcpwm_set_duty_type { i.duty# i.op# i.timer# i.pwm# -- i.err? }
#77 ccall: mcpwm_get_frequency { i.timer# i.pwm# -- i.freq }
#78 ccall: mcpwm_set_signal_high { i.op# i.timer# i.pwm# -- i.err? }
#79 ccall: mcpwm_set_signal_low { i.op# i.timer# i.pwm# -- i.err? }
#80 ccall: mcpwm_start { i.timer# i.pwm# -- i.err? }
#81 ccall: mcpwm_stop { i.timer# i.pwm# -- i.err? }
#82 ccall: set-alarm   { i.xt i.ms -- }
#83 ccall: repeat-alarm   { i.xt i.ms -- }
#84 ccall: set-alarm-us   { i.xt i.us -- }
#85 ccall: repeat-alarm-us   { i.xt i.us -- }
#86 ccall: us { i.us -- }
quozl commented 9 months ago

I continue to speculate. Have you added or removed any ccall? An inconsistent ccall table means the wrong function is called for the number passed.

jonsmirl commented 9 months ago

I am not sure where forth.dic came from. It is over a year old in my git.

the tccalls are right, this works and prints what is expected. The problem is more likely in something like forth.dic.

    forth_init();
    //execute_word("app", callback_up);

    char *cmd =
R"FORTH(
: .commit1  ( -- )  'version cscount type  ;

: .built1  ( -- )  'build-date cscount type  ;

: banner1  ( -- )
   cr ." CForth built " .built1
   ."  from " .commit1
   cr
;

banner1

)FORTH";

    spush((int)cmd, getCallback_up());
    spush(strlen(cmd), getCallback_up());
    execute_word("evaluate", getCallback_up());
jonsmirl commented 9 months ago

I took forth.dic out of the current cforth tree and it made no difference.

quozl commented 9 months ago

Guess still have to find out what is different at execution time between banner1 and banner, especially .built1 and .built. And see also is broken.

quozl commented 9 months ago

I took forth.dic out of the current cforth tree and it made no difference.

Yes, that also happened to me just now. My build was happening fine but there was no app.dic nor forth.dic file. Underneath, my system had a gcc upgrade, but the 32-bit package for libgcc was not installed. The build will use dict.h if app.dic is missing. I would have found this faster had I begun with a git clean or git clone rather than re-use my filesystem source tree.

jonsmirl commented 9 months ago

The interpreter is going off into the weeds, I am adding printf to track down why.

forth_init();
execute_word("banner", callback_up);
I (690) rodict: romvars 0x3c0668a5
I (690) rodict: sizeof(romvars) 2b4
I (690) rodict: ramvars 0x3fc93ef8
I (691) rodict: ramdict 0x3c083a68
I (695) rodict: sizeof(token_t) 4
I (699) interpreter: Token 14398
I (703) interpreter: scr 217
I (707) interpreter: Token 1395
I (711) interpreter: scr 222
I (714) interpreter: Token 86

I (718) interpreter: Token 2176
I (722) interpreter: scr 217
I (725) interpreter: Token 2164
I (729) interpreter: scr 217
I (733) interpreter: Token 22
I (736) interpreter: Token 28
I (740) interpreter: Token 2143
I (744) interpreter: scr 217
I (748) interpreter: Token 70
I (751) interpreter: Token 1455
I (755) interpreter: scr 217
I (759) interpreter: Token 13
I (762) interpreter: Token 13
I (766) interpreter: Token 79
I (769) interpreter: Token 5
I (773) interpreter: Token 37
I (777) interpreter: Token 2093
I (780) interpreter: scr 217
I (784) interpreter: Token 2088
I (788) interpreter: scr 218
I (791) interpreter: Token 133
I (795) interpreter: Token 6
I (799) interpreter: Token 5
I (802) interpreter: Token 2088
I (806) interpreter: scr 218
I (810) interpreter: Token 47
I (813) interpreter: Token 2
I (817) interpreter: Token 79
I (821) interpreter: Token 17
I (824) interpreter: Token 79
I (828) interpreter: Token 16
I (831) interpreter: Token 27
I (835) interpreter: Token 16
I (839) interpreter: Token 21
I (842) interpreter: Token 79
I (846) interpreter: Token 1400
I (850) interpreter: scr 222
I (853) interpreter: Token 3216
I (857) interpreter: scr 217
I (861) interpreter: Token 3195
I (865) interpreter: scr 217
I (868) interpreter: Token 13
I (872) interpreter: Token 5
I (875) interpreter: Token 12
I (879) interpreter: Token 79
I (883) interpreter: Token 139
I (886) interpreter: Token 73
I (890) interpreter: Token 49
I (894) interpreter: Token 3185
I (898) interpreter: scr 217
I (901) interpreter: Token 133
I (905) interpreter: Token 1277
I (909) interpreter: scr 221
I (912) interpreter: Token 65
I (916) interpreter: Token 1391
I (920) interpreter: scr 222
I (923) interpreter: Token 85
CI (927) interpreter: Token 79
I (931) interpreter: Token 141
I (934) interpreter: Token 73
I (938) interpreter: Token 49
I (942) interpreter: Token 3185
I (945) interpreter: scr 217
I (949) interpreter: Token 133
I (953) interpreter: Token 1277
I (957) interpreter: scr 221
I (960) interpreter: Token 65
I (964) interpreter: Token 1391
I (968) interpreter: scr 222
I (971) interpreter: Token 85
FI (975) interpreter: Token 79
I (978) interpreter: Token 141
I (982) interpreter: Token 73
I (986) interpreter: Token 49
I (990) interpreter: Token 3185
I (993) interpreter: scr 217
I (997) interpreter: Token 133
I (1001) interpreter: Token 1277
I (1005) interpreter: scr 221
I (1008) interpreter: Token 65
I (1012) interpreter: Token 1391
I (1016) interpreter: scr 222
I (1019) interpreter: Token 85
oI (1023) interpreter: Token 79
I (1027) interpreter: Token 141
I (1031) interpreter: Token 73
I (1035) interpreter: Token 49
I (1038) interpreter: Token 3185
I (1042) interpreter: scr 217
I (1046) interpreter: Token 133
I (1050) interpreter: Token 1277
I (1054) interpreter: scr 221
I (1057) interpreter: Token 65
I (1061) interpreter: Token 1391
I (1065) interpreter: scr 222
I (1069) interpreter: Token 85
rI (1072) interpreter: Token 79
I (1076) interpreter: Token 141
I (1080) interpreter: Token 73
I (1084) interpreter: Token 49
I (1087) interpreter: Token 3185
I (1091) interpreter: scr 217
I (1095) interpreter: Token 133
I (1099) interpreter: Token 1277
I (1103) interpreter: scr 221
I (1106) interpreter: Token 65
I (1110) interpreter: Token 1391
I (1114) interpreter: scr 222
I (1118) interpreter: Token 85
tI (1121) interpreter: Token 79
I (1125) interpreter: Token 141
I (1129) interpreter: Token 73
I (1133) interpreter: Token 49
I (1136) interpreter: Token 3185
I (1140) interpreter: scr 217
I (1144) interpreter: Token 133
I (1148) interpreter: Token 1277
I (1152) interpreter: scr 221
I (1155) interpreter: Token 65
I (1159) interpreter: Token 1391
I (1163) interpreter: scr 222
I (1167) interpreter: Token 85
hI (1170) interpreter: Token 79
I (1174) interpreter: Token 141
I (1178) interpreter: Token 73
I (1182) interpreter: Token 49
I (1185) interpreter: Token 3185
I (1189) interpreter: scr 217
I (1193) interpreter: Token 133
I (1197) interpreter: Token 1277
I (1201) interpreter: scr 221
I (1204) interpreter: Token 65
I (1208) interpreter: Token 1391
I (1212) interpreter: scr 222
I (1216) interpreter: Token 85
 I (1219) interpreter: Token 79
I (1223) interpreter: Token 141
I (1227) interpreter: Token 73
I (1231) interpreter: Token 49
I (1234) interpreter: Token 3185
I (1238) interpreter: scr 217
I (1242) interpreter: Token 133
I (1246) interpreter: Token 1277
I (1250) interpreter: scr 221
I (1253) interpreter: Token 65
I (1257) interpreter: Token 1391
I (1261) interpreter: scr 222
I (1265) interpreter: Token 85
bI (1268) interpreter: Token 79
I (1272) interpreter: Token 141
I (1276) interpreter: Token 73
I (1280) interpreter: Token 49
I (1284) interpreter: Token 3185
I (1287) interpreter: scr 217
I (1291) interpreter: Token 133
I (1295) interpreter: Token 1277
I (1299) interpreter: scr 221
I (1302) interpreter: Token 65
I (1306) interpreter: Token 1391
I (1310) interpreter: scr 222
I (1314) interpreter: Token 85
uI (1318) interpreter: Token 79
I (1321) interpreter: Token 141
I (1325) interpreter: Token 73
I (1329) interpreter: Token 49
I (1333) interpreter: Token 3185
I (1336) interpreter: scr 217
I (1340) interpreter: Token 133
I (1344) interpreter: Token 1277
I (1348) interpreter: scr 221
I (1351) interpreter: Token 65
I (1355) interpreter: Token 1391
I (1359) interpreter: scr 222
I (1363) interpreter: Token 85
iI (1367) interpreter: Token 79
I (1370) interpreter: Token 141
I (1374) interpreter: Token 73
I (1378) interpreter: Token 49
I (1382) interpreter: Token 3185
I (1386) interpreter: scr 217
I (1389) interpreter: Token 133
I (1393) interpreter: Token 1277
I (1397) interpreter: scr 221
I (1401) interpreter: Token 65
I (1404) interpreter: Token 1391
I (1408) interpreter: scr 222
I (1412) interpreter: Token 85
lI (1416) interpreter: Token 79
I (1419) interpreter: Token 141
I (1423) interpreter: Token 73
I (1427) interpreter: Token 49
I (1431) interpreter: Token 3185
I (1435) interpreter: scr 217
I (1438) interpreter: Token 133
I (1442) interpreter: Token 1277
I (1446) interpreter: scr 221
I (1450) interpreter: Token 65
I (1453) interpreter: Token 1391
I (1457) interpreter: scr 222
I (1461) interpreter: Token 85
tI (1465) interpreter: Token 79
I (1468) interpreter: Token 141
I (1472) interpreter: Token 73
I (1476) interpreter: Token 49
I (1480) interpreter: Token 3185
I (1484) interpreter: scr 217
I (1487) interpreter: Token 133
I (1491) interpreter: Token 1277
I (1495) interpreter: scr 221
I (1499) interpreter: Token 65
I (1502) interpreter: Token 1391
I (1506) interpreter: scr 222
I (1510) interpreter: Token 85
 I (1514) interpreter: Token 79
I (1517) interpreter: Token 141
I (1521) interpreter: Token 79
I (1525) interpreter: Token 79
I (1529) interpreter: Token 3683840
ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x8 (TG1WDT_SYS_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x420035cb
0x420035cb: panic_handler at /home/jonsmirl/esp/esp-idf/components/esp_system/port/panic_handler.c:145 (discriminator 3)
jonsmirl commented 9 months ago

This is the working banner1 trace:

tI (4300) interpreter: Token 79
I (4304) interpreter: Token 141
I (4308) interpreter: Token 73
I (4312) interpreter: Token 49
I (4315) interpreter: Token 3185
I (4319) interpreter: scr 217
I (4323) interpreter: Token 133
I (4327) interpreter: Token 1277
I (4331) interpreter: scr 221
I (4334) interpreter: Token 65
I (4338) interpreter: Token 1391
I (4342) interpreter: scr 222
I (4345) interpreter: Token 85
 I (4349) interpreter: Token 79
I (4353) interpreter: Token 141
I (4357) interpreter: Token 79
I (4361) interpreter: Token 79
I (4364) interpreter: Token 14390
I (4368) interpreter: scr 217
I (4372) interpreter: Token 13174

This is the failing banner trace

tI (4300) interpreter: Token 79
I (4304) interpreter: Token 141
I (4308) interpreter: Token 73
I (4312) interpreter: Token 49
I (4315) interpreter: Token 3185
I (4319) interpreter: scr 217
I (4323) interpreter: Token 133
I (4327) interpreter: Token 1277
I (4331) interpreter: scr 221
I (4334) interpreter: Token 65
I (4338) interpreter: Token 1391
I (4342) interpreter: scr 222
I (4345) interpreter: Token 85
 I (4349) interpreter: Token 79
I (4353) interpreter: Token 141
I (4357) interpreter: Token 79
I (4361) interpreter: Token 79
I (4364) interpreter: Token 3683840

From the first trace the token is 14390 which is 0x3836 From the second trace the token is 3683840 which is 0x383600

So an extra bye of zero has somehow gotten into my romdict.

quozl commented 9 months ago

I agree. Following written before I saw your last, showing how I validated what you said.

Check your prims.h for the token values. On my builds, 85 is EMIT, 79 is UNNEST and 141 is PAREN_LOOP. It's probably the same. So after it prints the "CForth built " string, returning to caller, the next token is invalid. That would be 'build-date, I guess.

I suggest using the host forth against the target's dictionary, and examine the state of the dictionary. I'm using different build tools, but it should be relevant;

$ .pio/build/host_forth/program .pio/build/target_makebi/target.dic 
C Forth  Copyright (c) 2008 FirmWorks
ok see .built
: .built 
   'build-date cscount type 
;
ok see 'build-date
ccall: 'build-date    (Body: $0   ....-a.. ) 
ok words
... 'version      'build-date   acall:        ...
ok ' 'version .x
f7e6e6c0 
ok ' 'build-date .x
f7e6e6a4 
ok ' acall: .x 
f7e6e670 
ok $f7e6e6c0 $f7e6e6a4 - .x
1c 
ok ' 'build-date $1c dump
           0  1  2  3 \/  5  6  7   8  9  a  b  c  d  e  f  0123v56789abcdef
f7e6e6a0  98 25 00 00 8f 25 00 00  00 00 00 00 2d 61 00 00  .%...%......-a..
f7e6e6b0  00 00 00 27 76 65 72 73  69 6f 6e 08 a5 25 00 00  ...'version.%%..
ok

In there should be the ccall number. Verify it. Then next step is to figure out why the number can't be dereferenced and the function executed.

jonsmirl commented 9 months ago 
jonsmirl@ares:~/l5/components/forth/regen$ ./forth app.dic
C Forth  Copyright (c) 2008 FirmWorks
ok see .built
: .built 
   'build-date cscount type 
;
ok see 'build-date
ccall: 'build-date    (Body: $15   ....-a.. ) 
ok ' 'version .x
f7babe04 
ok ' 'build-date .x
f7babde8 
ok ' acall: .x
f7ba99c0 
ok $f7babe04 $f7babde8 - .x
1c 
ok ' 'build-date $1c dump
           0  1  2  3  4  5  6  7  \/  9  a  b  c  d  e  f  01234567v9abcdef
f7babde0  61 74 65 0b 6f 33 00 00  63 2a 00 00 15 00 00 00  ate.o3..c*......
f7babdf0  2d 61 00 00 00 00 00 27  76 65 72 73 69 6f 6e 08  -a.....'version.
f7babe00  76 33 00 00 63 2a 00 00  16 00 00 00 2d 61 00 00  v3..c*......-a..
ok
quozl commented 9 months ago

Sorry, I've no idea. It's just like mine.

jonsmirl commented 9 months ago

is the instruction pointer messed up?

working...

I (4907) interpreter: scr 221
I (4911) interpreter: Token 65
I (4915) interpreter: Token 1391
I (4919) interpreter: scr 222
I (4922) interpreter: Token 85
 I (4926) interpreter: Token 79
I (4930) interpreter: UNNEST 0x3c059fa5
8d f0 4f 00 61 90 d9 b2 71 4f   --> bytes at *rp
I (4937) interpreter: Token 141
I (4941) interpreter: Token 79
I (4945) interpreter: UNNEST 0x3c058f55
4f 00 80 d9 83 22 69 69 4f 00 
I (4952) interpreter: Token 79
I (4956) interpreter: UNNEST 0x3c083a90
36 80 06 6f 2e 73 4f 62 65 0c 
I (4963) interpreter: Token 14390

failing

 I (4926) interpreter: Token 79
I (4930) interpreter: UNNEST 0x3c059fa5
8d f0 4f 00 61 90 d9 b2 71 4f 
I (4937) interpreter: Token 141
I (4941) interpreter: Token 79
I (4945) interpreter: UNNEST 0x3c058f55
4f 00 80 d9 83 22 69 69 4f 00 
I (4952) interpreter: Token 79
I (4956) interpreter: UNNEST 0x3c064e5c
00 00 00 72 00 00 00 00 2d 85 
I (4963) interpreter: Token 3683840

/$p unnest / case UNNEST: ESP_LOGI(TAG, "UNNEST %p", rp); for (int i = 0; i < 10; i++) printf("%02x ", (unsigned char )(rp + i)); printf("\n");

jonsmirl commented 9 months ago

My dicts are here: I (690) rodict: romvars 0x3c06695d I (690) rodict: sizeof(romvars) 2b4 I (690) rodict: ramvars 0x3fc93ef8 I (691) rodict: ramdict 0x3c083a68 I (695) rodict: sizeof(token_t) 4

This does not appear to be pointing into a dict.... I (4956) interpreter: UNNEST 0x3c064e5c

quozl commented 9 months ago

I agree. Something broke the Forth return stack?

jonsmirl commented 9 months ago

working

 I (5289) interpreter: Token 79
I (5293) interpreter: UNNEST 0x3c059fa9
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059fa9 0xffffffff 0x3c083a8e 0x3c059f99 0x3c058f59 
I (5305) interpreter: Token 141
I (5309) interpreter: Token 79
I (5312) interpreter: UNNEST 0x3c058f59
4f 00 80 d9 83 22 69 69 4f 00 
0x3c058f59 0x3c083a90 0x3c0586a5 0x3c05874d 0x3c0588cd 
I (5325) interpreter: Token 79
I (5328) interpreter: UNNEST 0x3c083a90
36 80 06 6f 2e 73 4f 62 65 0c 
0x3c083a90 0x3c0586a5 0x3c05874d 0x3c0588cd 0xf7deaff0 
I (5341) interpreter: Token 14390

broken

 I (5289) interpreter: Token 79
I (5293) interpreter: UNNEST 0x3c059fa9
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059fa9 0xffffffff 0x3c064e5f 0x3c059f99 0x3c058f59 
I (5305) interpreter: Token 141
I (5309) interpreter: Token 79
I (5312) interpreter: UNNEST 0x3c058f59
4f 00 80 d9 83 22 69 69 4f 00 
0x3c058f59 0x3c064e60 0x3c0586a5 0x3c05874d 0x3c0588cd 
I (5325) interpreter: Token 79
I (5328) interpreter: UNNEST 0x3c064e60
00 00 00 72 00 00 00 00 2d 85 
0x3c064e60 0x3c0586a5 0x3c05874d 0x3c0588cd 0xf7db4ff0 
I (5341) interpreter: Token 3683840
ESP-ROM:esp32s3-20210327

I build the dic like this; ./makeccalls <../extend.c >tccalls.fth ./forth forth.dic tccalls.fth app.fth ./makebi app.dic

The address of 'build-date is not the same between banner and banner1. It is correct in banner1, and wrong in banner.

So when app.dic was made, how did it know the address of 'build-date?

jonsmirl commented 9 months ago

My ccall numbers:

21 ccall: 'build-date { -- a.value }

22 ccall: 'version { -- a.value }

Up above when we did the see ok see 'build-date ccall: 'build-date (Body: $15 ....-a.. )

So they match which is why banner1 works.

So why is banner messed up? it is like it is using the wrong ccall numbers. If banner is calling the wrong external routine, my routines could easily leave an extra address on the stack.

quozl commented 9 months ago

So when app.dic was made, how did it know the address of 'build-date?

tccalls.fth creates the dictionary entries, which contain the number. The extern ccalls[] contains the address.

quozl commented 9 months ago

A link map can confirm the object file and therefore the source file for ccalls.

MitchBradley commented 9 months ago

Be careful not to put junk in your ccalls table that you think the compiler will ignore. makeccalls.c might not ignore it the same way as the compiler does. There is a step to run it through the preprocessor in order to sanitize it, but that is not foolproof. When in doubt, make the ccalls table "clean", without ifdefs or commented-out sections.

MitchBradley commented 9 months ago

In particular, commenting out sections of the ccalls table with #ifdef or #if will probably work, but commenting it out with / .. / probably will not work.

jonsmirl commented 9 months ago

I don't think this has anything to do with the external functions, instead it appears to be a problem with the way my app.dic is built. I ran three tests, and in the last one it still fails and it is using no external functions.

One thing I did, I did not build forth.dic myself. Instead I used the cforth tree esp32 build to make it, and then I copied it into my tree.

To me this seems to be implying some sort of issue with rodict.c, or a problem with 'makebi'?

top of my app.fth

\ Load file for application-specific Forth extensions

fl misc.fth
fl dl.fth
fl random.fth
fl ilog2.fth
fl tek.fth
fl stringar.fth

warning @ warning off
: bye standalone?  if  restart  then  bye  ;
warning !

: .commit  ( -- )  'version cscount type  ;

: .built  ( -- )  'build-date cscount type  ;

: banner  ( -- )
   cr ." CForth built " .built
   ."  from " .commit
   cr
;

which was compiled like this

./makeccalls <../extend.c >tccalls.fth
./forth forth.dic tccalls.fth app.fth
./makebi app.dic

This is of trace of execute_word(".built", callback_up); which works, note that at 752 it calls opcode 176 CCALL

I (690) rodict: romvars 0x3c0668fd
I (690) rodict: sizeof(romvars) 2b4
I (690) rodict: ramvars 0x3fc93ef8
I (691) rodict: ramdict 0x3c083a68
I (695) rodict: sizeof(token_t) 4
I (699) interpreter: Token 14390
I (703) interpreter: scr 217
I (707) interpreter: Token 13174
I (711) interpreter: scr 10851
I (714) interpreter: Token 10
I (718) interpreter: Token 2585
I (722) interpreter: scr 217
I (725) interpreter: Token 72
I (729) interpreter: Token 79
I (733) interpreter: UNNEST 0x3c06167d
0c 30 b0 4f 00 6c 60 d9 54 8f 
0x3c06167d 0x3c064dc9 0x3fc93b1c 0xfefefefe 0xfefefefe 
I (745) interpreter: Token 12
I (749) interpreter: Token 48
I (752) interpreter: Token 176
I (756) interpreter: doccall -a
I (760) extend: JDS build addr
I (763) interpreter: Token 79
I (767) interpreter: UNNEST 0x3c064dc9
be 78 4f 00 6e 36 d9 73 80 0d 
0x3c064dc9 0x3fc93b1c 0xfefefefe 0xfefefefe 0xaaaab432 
I (779) interpreter: Token 10686
I (783) interpreter: scr 217
I (787) interpreter: Token 10
I (790) interpreter: Token 10
I (794) interpreter: Token 49
I (798) interpreter: Token 78
I (801) interpreter: Token 37
I (805) interpreter: Token 77
I (809) interpreter: Token 10
I (812) interpreter: Token 49
I (816) interpreter: Token 78
I (820) interpreter: Token 37
I (823) interpreter: Token 77
I (827) interpreter: Token 10
I (830) interpreter: Token 49
I (834) interpreter: Token 78
I (838) interpreter: Token 37
I (841) interpreter: Token 77
I (845) interpreter: Token 10
I (849) interpreter: Token 49
I (852) interpreter: Token 78
I (856) interpreter: Token 37
I (860) interpreter: Token 77
I (863) interpreter: Token 10
I (867) interpreter: Token 49
I (871) interpreter: Token 78
I (874) interpreter: Token 37
I (878) interpreter: Token 77
I (881) interpreter: Token 10
I (885) interpreter: Token 49
I (889) interpreter: Token 78
I (892) interpreter: Token 37
I (896) interpreter: Token 77
I (900) interpreter: Token 10
I (903) interpreter: Token 49
I (907) interpreter: Token 78
I (911) interpreter: Token 37
I (914) interpreter: Token 77
I (918) interpreter: Token 10
I (922) interpreter: Token 49
I (925) interpreter: Token 78
I (929) interpreter: Token 37
I (933) interpreter: Token 77
I (936) interpreter: Token 10
I (940) interpreter: Token 49
I (943) interpreter: Token 78
I (947) interpreter: Token 37
I (951) interpreter: Token 77
I (954) interpreter: Token 10
I (958) interpreter: Token 49
I (962) interpreter: Token 78
I (965) interpreter: Token 37
I (969) interpreter: Token 77
I (973) interpreter: Token 10
I (976) interpreter: Token 49
I (980) interpreter: Token 78
I (984) interpreter: Token 37
I (987) interpreter: Token 77
I (991) interpreter: Token 10
I (995) interpreter: Token 49
I (998) interpreter: Token 78
I (1002) interpreter: Token 37
I (1006) interpreter: Token 77
I (1009) interpreter: Token 10
I (1013) interpreter: Token 49
I (1017) interpreter: Token 78
I (1020) interpreter: Token 37
I (1024) interpreter: Token 77
I (1028) interpreter: Token 10
I (1032) interpreter: Token 49
I (1035) interpreter: Token 78
I (1039) interpreter: Token 37
I (1043) interpreter: Token 77
I (1047) interpreter: Token 10
I (1050) interpreter: Token 49
I (1054) interpreter: Token 78
I (1058) interpreter: Token 37
I (1062) interpreter: Token 77
I (1065) interpreter: Token 10
I (1069) interpreter: Token 49
I (1073) interpreter: Token 78
I (1076) interpreter: Token 37
I (1080) interpreter: Token 77
I (1084) interpreter: Token 10
I (1088) interpreter: Token 49
I (1091) interpreter: Token 78
I (1095) interpreter: Token 13
I (1099) interpreter: Token 6
I (1102) interpreter: Token 79
I (1106) interpreter: UNNEST 0x3c064dcd
78 4f 00 6e 36 d9 73 80 0d 72 
0x3c064dcd 0x3fc93b1c 0xfefefefe 0xfefefefe 0xaaaab432 
I (1118) interpreter: Token 1400
I (1122) interpreter: scr 222
I (1126) interpreter: Token 3216
I (1130) interpreter: scr 217
I (1134) interpreter: Token 3195
I (1137) interpreter: scr 217
I (1141) interpreter: Token 13
I (1145) interpreter: Token 5
I (1149) interpreter: Token 12
I (1152) interpreter: Token 79
I (1156) interpreter: UNNEST 0x3c059f31
8b 18 49 31 71 8d f0 4f 00 61 
0x3c059f31 0x3c064dd1 0x3fc93b1c 0xfefefefe 0xfefefefe 
I (1168) interpreter: Token 139
I (1172) interpreter: Token 73
I (1176) interpreter: Token 49
I (1179) interpreter: Token 3185
I (1183) interpreter: scr 217
I (1187) interpreter: Token 133
I (1191) interpreter: Token 1277
I (1195) interpreter: scr 221
I (1198) interpreter: Token 65
I (1202) interpreter: Token 1391
I (1206) interpreter: scr 222
I (1210) interpreter: Token 85
2I (1213) interpreter: Token 79
I (1217) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff0 0x3c066e08 0x3c059f35 0x3c064dd1 
I (1230) interpreter: Token 141
I (1233) interpreter: Token 73
I (1237) interpreter: Token 49
I (1241) interpreter: Token 3185
I (1245) interpreter: scr 217
I (1248) interpreter: Token 133
I (1252) interpreter: Token 1277
I (1256) interpreter: scr 221
I (1260) interpreter: Token 65
I (1263) interpreter: Token 1391
I (1267) interpreter: scr 222
I (1271) interpreter: Token 85
0I (1275) interpreter: Token 79
I (1279) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff1 0x3c066e08 0x3c059f35 0x3c064dd1 
I (1291) interpreter: Token 141
I (1295) interpreter: Token 73
I (1298) interpreter: Token 49
I (1302) interpreter: Token 3185
I (1306) interpreter: scr 217
I (1310) interpreter: Token 133
I (1313) interpreter: Token 1277
I (1317) interpreter: scr 221
I (1321) interpreter: Token 65
I (1325) interpreter: Token 1391
I (1329) interpreter: scr 222
I (1332) interpreter: Token 85
2I (1336) interpreter: Token 79
I (1340) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff2 0x3c066e08 0x3c059f35 0x3c064dd1 
I (1352) interpreter: Token 141
I (1356) interpreter: Token 73
I (1360) interpreter: Token 49
I (1363) interpreter: Token 3185
I (1367) interpreter: scr 217
I (1371) interpreter: Token 133
I (1375) interpreter: Token 1277
I (1379) interpreter: scr 221
I (1382) interpreter: Token 65
I (1386) interpreter: Token 1391
I (1390) interpreter: scr 222
I (1394) interpreter: Token 85
3I (1397) interpreter: Token 79
I (1401) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff3 0x3c066e08 0x3c059f35 0x3c064dd1 
I (1413) interpreter: Token 141
I (1417) interpreter: Token 73
I (1421) interpreter: Token 49
I (1425) interpreter: Token 3185
I (1429) interpreter: scr 217
I (1432) interpreter: Token 133
I (1436) interpreter: Token 1277
I (1440) interpreter: scr 221
I (1444) interpreter: Token 65
I (1447) interpreter: Token 1391
I (1451) interpreter: scr 222
I (1455) interpreter: Token 85
-I (1459) interpreter: Token 79
I (1462) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff4 0x3c066e08 0x3c059f35 0x3c064dd1 
I (1475) interpreter: Token 141
I (1478) interpreter: Token 73
I (1482) interpreter: Token 49
I (1486) interpreter: Token 3185
I (1490) interpreter: scr 217
I (1493) interpreter: Token 133
I (1497) interpreter: Token 1277
I (1501) interpreter: scr 221
I (1505) interpreter: Token 65
I (1509) interpreter: Token 1391
I (1512) interpreter: scr 222
I (1516) interpreter: Token 85
0I (1520) interpreter: Token 79
I (1524) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff5 0x3c066e08 0x3c059f35 0x3c064dd1 
I (1536) interpreter: Token 141
I (1540) interpreter: Token 73
I (1543) interpreter: Token 49
I (1547) interpreter: Token 3185
I (1551) interpreter: scr 217
I (1555) interpreter: Token 133
I (1559) interpreter: Token 1277
I (1562) interpreter: scr 221
I (1566) interpreter: Token 65
I (1570) interpreter: Token 1391
I (1574) interpreter: scr 222
I (1577) interpreter: Token 85
3I (1581) interpreter: Token 79
I (1585) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff6 0x3c066e08 0x3c059f35 0x3c064dd1 
I (1597) interpreter: Token 141
I (1601) interpreter: Token 73
I (1605) interpreter: Token 49
I (1608) interpreter: Token 3185
I (1612) interpreter: scr 217
I (1616) interpreter: Token 133
I (1620) interpreter: Token 1277
I (1624) interpreter: scr 221
I (1627) interpreter: Token 65
I (1631) interpreter: Token 1391
I (1635) interpreter: scr 222
I (1639) interpreter: Token 85
-I (1642) interpreter: Token 79
I (1646) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff7 0x3c066e08 0x3c059f35 0x3c064dd1 
I (1658) interpreter: Token 141
I (1662) interpreter: Token 73
I (1666) interpreter: Token 49
I (1670) interpreter: Token 3185
I (1674) interpreter: scr 217
I (1677) interpreter: Token 133
I (1681) interpreter: Token 1277
I (1685) interpreter: scr 221
I (1689) interpreter: Token 65
I (1692) interpreter: Token 1391
I (1696) interpreter: scr 222
I (1700) interpreter: Token 85
1I (1704) interpreter: Token 79
I (1708) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff8 0x3c066e08 0x3c059f35 0x3c064dd1 
I (1720) interpreter: Token 141
I (1724) interpreter: Token 73
I (1727) interpreter: Token 49
I (1731) interpreter: Token 3185
I (1735) interpreter: scr 217
I (1739) interpreter: Token 133
I (1742) interpreter: Token 1277
I (1746) interpreter: scr 221
I (1750) interpreter: Token 65
I (1754) interpreter: Token 1391
I (1758) interpreter: scr 222
I (1761) interpreter: Token 85
8I (1765) interpreter: Token 79
I (1769) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff9 0x3c066e08 0x3c059f35 0x3c064dd1 
I (1781) interpreter: Token 141
I (1785) interpreter: Token 73
I (1789) interpreter: Token 49
I (1792) interpreter: Token 3185
I (1796) interpreter: scr 217
I (1800) interpreter: Token 133
I (1804) interpreter: Token 1277
I (1808) interpreter: scr 221
I (1811) interpreter: Token 65
I (1815) interpreter: Token 1391
I (1819) interpreter: scr 222
I (1823) interpreter: Token 85
 I (1826) interpreter: Token 79
I (1830) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffffa 0x3c066e08 0x3c059f35 0x3c064dd1 
I (1842) interpreter: Token 141
I (1846) interpreter: Token 73
I (1850) interpreter: Token 49
I (1854) interpreter: Token 3185
I (1858) interpreter: scr 217
I (1861) interpreter: Token 133
I (1865) interpreter: Token 1277
I (1869) interpreter: scr 221
I (1873) interpreter: Token 65
I (1876) interpreter: Token 1391
I (1880) interpreter: scr 222
I (1884) interpreter: Token 85
1I (1888) interpreter: Token 79
I (1891) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffffb 0x3c066e08 0x3c059f35 0x3c064dd1 
I (1904) interpreter: Token 141
I (1907) interpreter: Token 73
I (1911) interpreter: Token 49
I (1915) interpreter: Token 3185
I (1919) interpreter: scr 217
I (1922) interpreter: Token 133
I (1926) interpreter: Token 1277
I (1930) interpreter: scr 221
I (1934) interpreter: Token 65
I (1938) interpreter: Token 1391
I (1941) interpreter: scr 222
I (1945) interpreter: Token 85
7I (1949) interpreter: Token 79
I (1953) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffffc 0x3c066e08 0x3c059f35 0x3c064dd1 
I (1965) interpreter: Token 141
I (1969) interpreter: Token 73
I (1972) interpreter: Token 49
I (1976) interpreter: Token 3185
I (1980) interpreter: scr 217
I (1984) interpreter: Token 133
I (1988) interpreter: Token 1277
I (1991) interpreter: scr 221
I (1995) interpreter: Token 65
I (1999) interpreter: Token 1391
I (2003) interpreter: scr 222
I (2006) interpreter: Token 85
:I (2010) interpreter: Token 79
I (2014) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffffd 0x3c066e08 0x3c059f35 0x3c064dd1 
I (2026) interpreter: Token 141
I (2030) interpreter: Token 73
I (2034) interpreter: Token 49
I (2037) interpreter: Token 3185
I (2041) interpreter: scr 217
I (2045) interpreter: Token 133
I (2049) interpreter: Token 1277
I (2053) interpreter: scr 221
I (2056) interpreter: Token 65
I (2060) interpreter: Token 1391
I (2064) interpreter: scr 222
I (2068) interpreter: Token 85
3I (2071) interpreter: Token 79
I (2075) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffffe 0x3c066e08 0x3c059f35 0x3c064dd1 
I (2087) interpreter: Token 141
I (2091) interpreter: Token 73
I (2095) interpreter: Token 49
I (2099) interpreter: Token 3185
I (2103) interpreter: scr 217
I (2106) interpreter: Token 133
I (2110) interpreter: Token 1277
I (2114) interpreter: scr 221
I (2118) interpreter: Token 65
I (2121) interpreter: Token 1391
I (2125) interpreter: scr 222
I (2129) interpreter: Token 85
1I (2133) interpreter: Token 79
I (2137) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xffffffff 0x3c066e08 0x3c059f35 0x3c064dd1 
I (2149) interpreter: Token 141
I (2153) interpreter: Token 79
I (2156) interpreter: UNNEST 0x3c064dd1
4f 00 6e 36 d9 73 80 0d 72 62 
0x3c064dd1 0x3fc93b1c 0xfefefefe 0xfefefefe 0xaaaab432 
I (2169) interpreter: Token 79
I (2172) interpreter: UNNEST 0x3fc93b1c
60 00 00 00 00 00 00 00 00 00 
0x3fc93b1c 0xfefefefe 0xfefefefe 0xaaaab432 0x3c083a60 
I (2185) interpreter: Token 96
I (2188) main_task: Returned from app_main()

And then this is a trace of execute_word("banner", callback_up); note there is no opcode 176 call in this trace

I (690) rodict: romvars 0x3c0668fd
I (690) rodict: sizeof(romvars) 2b4
I (690) rodict: ramvars 0x3fc93ef8
I (691) rodict: ramdict 0x3c083a68
I (695) rodict: sizeof(token_t) 4
I (699) interpreter: Token 14398
I (703) interpreter: scr 217
I (707) interpreter: Token 1395
I (711) interpreter: scr 222
I (714) interpreter: Token 86

I (718) interpreter: Token 2176
I (722) interpreter: scr 217
I (725) interpreter: Token 2164
I (729) interpreter: scr 217
I (733) interpreter: Token 22
I (736) interpreter: Token 28
I (740) interpreter: Token 2143
I (744) interpreter: scr 217
I (747) interpreter: Token 70
I (751) interpreter: Token 1455
I (755) interpreter: scr 217
I (758) interpreter: Token 13
I (762) interpreter: Token 13
I (766) interpreter: Token 79
I (769) interpreter: UNNEST 0x3c058e71
05 25 2d 11 4f 00 5f d9 0f 66 
0x3c058e71 0x3c058ec9 0x3c064ded 0x3fc93b1c 0xfefefefe 
I (782) interpreter: Token 5
I (785) interpreter: Token 37
I (789) interpreter: Token 2093
I (793) interpreter: scr 217
I (796) interpreter: Token 2088
I (800) interpreter: scr 218
I (804) interpreter: Token 133
I (807) interpreter: Token 6
I (811) interpreter: Token 5
I (814) interpreter: Token 2088
I (818) interpreter: scr 218
I (822) interpreter: Token 47
I (825) interpreter: Token 2
I (829) interpreter: Token 79
I (833) interpreter: UNNEST 0x3c058e7d
11 4f 00 5f d9 0f 66 45 26 58 
0x3c058e7d 0x3c058ec9 0x3c064ded 0x3fc93b1c 0xfefefefe 
I (845) interpreter: Token 17
I (848) interpreter: Token 79
I (852) interpreter: UNNEST 0x3c058ec9
10 1b 10 15 4f 00 2e 74 d9 74 
0x3c058ec9 0x3c064ded 0x3fc93b1c 0xfefefefe 0xfefefefe 
I (864) interpreter: Token 16
I (868) interpreter: Token 27
I (872) interpreter: Token 16
I (875) interpreter: Token 21
I (879) interpreter: Token 79
I (882) interpreter: UNNEST 0x3c058ef1
78 4f 00 80 d9 83 22 69 69 4f 
0x3c058ef1 0x3c064dfc 0x3fc93b1c 0xfefefefe 0xfefefefe 
I (895) interpreter: Token 1400
I (898) interpreter: scr 222
I (902) interpreter: Token 3216
I (906) interpreter: scr 217
I (909) interpreter: Token 3195
I (913) interpreter: scr 217
I (917) interpreter: Token 13
I (920) interpreter: Token 5
I (924) interpreter: Token 12
I (928) interpreter: Token 79
I (931) interpreter: UNNEST 0x3c059f31
8b 18 49 31 71 8d f0 4f 00 61 
0x3c059f31 0x3c058ef5 0x3c064dfc 0x3fc93b1c 0xfefefefe 
I (943) interpreter: Token 139
I (947) interpreter: Token 73
I (951) interpreter: Token 49
I (954) interpreter: Token 3185
I (958) interpreter: scr 217
I (962) interpreter: Token 133
I (966) interpreter: Token 1277
I (969) interpreter: scr 221
I (973) interpreter: Token 65
I (977) interpreter: Token 1391
I (980) interpreter: scr 222
I (984) interpreter: Token 85
CI (988) interpreter: Token 79
I (991) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff3 0x3c064dfb 0x3c059f35 0x3c058ef5 
I (1003) interpreter: Token 141
I (1007) interpreter: Token 73
I (1011) interpreter: Token 49
I (1015) interpreter: Token 3185
I (1019) interpreter: scr 217
I (1022) interpreter: Token 133
I (1026) interpreter: Token 1277
I (1030) interpreter: scr 221
I (1034) interpreter: Token 65
I (1037) interpreter: Token 1391
I (1041) interpreter: scr 222
I (1045) interpreter: Token 85
FI (1049) interpreter: Token 79
I (1052) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff4 0x3c064dfb 0x3c059f35 0x3c058ef5 
I (1065) interpreter: Token 141
I (1069) interpreter: Token 73
I (1072) interpreter: Token 49
I (1076) interpreter: Token 3185
I (1080) interpreter: scr 217
I (1084) interpreter: Token 133
I (1087) interpreter: Token 1277
I (1091) interpreter: scr 221
I (1095) interpreter: Token 65
I (1099) interpreter: Token 1391
I (1103) interpreter: scr 222
I (1106) interpreter: Token 85
oI (1110) interpreter: Token 79
I (1114) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff5 0x3c064dfb 0x3c059f35 0x3c058ef5 
I (1126) interpreter: Token 141
I (1130) interpreter: Token 73
I (1134) interpreter: Token 49
I (1137) interpreter: Token 3185
I (1141) interpreter: scr 217
I (1145) interpreter: Token 133
I (1149) interpreter: Token 1277
I (1153) interpreter: scr 221
I (1156) interpreter: Token 65
I (1160) interpreter: Token 1391
I (1164) interpreter: scr 222
I (1168) interpreter: Token 85
rI (1171) interpreter: Token 79
I (1175) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff6 0x3c064dfb 0x3c059f35 0x3c058ef5 
I (1187) interpreter: Token 141
I (1191) interpreter: Token 73
I (1195) interpreter: Token 49
I (1199) interpreter: Token 3185
I (1202) interpreter: scr 217
I (1206) interpreter: Token 133
I (1210) interpreter: Token 1277
I (1214) interpreter: scr 221
I (1218) interpreter: Token 65
I (1221) interpreter: Token 1391
I (1225) interpreter: scr 222
I (1229) interpreter: Token 85
tI (1233) interpreter: Token 79
I (1236) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff7 0x3c064dfb 0x3c059f35 0x3c058ef5 
I (1249) interpreter: Token 141
I (1252) interpreter: Token 73
I (1256) interpreter: Token 49
I (1260) interpreter: Token 3185
I (1264) interpreter: scr 217
I (1267) interpreter: Token 133
I (1271) interpreter: Token 1277
I (1275) interpreter: scr 221
I (1279) interpreter: Token 65
I (1283) interpreter: Token 1391
I (1286) interpreter: scr 222
I (1290) interpreter: Token 85
hI (1294) interpreter: Token 79
I (1298) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff8 0x3c064dfb 0x3c059f35 0x3c058ef5 
I (1310) interpreter: Token 141
I (1314) interpreter: Token 73
I (1317) interpreter: Token 49
I (1321) interpreter: Token 3185
I (1325) interpreter: scr 217
I (1329) interpreter: Token 133
I (1333) interpreter: Token 1277
I (1336) interpreter: scr 221
I (1340) interpreter: Token 65
I (1344) interpreter: Token 1391
I (1348) interpreter: scr 222
I (1351) interpreter: Token 85
 I (1355) interpreter: Token 79
I (1359) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffff9 0x3c064dfb 0x3c059f35 0x3c058ef5 
I (1371) interpreter: Token 141
I (1375) interpreter: Token 73
I (1379) interpreter: Token 49
I (1382) interpreter: Token 3185
I (1386) interpreter: scr 217
I (1390) interpreter: Token 133
I (1394) interpreter: Token 1277
I (1398) interpreter: scr 221
I (1401) interpreter: Token 65
I (1405) interpreter: Token 1391
I (1409) interpreter: scr 222
I (1413) interpreter: Token 85
bI (1416) interpreter: Token 79
I (1420) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffffa 0x3c064dfb 0x3c059f35 0x3c058ef5 
I (1432) interpreter: Token 141
I (1436) interpreter: Token 73
I (1440) interpreter: Token 49
I (1444) interpreter: Token 3185
I (1448) interpreter: scr 217
I (1451) interpreter: Token 133
I (1455) interpreter: Token 1277
I (1459) interpreter: scr 221
I (1463) interpreter: Token 65
I (1466) interpreter: Token 1391
I (1470) interpreter: scr 222
I (1474) interpreter: Token 85
uI (1478) interpreter: Token 79
I (1481) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffffb 0x3c064dfb 0x3c059f35 0x3c058ef5 
I (1494) interpreter: Token 141
I (1498) interpreter: Token 73
I (1501) interpreter: Token 49
I (1505) interpreter: Token 3185
I (1509) interpreter: scr 217
I (1513) interpreter: Token 133
I (1516) interpreter: Token 1277
I (1520) interpreter: scr 221
I (1524) interpreter: Token 65
I (1528) interpreter: Token 1391
I (1532) interpreter: scr 222
I (1535) interpreter: Token 85
iI (1539) interpreter: Token 79
I (1543) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffffc 0x3c064dfb 0x3c059f35 0x3c058ef5 
I (1555) interpreter: Token 141
I (1559) interpreter: Token 73
I (1563) interpreter: Token 49
I (1566) interpreter: Token 3185
I (1570) interpreter: scr 217
I (1574) interpreter: Token 133
I (1578) interpreter: Token 1277
I (1582) interpreter: scr 221
I (1585) interpreter: Token 65
I (1589) interpreter: Token 1391
I (1593) interpreter: scr 222
I (1596) interpreter: Token 85
lI (1600) interpreter: Token 79
I (1604) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffffd 0x3c064dfb 0x3c059f35 0x3c058ef5 
I (1616) interpreter: Token 141
I (1620) interpreter: Token 73
I (1624) interpreter: Token 49
I (1628) interpreter: Token 3185
I (1631) interpreter: scr 217
I (1635) interpreter: Token 133
I (1639) interpreter: Token 1277
I (1643) interpreter: scr 221
I (1646) interpreter: Token 65
I (1650) interpreter: Token 1391
I (1654) interpreter: scr 222
I (1658) interpreter: Token 85
tI (1662) interpreter: Token 79
I (1665) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xfffffffe 0x3c064dfb 0x3c059f35 0x3c058ef5 
I (1678) interpreter: Token 141
I (1681) interpreter: Token 73
I (1685) interpreter: Token 49
I (1689) interpreter: Token 3185
I (1693) interpreter: scr 217
I (1696) interpreter: Token 133
I (1700) interpreter: Token 1277
I (1704) interpreter: scr 221
I (1708) interpreter: Token 65
I (1712) interpreter: Token 1391
I (1715) interpreter: scr 222
I (1719) interpreter: Token 85
 I (1723) interpreter: Token 79
I (1727) interpreter: UNNEST 0x3c059f45
8d f0 4f 00 61 90 d9 b2 71 4f 
0x3c059f45 0xffffffff 0x3c064dfb 0x3c059f35 0x3c058ef5 
I (1739) interpreter: Token 141
I (1743) interpreter: Token 79
I (1746) interpreter: UNNEST 0x3c058ef5
4f 00 80 d9 83 22 69 69 4f 00 
0x3c058ef5 0x3c064dfc 0x3fc93b1c 0xfefefefe 0xfefefefe 
I (1759) interpreter: Token 79
I (1762) interpreter: UNNEST 0x3c064dfc
00 00 00 72 00 00 00 00 2d 85 
0x3c064dfc 0x3fc93b1c 0xfefefefe 0xfefefefe 0xaaaab432 
I (1775) interpreter: Token 3683840

So why does execute_word(".built", callback_up); work and execute_word("banner", callback_up); fail?

Next I tried this which does not use any CCALL, it should work, right? But it fails the same way.

    forth_init();
    //execute_word(".built", callback_up);

        char *cmd =
R"FORTH(

see see

)FORTH";

    spush((int)cmd, getCallback_up());
    spush(strlen(cmd), getCallback_up());
    execute_word("evaluate", getCallback_up());
snipped lots of lines

 (11193) interpreter: Token 6
I (11196) interpreter: Token 5
I (11200) interpreter: Token 2088
I (11204) interpreter: scr 218
I (11208) interpreter: Token 47
I (11212) interpreter: Token 2
I (11215) interpreter: Token 79
I (11219) interpreter: UNNEST 0x3c058e89
11 4f 00 5f d9 0f 66 45 26 58 
0x3c058e89 0x3c058ed5 0x3c05fb69 0x3c05fd49 0x3c05f6bd 
I (11231) interpreter: Token 17
I (11235) interpreter: Token 79
I (11239) interpreter: UNNEST 0x3c058ed5
10 1b 10 15 4f 00 2e 74 d9 74 
0x3c058ed5 0x3c05fb69 0x3c05fd49 0x3c05f6bd 0x3c06088d 
I (11251) interpreter: Token 16
I (11255) interpreter: Token 27
I (11259) interpreter: Token 16
I (11263) interpreter: Token 21
I (11267) interpreter: Token 79
I (11270) interpreter: UNNEST 0x3c05a4a5
bf 4f 00 72 de d9 90 de 86 4f 
0x3c05a4a5 0x3c05fb84 0x3c05fd49 0x3c05f6bd 0x3c06088d 
I (11283) interpreter: Token 1471
I (11287) interpreter: scr 217
I (11291) interpreter: Token 11
I (11294) interpreter: Token 11
I (11298) interpreter: Token 79
I (11302) interpreter: UNNEST 0x3c05a4a9
4f 00 72 de d9 90 de 86 4f 00 
0x3c05a4a9 0x3c05fb84 0x3c05fd49 0x3c05f6bd 0x3c06088d 
I (11314) interpreter: Token 79
I (11318) interpreter: UNNEST 0x3c05fb84
00 00 00 00 00 00 00 00 00 00 
0x3c05fb84 0x3c05fd49 0x3c05f6bd 0x3c06088d 0x3c0608e1 
I (11330) interpreter: Token 2277376

when I turn the debug off

I (700) rodict: romvars 0x3c066845
I (700) rodict: sizeof(romvars) 2b4
I (700) rodict: ramvars 0x3fc93ef8
I (701) rodict: ramdict 0x3c083a68
I (705) rodict: sizeof(token_t) 4
: see 
ESP-ROM:esp32s3-20210327

see is running and prints the first line, but when it tries to decompile it is dying.

Using the same app.dic and the host interpreter, "see see" works.

onsmirl@ares:~/l5/components/forth/regen$ ./forth app.dic
C Forth  Copyright (c) 2008 FirmWorks
ok see see
: see 
   ' ['] (see) catch if
      drop 
   then
;
ok

My rodict.c source

// C Forth 2005.  Copyright (c) 2005 FirmWorks

#include "forth.h"
#include "compiler.h"
#include "esp_log.h"
#include "esp_heap_caps.h"

static const char *TAG = "rodict";

const struct header builtin_hdr = {
#include "dicthdr.h"
};

const u_char romvars[] = {
#include "userarea.h"
};

cell ramvars[MAXVARS];

const u_char romdict[] = {
#include "dict.h"
};

u_char *ramdict;

// dictmax is ignored because the dictionary is defined statically
cell *
prepare_builtin_dictionary(int dictmax)
{
    const u_char *p;
    const u_char *lastp;
    u_char *q;
    p = romvars;
    lastp = romvars + sizeof(romvars);
    q = (u_char *)ramvars;

ESP_LOGI(TAG, "romvars %p", p);
ESP_LOGI(TAG, "sizeof(romvars) %x", sizeof(romvars));
ESP_LOGI(TAG, "ramvars %p", q);
    while (p < lastp) {
        *q++ = *p++;
    }
    ramdict = heap_caps_malloc(MAXDICT, MALLOC_CAP_SPIRAM);
ESP_LOGI(TAG, "ramdict %p", ramdict);
ESP_LOGI(TAG, "sizeof(token_t) %x", sizeof(token_t));
    init_compiler(romdict, ramdict, 
                  (token_t)(sizeof(romdict) / sizeof(token_t)),
                  ramdict, ramdict + MAXDICT, ramvars);
    return ramvars;
}
quozl commented 9 months ago

Starting to smell like a rogue write somewhere. Try dumping the ccalls table and dictionary fragments at interpreter start and at the time the bad token is encountered?

jonsmirl commented 9 months ago

I'll start a new issue with better info