Error running validate.tcl

[zdia]

Compiling and linking has finished successfully on a FreeBSD 8.2 system. But there was an error message caused by file "/lib/tcl8" in the following source lines of setupvfs.tcl.

lappend clifiles lib/tcl8 \ lib/tcl8@/clock.tcl \ lib/tcl8@/tm.tcl

I deleted this element and then setupvfs.tcl passed. But validate.tcl is complaining:

Version : Tcl 8.5.11 8.2-RELEASE i386 package rechan: 1.0 couldn't read file "/usr/home/dia/gorilla/work/kitgen/8.5/kit-mybuild/tclkit-cli/lib/tcl8.5/tm.tcl": no such file or directory while executing "source /usr/home/dia/gorilla/work/kitgen/8.5/kit-mybuild/tclkit-cli/lib/tcl8.5/tm.tcl" (in namespace eval "::" script line 1) invoked from within "namespace eval :: $auto_index($name)" (procedure "auto_load" line 35) invoked from within "auto_load $name [uplevel 1 {::namespace current}]" (autoloading "::tcl::tm::UnknownHandler") (procedure "::unknown" line 30) invoked from within "::tcl::tm::UnknownHandler ::tclPkgUnknown starkit 0-" ("package unknown" script) invoked from within "package require $pkg" ("foreach" body line 1) invoked from within "foreach pkg $ext {lappend r $pkg [package require $pkg]; puts "package $pkg: [package require $pkg]"}" (file "../../validate.tcl" line 19)

The program tclkit cannot find the package starkit:

dhcppc3# ./tclkit-cli % package require rechan 1.0 % package require starkit couldn't read file "/usr/home/dia/gorilla/work/kitgen/8.5/kit-mybuild/tclkit-cli/lib/tcl8.5/tm.tcl": no such file or directory % puts $auto_path /usr/home/dia/gorilla/work/kitgen/8.5/kit-mybuild/tclkit-cli/lib/tcl8.5 /usr/home/dia/gorilla/work/kitgen/8.5/kit-mybuild/tclkit-cli/lib

Well, there is no path .../kit-mybuild/tclkit-cli/lib" in my folder kit-mybuild:

$ tree kit-mybuild/ -L 2 -F kit-mybuild/ |-- Makefile |-- build/ | |-- bin/ | |-- files/ | |-- include/ | |-- lib/ | |-- man/ | |-- pwb.o | |-- rechan.o | |-- share/ | |-- tcl/ | |-- tclAppInit.o | |-- tclvfs/ | |-- vqtcl/ | |-- zlib/ | -- zlib.o* |-- kit-cli* |-- lib/ |-- tclConfig.sh `-- tclkit-cli

It seems kit-cli -init- with setupvfs.tcl did not write correctly the vfs into the folder tclkit-cli.

How could I debug this issue?

[patthoyts]

I can';t really tell what is wrong - but deleting things from the setupvfs is going to break it. tclkit has a virtual filesystem that contains all the tcl library files that are required to run tcl. If you remove some, then you have a broken tcl installation. The odd path you are seeing is because tcl itself can mount a virtual filesystem and then any errors from that are reported as a path that leads into a vfs file. In this case tclkit-cli is a file that is being mounted as a filesystem. So it is talking about the lib subdirectory in the tclkit-cli vfs. If you couldn't get tm.tcl -- either thats something you removed (bad idea) or it cannot mount the vfs at all - in which case the issue is in the build someplace. If you have tcl and tclvfs then you can mount these files separately. Something like package require vfs; package require vfs::mk4; vfs::mk4::Mount filename mountpoint. Then within the tcl interp you can cd into the mountpoint and use glob to explore the virtual filesystem.

[zdia]

but deleting things from the setupvfs is going to break it.

Ok, I tried a build with the original setupvfs.tcl. Then gmake, respectively kit-cli reports the following error:

Must remake target tclkit-cli'. Invoking recipe from ../../makefile.include:25 to update targettclkit-cli'. cp kit-cli tclkit-cli && strip tclkit-cli && : tclkit-cli ./kit-cli -init- ../../setupvfs.tcl tclkit-cli cli application-specific initialization failed: error copying "build/lib/tcl8": no such file or directory

This is true:

kit-mybuild/ |-- build | |-- bin | |-- files | |-- include | | |-- generic | | -- unix | |-- lib | | |-- pkgconfig | | |-- tcl8.5 | | |-- vfs1.4.1 | |-- vqtcl4.1 | |-- man | | -- mann | |-- share | |-- man | |-- tcl | | -- dltest | |-- tclvfs | |-- library | |-- vqtcl

There is a build/lib/tcl8.5 but no folder /build/lib/tcl8

This is the reason why I deleted the element in the lines 254ff:

if {$tcl_version eq "8.4"} { ... } else { lappend clifiles lib/tcl8 \ lib/tcl8@/clock.tcl \ lib/tcl8@/tm.tcl ...

tcl8@ is mapped to tcl8.5, clock.tcl and tm.tcl are present.

So some setup file failed to create build/lib/tcl8?

[patthoyts]

Yes - when run on linux, here is a listing from the build/lib directory:

pat@fox /opt/src/kitgen/8.5/linux-ix86/build/lib (master) $ ls itcl3.4 libtclstub8.5.a libz.a libz.so.1.2.5 tcl8 thread2.6.6 vfs1.4.1 itclConfig.sh libtk8.5.so libz.so Mk4tcl2.4.9.7 tcl8.5 tk8.5 libtcl8.5.a libtkstub8.5.a libz.so.1 pkgconfig tclConfig.sh tkConfig.sh

Note there is a tcl8 there on the first row. If this is missing then something went wrong during the tcl build. That is done from the build/tcl directory. We do 'make install-binaries install-libraries install-private-headers' so if you change to that folder and issue that command, you should be able to examine the output for anything going wrong. Presumably something freebsd specific is missing or causing this directory to be mis-created.

[zdia]

Note there is a tcl8 there on the first row. If this is missing then something went wrong during the tcl build

Alas, it seems to be like that. It would have been to easy just to call "configure && make"

Following your hints I made some vfs test calls on Linux with my actual 8.5.11 tclsh

(gorilla) 1 % package require vfs 1.4.1 (gorilla) 2 % package require vfs::mk4 1.10.1 (gorilla) 4 % exec touch base (gorilla) 5 % vfs::mk4::Mount base virtual mk4vfs1 (gorilla) 6 % file copy viewhelp.txt virtual/viewhelp.tst 24365 (gorilla) 7 % cd virtual/ (virtual) 8 % glob * viewhelp.tst

On FreeBSD I get:

% vfs::mk4::Mount base virtual cannot open file: base

Testing the general file access:

% load libvfs1.4.1.a invalid file format

though FreeBSD recognizes the format:

file libvfs1.4.1.a libvfs1.4.1.a: current ar archive

Presumably something freebsd specific is missing or causing this directory to be mis-created.

Yes, there is still some work to do.

... We do 'make install-binaries install-libraries install-private-headers' so if you change to that folder and issue that command, you should be able to examine the output for anything going wrong.

Ok, I have to check compilation process and especially the install procedures.

Thank you for the hints

[zdia]

STEP #1: issue "load libvfs1.4.1.a"

Tcl expects a shared library to load but is faced with a indexed bundle of files, created with ar and ranlib. I have 'grep'ed the repo for "vfs1" to see if there is some additional linking action to create a *.so file based on libvfs1.4.1.a but I didn't find it.

In my teapot based Linux packages folder I have found a libvfs1.4.1.so:

~/ActiveTcl-8.5/lib/teapot/package/linux-glibc2.3-ix86/lib/vfs1.4.1/libvfs1.4.1.so

The teapot loading script vfs1.4.1/vfs.tcl does:

load [file join $::vfs::self libvfs1.4.1.so]

whereas in my kitgen based build environment I have:

load [file join $::vfs::self libvfs1.4.1.a]

which will not work.

Investigating the Makefile in the tclvfs/ folder I have noticed that there is no target for a shared library.

line 226-:

# $(PKG_LIB_FILE) should be listed as part of the BINARIES variable
# mentioned above.  That will ensure that this target is built when you
# run "make binaries". '
...
$(PKG_LIB_FILE): $(PKG_OBJECTS)
    -rm -f $(PKG_LIB_FILE)
    ${MAKE_LIB}
    $(RANLIB) $(PKG_LIB_FILE)

whilst the variable MAKE_LIB is defined like this:

MAKE_LIB = ${STLIB_LD} $@ $(PKG_OBJECTS)

the target will run just the static version with AR and RANLIB for libvfs1.4.1.a

If the target will be:

$(PKG_LIB_FILE): $(PKG_OBJECTS) -rm -f $(PKG_LIB_FILE) ${MAKE_SHARED_LIB}

$(RANLIB) $(PKG_LIB_FILE)

with default Makefile definition:

MAKE_SHARED_LIB = ${SHLIB_LD} -o $@ $(PKG_OBJECTS) ${SHLIB_LD_LIBS}

then I get the libvfs1.4.1.so

$ tclsh8.5 
% load libvfs1.4.1.so
% package require vfs
1.4.1

The /library/vfs.tcl in my build folder has to be patched, too:

load [file join $::vfs::self libvfs1.4.1.so]

and then I will see why the folder /lib/tc8 is missing on my FreeBSD installation.

Ah, my last configuration command for kitgen was:

sh config.sh 8.5/kit-mybuild thread mk cli dyn

[patthoyts]

I got a bit lost reading through this. The tcl load command loads shared object (aka dynamic libraries). So those usually have a .so extension. However, tclkit has nearly everything statically linked into a single executable. So we build metakit and vfs as static libraries (with a .a extension) and link them in. In this case, the load command to use must be 'load "" vfs' which says try doing a package require for vfs from this executable. This is often seen for Tk when done from wish. For extensions that are staticlly linked, we create pkgIndex files that contain 'package ifneeded Xyzzy [list load "" Xyzzzy]' which makes [package require Xyzzy] happy. All this should already be working you should be making a static vfs extension and it should be getting linked into the kit or kit-cli executable. Note the setupvfs.tcl script has a 'load {} vfs' quite early on. The 'dyn' or 'gui' args for 'sh config.sh ....' just specify how to make Tk. On unix we can either include a shared Tk library in the vfs or we can build it static and link it in. The first option produces a binary that can be run on systems without Tk installed until someone tries to load Tk. The second is used on windows really but can also work on unix and makes a tclkit binary that is linked to X libraries.

[zdia]

Ok, I am seeing clearer now ... A module for a pure tclsh needs a *.so library whereas a tclkit has all the libraries inside it.

Finally I have got further and resolved the "/lib/tcl8" issue:

The Makefile for my build/tcl target was taken from the FreeBSD 8.5.11 Tcl installation and it had no target for "install-tm"

install-libraries: libraries $(INSTALL_TZDATA) install-msgs

I have added "install-tm"

install-libraries: libraries $(INSTALL_TZDATA) install-msgs install-tm
...
install-tm:
for i in tcl8 tcl8/8.3 tcl8/8.4  tcl8/8.4/platform tcl8/8.5 tcl8/8.6; \
    do \
...

Just for curiosity I took a look on the unix/Makefile on Sourceforge and to my surprise it was different from the BSD version:

install-libraries: libraries
for i in opt0.4 http1.0 encoding ../tcl8 ../tcl8/8.4  ../tcl8/8.4/platform ../tcl8/8.5 ../tcl8/8.6; \
    do \
...

With my little Makefile patch there was no error message in regard to lib/tcl8 but:

./tclkit-cli ../../validate.tcl version : Tcl 8.5.11 8.2-RELEASE i386 package rechan: 1.0 unable to alloc 3825270792 bytes gmake: *\ [tclkit-cli] Abort trap: 6 (core dumped)

Well, I have given FreeBSD just 10 GB in my VirtualBox installation so I have to make a new installation. And this time I will take the Tcl sources from Sourceforge and not from the BSD port ...

By the way: Is it the tclkit which wants to allocate so much space?

[patthoyts]

unable to alloc 3825270792 bytes that is some error. Either from metakit failing to work or zlib. 'files' in the vfs are compressed using zlib. If there is a problem with the database then sometimes zlib will be passed stupid numbers for the uncompressed size of the file and will attempt to allocate that to hold the unpacked file. You'd have to examine the core file and work out which component was to blame. You might want to run the metakit build and all its tests. Also - notify the FreeBSD maintainer they broke the makefiles so it gets fixed upstream of you. There should not be any reason to have modified build files on any unix system. Tcl is supposed to install out-of-the-box on all of them.