Error loading IQM files containing animation under Windows

[Negdayen]

$ run

-- testing load-iqm ----------------------------------------------------------
(load-iqm "./cube.iqm") .............................................. [ PASS]
(load-iqm "../hypergiant.scm") ....................................... [ PASS]
2 tests completed in 0.006 seconds.
2 out of 2 (100%) tests passed.
-- done testing load-iqm -----------------------------------------------------

Error: (load-iqm) Poorly formed animation

        Call history:

        iqm.scm:422: text45854586
        iqm.scm:422: text45854586
        iqm.scm:422: ##sys#dynamic-wind
        iqm.scm:422: n-vertices45984599
        iqm.scm:422: n-vertices45984599
        iqm.scm:250: offset->bitstring
        iqm.scm:114: iqm-file
        iqm.scm:115: iqm-file
        iqm.scm:115: bitstring#bitstring-end
        iqm.scm:114: bitstring#bitstring-share
        iqm.scm:249: bitstring-map
        iqm.scm:124: getter
        iqm.scm:208: bitstring#->bitstring
        iqm.scm:208: bitstring#->bitstring
        iqm.scm:208: bitstring#bitstring-read
        iqm.scm:217: error              <--

[AlexCharlton]

This is very surprising. I'm not sure I know how to approach this one :(

I guess as a basic sanity: are you able to load that same file with something else (e.g. Blender)?

[Negdayen]

I couldn't find an importer for Blender, but I did find another piece of software able to view IQM models and was able to view mrfixit.iqm idling and moving its fingers (the viewer did not load the texture, but I think that may be unrelated).

I also ran the tests under Ubuntu and everything passed there, so there's that.

[Negdayen]

I also downloaded and ran the tests provided by the bitstring library--everything passed.

[AlexCharlton]

Dang, this is a real pain. Do non-animated IQM files load alright? Are you able to compare the results of the header that you get from Windows to the header on Linux. Insert

(print "magic " magic "\n"
       "check " check "\n"
       "version " version "\n"
       "file-size " file-size "\n"
       "flags " flags "\n"
       "n-text " n-text "\n"
       "text-offset " text-offset "\n"
       "n-meshes " n-meshes "\n"
       "meshes-offset " meshes-offset "\n"
       "n-vertex-arrays " n-vertex-arrays "\n"
       "n-vertexes " n-vertexes "\n"
       "vertex-arrays-offset " vertex-arrays-offset "\n"
       "n-triangles " n-triangles "\n"
       "triangles-offset " triangles-offset "\n"
       "adjacency-offset " adjacency-offset "\n"
       "n-joints " n-joints "\n"
       "joints-offset " joints-offset "\n"
       "n-poses " n-poses "\n"
       "poses-offset " poses-offset "\n"
       "n-animations " n-animations "\n"
       "animations-offset " animations-offset "\n"
       "n-frames " n-frames "\n"
       "n-frame-channels " n-frame-channels "\n"
       "frames-offset " frames-offset "\n"
       "bounds-offset " bounds-offset "\n"
       "n-comment " n-comment "\n"
       "comment-offset " comment-offset "\n"
       "n-extensions " n-extensions "\n"
       "extensions-offset " extensions-offset)

At line 400 of iqm.scm to see the results.

[Negdayen]

I inserted this code into iqm.scm (I had to comment out check as this was causing csc to complain):

(define (load-iqm file #!optional base-iqm)
  (bitmatch (file->u8vector file)
    (((iqm-header bitpacket)
      (rest bitstring))
     (unless (= version 2)
       (error 'load-iqm "Only IQM version 2 is supported:" file))
     (with-output-to-file (string-append file "-header")
       (lambda ()
         (print "magic " magic "\n"
              ;;"check " check "\n"
                "version " version "\n"
                "file-size " file-size "\n"
                "flags " flags "\n"
                "n-text " n-text "\n"
                "text-offset " text-offset "\n"
                "n-meshes " n-meshes "\n"
                "meshes-offset " meshes-offset "\n"
                "n-vertex-arrays " n-vertex-arrays "\n"
                "n-vertexes " n-vertexes "\n"
                "vertex-arrays-offset " vertex-arrays-offset "\n"
                "n-triangles " n-triangles "\n"
                "triangles-offset " triangles-offset "\n"
                "adjacency-offset " adjacency-offset "\n"
                "n-joints " n-joints "\n"
                "joints-offset " joints-offset "\n"
                "n-poses " n-poses "\n"
                "poses-offset " poses-offset "\n"
                "n-animations " n-animations "\n"
                "animations-offset " animations-offset "\n"
                "n-frames " n-frames "\n"
                "n-frame-channels " n-frame-channels "\n"
                "frames-offset " frames-offset "\n"
                "bounds-offset " bounds-offset "\n"
                "n-comment " n-comment "\n"
                "comment-offset " comment-offset "\n"
                "n-extensions " n-extensions "\n"
                "extensions-offset " extensions-offset))) ...

The resulting cube.linux-iqm-header and cube.windows-iqm-header file look to be identical. However, mrfixit.linux-iqm-header and mrfixit.windows-iqm-header have a significantly different magic bitstring. mrfixit.linux-iqm-header has 257808 elements in its magic u8vector, while mrfixit.windows-iqm-header has 1544 elements in its magic u8vector. They look to identical up to the point where the Windows version is cut short. As for the other variables, they are identical.

[AlexCharlton]

Ah ha! We're getting somewhere. The contents of magic are the entire file that was internalized by the bitstring library (contained in a bitstring object that defines what segment of that vector are being referenced -- in this case, the first 128 bits). So: the windows file is not being completely loaded, either by the bitstring library or when the file is read. To figure out which is which, can you look at the result of calling this function on mrfixit.iqm on Windows:

(use srfi-4)

(define (file->u8vector file)
  (with-input-from-file file
    (lambda ()
      (read-u8vector #f))))

If read-u8vector is working correctly, you should end up with a 257808 element vector (like you got on Linux). If not, we have a bug in a core library!

[Negdayen]

Yes, read-u8vector under Chicken 4.10 compiled with MinGW is not reading more than 1544 u8.

[AlexCharlton]

In doing some digging, I just learned about an undocumented argument that with-input-from-file accepts. Can you try the same experiment, but with:

(use srfi-4)

(define (file->u8vector file)
  (with-input-from-file file
    (lambda ()
      (read-u8vector #f))
    #:binary))

[Negdayen]

With the added #:binary argument:

=> (u8vector-length (file->u8vector "eggs/hypergiant/examples/mrfixit.iqm"))
257808

[AlexCharlton]

Amazing! Thanks for your help in debugging this, @Negdayen . I've pushed a new release that should fix the problem. Let me know!

Fix in 9411cf6ef3ec65b1f82e251b871d4a1d0c5ff8a6

[Negdayen]

Tests pass and can successfully run examples/models.scm now!

[AlexCharlton]

Woo!