Support for pre-processed, non-shell files

[shiblon]

Originally reported on Google Code with ID 133

My project requires that I run my TeX file through a pre-processor before creating a
PDF. Specifically:

1. Edit foo.tex
2. Run lhs2tex, producing foo.f.tex
3. Run pdflatex foo.f.tex.

"foo.f.tex" is the file that pdflatex sees, but its really an intermediate file. This
causes several problems:

1. Have to execute "make foo.f.pdf" rather thank "make foo".
2. Have to write Makefile rules in terms of "foo.f.tex" rather than 
"foo.tex"

One solution is to add ".f.tex.sh" files for each ".tex" file. This creates the right
dependencies, but executes the scripts every time. 

The best solution I have found so far is two-fold:

1. Redefine the .sh rule so it has no recipe:
  %.tex:: %.tex.sh

2. Define my own rule to run lhs2tex:
  %.f.tex: %.tex
    lhs2tex --poly -o $@ $<

This doesn't allow me to execute "make foo" (I still have to write "make foo.f.pdf")
but otherwise it works.

I've attached a tar file containing a sample directory with Makefile 2.2.0 and Targets.ini
set up so you can build foo.f.pdf. If you just build foo.pdf, I have LaTeX report an
error. 

Reported by jgbailey on 2011-06-06 16:18:37

---

• Attachment: foo.tar.gz

[shiblon]

Seems unchanged from before - touching b.tex does not cause a.pdf to get rebuilt.

A transcript follows, and I've attached a tarball of the directory after the last command
was run. Note the two ".fls" files.

== transcript ==

>make a
NOTE: You may ignore warnings about the following files:

     a.d

Makefile:2711: a.d: No such file or directory
= a.tex --> a.d a.pdf.1st.make (0-1) =
sed: can't read a.fls: No such file or directory
= a.tex --> a.pdf (1-2) =
Success!  Wrote 1 page, 10800 bytes to a.pdf

12:52:13.44 Wed 06/22/2011 ] C:\Users\baileyj\Source\foo
>make a
= a.tex --> a.d a.pdf.1st.make (0-1) =
sed: can't read a.fls: No such file or directory
= a.tex --> a.pdf (0-2) =
Success!  Wrote 1 page, 10800 bytes to a.pdf

12:52:44.29 Wed 06/22/2011 ] C:\Users\baileyj\Source\foo
>make a
make: Nothing to be done for `a'.

12:52:49.39 Wed 06/22/2011 ] C:\Users\baileyj\Source\foo
>touch b.tex

12:52:54.17 Wed 06/22/2011 ] C:\Users\baileyj\Source\foo
>make a
make: Nothing to be done for `a'.

Reported by jgbailey on 2011-06-22 19:55:01

---

• Attachment: foo.tar.gz

[shiblon]

Looks like you attached something *before* the last command was run.

I'm mystified by this.  Did you do "make clean" before trying again?

Reported by shiblon on 2011-06-22 19:58:35

[shiblon]

Wait, I might have the wrong file.  Hold on.

Reported by shiblon on 2011-06-22 19:58:53

[shiblon]

OK - got the right file.  Looking now.

Reported by shiblon on 2011-06-22 19:59:51

[shiblon]

All right.  This is very strange.  It looks like your pdflatex didn't honor the "jobname"
setting.

Try this for me.

> make clean
> rm *.fls
> pdflatex -recorder -jobname=a -interaction=batchmode -file-line-error a

After that, is there a file 'a.fls' in the directory?

Reported by shiblon on 2011-06-22 20:01:56

[shiblon]

Nope, a.fls still doesn't appear. 

I tried various combinations of options (w/ and w/o -jobname, w/ and w/o -file-line-error,
etc.) and it looks like -recorder always produces an fls file named after the PID of
pdflatex.

Reported by jgbailey on 2011-06-22 20:08:48

[shiblon]

OK.  Well, that sucks.  What it is *supposed* to do is create the file with the PID,
then *rename* it to the jobname.fls file.

Why that isn't happening is beyond me.

Minimally, you can try running it with just the recorder and jobname flags.  Perhaps
interaction mode is mucking with it (which would also suck, but at least we'd know).

Reported by shiblon on 2011-06-22 20:11:37

[shiblon]

I've been poking around even more, and everything I'm reading indicates that this should
be working even without jobname specified.  If you just run "pdflatex -recorder a",
what happens?

Reported by shiblon on 2011-06-22 20:25:50

[shiblon]

I think it's a bug. I found another report of the same behavior
(http://lists.metaprl.org/pipermail/omake/2011-April/002166.html). I
have emailed the tex-live mailing list.

At least the .lhs stuff is working fine! Thanks for all your work on this ...

Reported by jgbailey on 2011-06-22 20:53:03

[shiblon]

Further along in that discussion thread, they talk about using jobname as a working
fix.  I'm surprised it doesn't work.

Reported by shiblon on 2011-06-22 22:56:24

[shiblon]

According to the tex-live mailing list, this feature works in TeXLive 2011 (http://tug.org/pipermail/tex-live/2011-June/029483.html).
I installed it and can confirm the fls file gets the right name and everything works.

I think this issue can be closed! Really appreciate all your time on it ... your Makefile
rocks :)

Reported by jgbailey on 2011-06-23 18:06:06

[shiblon]

Super!  I'm very glad that things are working.  Don't hesitate to file more bugs as
you see them.  Meanwhile, I'm shipping this version out.

Reported by shiblon on 2011-06-23 18:07:05

[shiblon]

Uploading version 2.2.1-alpha6 now.  Source tip is r29431244963c.

Reported by shiblon on 2011-06-23 18:12:30

• Status changed: Verified

[shiblon]

Found a new issue with LHS files - do you want me to report here or post a new one?

Or would you rather I waited a week? :)

Reported by jgbailey on 2011-06-23 19:39:57

[shiblon]

Here is fine.  What's up?

Reported by shiblon on 2011-06-23 19:50:49

• Status changed: Started

[shiblon]

Take two files, a.lhs and b.lhs. If a.lhs \inputs b (meaning b.tex),
you will get an error indicating b.tex is missing, however b.tex does
get created.

I think that what happens is b.tex is missing during at least one run
(since b.lhs produces b.tex), but it gets created when dependency
magic figures out that a.tex needs b.tex.

I can provide a complete tar/gz file if that isn't enough info.

Reported by jgbailey on 2011-06-23 20:32:59

[shiblon]

You have run headlong into the impossible problem.  I can fix this, but it will only
ever work for *one file at a time*.  In other words, if you do

\input{foo1}
\input{foo2}

And you have foo1.lhs and foo2.lhs, the makefile will correctly determine that foo1.tex
is missing and rebuild it (after I change it to do that), but will then complain that
foo2 is missing and try to rebuild again.

The reason this happens is latex can't possibly go on after a failed \input, since
the file it is reading is necessarily part of the source it wants to interpret.  So,
it's missing some of its input code and can't do anything about it.

This is the problem in general with trying to automatically build \input files, and
there's unfortunately no way around it (theoretically, not just practically).

I welcome your thoughts on this, though.

Reported by shiblon on 2011-06-23 20:39:19

[shiblon]

Doh, of course.

What about having a variable I can set in Variables.ini (or
Targets.ini) that will build all .tex files from .lhs before doing
anything else? I'm thinking along the lines of:

== Variables.ini ==
LHSFILES = foo.lhs bar.lhs baz.lhs

== Makefile ==

# LHS2TEXFILES should contain "foo.tex bar.tex baz.tex"
LHS2TEXFILES = $(addsuffix $LHSFILES, .tex)

# Generate .tex from .lhs using lhs2tex
$(LHS2TEXFILES): $(LHSFILES)
  lhs2tex ...

all: $(LHS2TEXFILES)

Reported by jgbailey on 2011-06-23 20:48:33

[shiblon]

Actually, if you're going to go to that trouble, you can just do this:

a.tex: foo.tex bar.tex baz.tex

And that should be sufficient to get them to build (it just adds dependencies, it doesn't
overwrite them).  Does that work for you?

Reported by shiblon on 2011-06-24 12:24:45

[shiblon]

That works very well! With the automatic dependency tracking I only
need to write down files that can't be guessed (i.e., those generated
from .lhs source).

I have been using alpha6 and it's working great.

Reported by jgbailey on 2011-06-24 18:03:00

[shiblon]

Hooray!  I'll mark this bug closed.  Feel free to reopen or submit a new one as you
run into problems.

Reported by shiblon on 2011-06-25 12:37:05

• Status changed: Verified