Cross Compilation on Windows/Linux

[shiblon]

Originally reported on Google Code with ID 71

What is really cool is if the same makefile can be used to cross-compile in
linux/unix and windows.

This can be done on the fly with simple makefiles by using 
http://gnuwin32.sourceforge.net/packages/coreutils.htm

I have tried yours with pdflatex, however this seems far more complicated,
but you may want to support it.

Reported by akis.karnouskos on 2010-03-12 20:25:54

[shiblon]

So, my first question is whether this does *not* work on Windows already?  If it doesn't,

I'd be interested in any ways that it fails.  I suspect that it would work just fine,
provided 
that you don't have spaces in your filenames.

Reported by shiblon on 2010-03-12 21:21:45

[shiblon]

OK, it doesn't fully. At least not in a standalone manner by copying the binaries of
the gnuwin32. Maybe with cygwin it works, but the idea would be to have everything
in
1 directory and this should cross-compile by requiring only a latex installation.

info: Win7, texlive

make clean

process_begin: CreateProcess(NULL, which echo, ...) failed.
process_begin: CreateProcess(NULL, tput setaf 0, ...) failed.
process_begin: CreateProcess(NULL, tput setaf 1, ...) failed.
process_begin: CreateProcess(NULL, tput setaf 2, ...) failed.
process_begin: CreateProcess(NULL, tput setaf 3, ...) failed.
process_begin: CreateProcess(NULL, tput setaf 4, ...) failed.
process_begin: CreateProcess(NULL, tput setaf 5, ...) failed.
process_begin: CreateProcess(NULL, tput setaf 6, ...) failed.
process_begin: CreateProcess(NULL, tput setaf 7, ...) failed.
process_begin: CreateProcess(NULL, tput bold, ...) failed.
process_begin: CreateProcess(NULL, tput smul, ...) failed.
process_begin: CreateProcess(NULL, tput sgr0, ...) failed.
process_begin: CreateProcess(NULL, which gnuplot, ...) failed.

make
....

kpathsea: Running mktextex myfile.out

The command name is C:\texlive\2009\bin\win32\mktextex
'!g'' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the drive specified.
'/p'' is not recognized as an internal or external command,
operable program or batch file.
make: *** [myfile.d] Error 255

Reported by akis.karnouskos on 2010-03-13 16:19:15

[shiblon]

It already doesn't run with only a latex installation.  It requires several utilities
that 
are present on any *nix system (gnu utilities should do it, though), including

sed
which
tput

and several others.  Run

make _check_programs

to see the list and which ones you have (or don't).  It may fail on all of them, because

it uses "which" to do its work.  But the list is pretty prominent in the Makefile if

you're trying to see what is needed.

Reported by shiblon on 2010-03-13 21:13:08

[shiblon]

Its clear that it requires some utils.
sed and which could be found.  I could not locate a tput file though.
Even then, for very simple compilation with pdflatex the Makefile seems to have a
problem with the script i.e. 

'!g'' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the drive specified.
'/p'' is not recognized as an internal or external command,
operable program or batch file.

Reported by akis.karnouskos on 2010-03-13 22:36:29

[shiblon]

Yeah, that's indicative of a problem parsing the sed commands.  !g and /p are both at

the end of several sed commands.

tput shouldn't be a big deal - I can probably work around that easily enough.  What,
in 
that list of utilities in the Makefile, is not present on your system?

Reported by shiblon on 2010-03-14 17:32:30

[shiblon]

Well several utils are not there (but are also not needed for the average user that
requires simple pdflatex compilations without conversions etc).

I think the biggest bottlenecks are
- tput (since this is not generally available and has even been removed in the latest
cygwin version)
- the parameter parsing which might be different in windows such as the sed.

If somehow the system identification (win/Linux) could be done and these are
adjusted, it has good potential.

This would be really good since this script works already excellently in Linux with
pdflatex. For windows users one could pack a couple of exe files such as make/sed...
and have everything in one location.

cheers

Reported by akis.karnouskos on 2010-03-14 19:20:34

[shiblon]

For the tput issue, can you try running it with make NO_COLOR=1 and let me know if 
that fixes at least some things?

I'm also curious about the sed invocation.  There is a SHELL_DEBUG parameter you can

pass that will cause make to output everything it tries to run on the shell.  I'm guessing

that the biggest issue here is that the makefile absolutely requires you to run stuff
in 
"sh" or "ash" or "bash", etc.  Some form of Bourne shell is a requirement (this won't
work 
in a normal Windows command window).

Reported by shiblon on 2010-03-15 15:53:01

[shiblon]

r8c654f3d1870 makes tput optional.

Reported by shiblon on 2010-03-15 16:00:10

[shiblon]

Great tput dependencies are out.
The remaining errors include parameter passing (Testing with 2.2.0-beta5):

The command name is C:\texlive\2009\bin\win32\mktextex
'/p'' is not recognized as an internal or external command,
operable program or batch file.
'-' is not recognized as an internal or external command,
operable program or batch file.
'/'' is not recognized as an internal or external command,
operable program or batch file.
'!g'' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the drive specified.
'/p'' is not recognized as an internal or external command,
operable program or batch file.
make: *** [test.d] Error 255

Reported by akis.karnouskos on 2010-03-17 14:56:34

[shiblon]

There is some way to quote parameters, but I'm not sure what it is in Windows.  I don't

have access to a Windows machine to test it on, so if you can find out how to run,
e.g.,

> echo "bar stuff" | sed -e 's/foo/bar/'
foo stuff

That would be great.

Reported by shiblon on 2010-03-17 17:33:35

[shiblon]

> echo "foo stuff" | sed -e 's/foo/bar/'
sed: -e expression #1, char 1: unknown command: `''

> echo "foo stuff" | sed -e "s/foo/bar/"
"bar stuff"

>sed.exe
Usage: sed.exe [OPTION]... {script-only-if-no-other-script} [input-file]...

  -n, --quiet, --silent
                 suppress automatic printing of pattern space
  -e script, --expression=script
                 add the script to the commands to be executed
  -f script-file, --file=script-file
                 add the contents of script-file to the commands to be executed
  -i[SUFFIX], --in-place[=SUFFIX]
                 edit files in place (makes backup if extension supplied)
  -b, --binary
                 open files in binary mode (CR+LFs are not processed specially)
  -c, --copy
                 use copy instead of rename when shuffling files in -i mode
                 (avoids change of input file ownership)
  -l N, --line-length=N
                 specify the desired line-wrap length for the `l' command
  --posix
                 disable all GNU extensions.
  -r, --regexp-extended
                 use extended regular expressions in the script.
  -s, --separate
                 consider files as separate rather than as a single continuous
                 long stream.
  -u, --unbuffered
                 load minimal amounts of data from the input files and flush
                 the output buffers more often
      --help     display this help and exit
      --version  output version information and exit

If no -e, --expression, -f, or --file option is given, then the first
non-option argument is taken as the sed script to interpret.  All
remaining arguments are names of input files; if no input files are
specified, then the standard input is read.

GNU sed home page: <http://www.gnu.org/software/sed/>.
General help using GNU software: <http://www.gnu.org/gethelp/>.

Reported by akis.karnouskos on 2010-03-17 21:52:31

[shiblon]

Hmmm... I noticed that I didnt put the path, which ended up executing the default
windows echo and not the GNU one...

>.\echo "foo stuff" | sed -e 's/foo/bar/'
sed: -e expression #1, char 1: unknown command: `''

>.\echo "foo stuff" | sed -e "s/foo/bar/"
bar stuff

Let me try this with absolute paths and come back to you...

Reported by akis.karnouskos on 2010-03-17 22:04:17

[shiblon]

Finally I tried different configurations.
The script works in Cygwin (and compiles several docs). So the cross-compilation is
generally not a problem (if you have installed Cygwin). This is a bit different, as
my goal was to have everything needed in a local (to the document) bin directory and
require only a valid external latex installation.
As you pointed out, it needs to run via a shell; the default windows command-line
even with the correct paths to the utils still complains (as in
http://code.google.com/p/latex-makefile/issues/detail?id=71#c11). Changing the ' to
"
might help, but there are several places where this could cause a problem.

Reported by akis.karnouskos on 2010-03-17 23:39:14

[shiblon]

Yeah, I see that.  It looks like switching ' to " would be helpful, but there are some
crazy 
escaping things that depend on shell semenatics in the makefile, and I'm not at all

confident that we can get it working properly.

I think unless we have a really big insight into how to fix this, I'm happy with Cygwin

being required :)

Reported by shiblon on 2010-03-18 14:16:26

[shiblon]

Just FYI it looks like using MinGW (www.mingw.org) makes this possible.

I didnt actually install MinGW but since I was installing msysgit (code.google.com/p/msysgit)
I which has a mingw installation, I just copied all latex files and the Makefile to
that directory and it compiled on the 2nd run and created the final PDF. On the 1st
run it complained about a call as shown below (Makefile 2.2.0-rc8).

makefile:2445: x.d: No such file or directory
= x.tex --> x.d x.pdf.1st.make (0-1) =
sed: can't read x.fls: No such file or directory
make: *** No rule to make target `$(call)', needed by `x.d'.  Stop.

So it looks like the problems areas in the discussion above dont seem to exist with
this installation. I wont test/use it further, but note it here just in case someone
else discovers this thread (or if you decide to go cross-platform) ...

Reported by akis.karnouskos on 2011-01-16 20:44:13

[shiblon]

Thanks for the comment.

I'm curious as to why the .fls file wasn't there.  When running pdflatex with the --recorder
option, that file should show up.  Occasionally I get hard-to-debug problems reported
on Windows, and it appears that it can sometimes have to do with the timing of files
showing up in the file system.  If that's the case, we're in trouble.

So, if you don't mind...  Would you do a make clean, then do make *once*, then tell
me if the .fls file is there (and contains stuff)?  If it is there, we have a timing
issue.  If not, there is probably another issue.

Reported by shiblon on 2011-01-20 16:58:10

[shiblon]

A .fls file is generated but named pdflatexXXXX.fls where XXXX is a four-digit number.
My guess would be that somewhere it doesnt get renamed correctly...

Reported by akis.karnouskos on 2011-01-27 17:57:30

[shiblon]

Looking at your previous post, it looks like the name of the .fls file is the least
of our problems.  Note that it complains about "no rule to make target `$(call)'",
which should *never* happen.  $(call ...) should always be expanded, and somehow here
it is not.

I think you may be dealing with a non-GNU make program.  What version of make are you
running?

Reported by shiblon on 2011-01-27 20:28:47

[shiblon]

Hmm, in the meantime there has been an update and a rebuild (in libraries). This doesn't
appear anymore. FYI

$ make --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i686-pc-msys

Reported by akis.karnouskos on 2011-01-29 21:01:56

[shiblon]

Ah, interesting.  Well, thanks for the update.  I'm mystified :)

Reported by shiblon on 2011-01-30 01:49:37