search

brushtechnology / fabricate

The better build tool. Finds dependencies automatically for any language.
132 stars 36 forks source link

Use other flavors of strace for specific OSs #3

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
There are other programs like strace for other OSs. A good source for them
is Wikipedia: http://en.wikipedia.org/wiki/Strace#Other_Tools

Here is the current list presented there:
    * Linux distributions after 2006 have SystemTap
    * Solaris has Truss and DTrace
    * FreeBSD provides the Truss command, ktrace and DTrace
    * Mac OS X provides ktrace (10.4 and earlier) and DTrace (from Solaris)
      in 10.5 and later.
    * MS Windows has a similar tool called StraceNT created by Pankaj Garg.

Maybe they can be used to increase portability while keeping a good
performance.

Original issue reported on code.google.com by llu...@gmail.com on 30 Jul 2009 at 3:30

GoogleCodeExporter commented 9 years ago 
Change from Defect to Enhancement.

Original comment by benh...@gmail.com on 3 Aug 2009 at 2:13

GoogleCodeExporter commented 9 years ago 
This is a patch that adds (very very very hacky) support for the dotnetfx 
command line tool 'tracker.exe' that does dependency tracking. (See issue #34.)

Original comment by jaros...@gmail.com on 9 Jan 2013 at 6:07

Attachments:

GoogleCodeExporter commented 9 years ago 
For reference, the "tup" build tool (which looks very cool) uses FUSE on Linux 
(user-level file systems) and DLL injection on Windows to achieve 
auto-dependency finding. http://gittup.org/tup/

Original comment by benh...@gmail.com on 28 Feb 2013 at 10:06

GoogleCodeExporter commented 9 years ago 
I have added a modified version of the Tracker.exe patch to the git repository 
on a branch called 'tracker'.

I did my testing with the version supplied with the .NET 4.5 SDK supplied in 
the Windows 8 SDK. http://msdn.microsoft.com/en-us/windows/desktop/hh852363.aspx

It seems to work well and tracks the input and output files for most 
operations. I tested using the few simple build scripts committed to the 
repository. I ran them in "git bash" to have access the commands being tested. 
This shows that the tracking works for all cases except rename/move operations. 

The sysmlink test also fails to build but that is because the 'ln' command in 
git bash does not behave as a true Linux one would. The ln command also only 
simulates symlinks in git bash, as you cannot create symlinks on windows XP at 
all, and on windows 7 or greater you must be administrator to create them. I 
had a brief attempt at tracking the mklink command on Windows 7, but it appears 
to be command line built-in so you can't get Tracker to analyse it. I came to 
the conclusion that symlinks are so difficult to use in Windows fabricate is 
unlikely to encounter them.

Does anyone have any ideas on how we could solve the failure of tracker to log 
the rename/move operations?

Does any one have any large windows builds that they would like to test this 
mod on? All you need is to have the Windows 8 SDK installed and update you 
fabricate. The SmartRunner will automatically switch you to a TrackerRunner 
instead of the AtimesRunner.

Original comment by simon.al...@gmail.com on 2 May 2013 at 2:05

GoogleCodeExporter commented 9 years ago 
This is awesome -- great work, jaroslov and Simon!

It worked fine for our medium-sized fabricate project, and produced a .deps 
file identical to that produced with AtimesRunner. Clean build with 
TrackerRunner was also 30% faster than with AtimesRunner, so that's good. :-) 
And of course safer, as it doesn't have all the caveats that AtimesRunner does, 
so (maybe with a bit of work) should be able to support parallel building and 
the like.

One thing I had to change has to do with quoting of command-line arguments to 
the Tracker.exe command line. We had one argument with quotes in it, like so:

    -DSETTINGS_DEFINITION="settings.h"

And to get this to work with Tracker.exe, I had to add just before the 
shell(TrackerRunner.__TRACKER...) line:

    args = [a.replace('"', '\\"') for a in args]

However, that's just a hack-fix, and I'm not at all sure it's a complete or 
correct fix. Will need a bit of further thought.

If anyone who doesn't have the SDK installed wants to try it, I'll try to 
attach a zip with Tracker.exe and its DLL. You'll need to unzip that somewhere 
and then change the __TRACKER path in fabricate.py.

In terms of distributing Tracker.exe with fabricate, it's (c) Microsoft, so 
that's probably not allowed. However, I think it's worth a shot. Can we contact 
an open-source friendly person in the right Microsoft department and ask if 
we're allowed to do this? I'd hate for folks to have to download and install a 
5GB SDK package just to get this working.

Original comment by benh...@gmail.com on 3 May 2013 at 7:27

Attachments:

GoogleCodeExporter commented 9 years ago 
Anyway, once the args quoting issue is addressed, I'm definitely +1 on merging 
this into the main fabricate.py -- obviously it'll only use Tracker.exe if 
available (assuming we can't ship it with fabricate), but that's okay.

Also, just a couple of coding style points:

* Rename __TRACKER to _tracker_exe. No need for dunders, and just in case 
anyone wants to subclass.
* Rename ParseTouched function to fabricate / PEP 8 style parse_touched.
* Have ParseTouched() build deps as set and .add() as it goes, then only 
convert to list() at end.
* Might be slightly more efficient to avoid reading the whole file as once by 
changing:

            lines = tlog.readlines()[1:]
            for name in lines: ...

to

            lines_iter = iter(tlog)
            lines_iter.next()  # skip first line
            for line in lines_iter: ...

* Error handling when you have compile errors or whatever is a little confusing 
(I don't know what it's like with strace). With AtimesRunner I get:

hub\main.cpp:23:1: error: expected initializer before 'AppClass'
fabricate: 'arm-none-eabi-g++' exited with status 1

However, with TrackerRunner it says:

hub\main.cpp:23:1: error: expected initializer before 'AppClass'
fabricate: 'Tracker.exe' exited with status 1

Wonder if we can change that to say the name of the called program as per 
AtimesRunner.

* Do alldeps and allouts need to be built as sets rather than lists too in case 
there are duplicates?

Original comment by benh...@gmail.com on 3 May 2013 at 7:43

GoogleCodeExporter commented 9 years ago 
Tracker.exe is not in the Windows SDK redist.txt file and may not be 
redistributed as a separate file.

*HOWEVER*, the Windows SDK is part of the standard development environment for 
any Windows developer using the MS tool chain. The following paths are the 
places to look for Tracker.exe:

  1. %WinDir%\Microsoft.NET\Framework64\v4.0.21006\Tracker.exe
  2. %ProgramFiles%\Microsoft SDKs\Windows\v7.0\bin\NETFX 4.0 Tools\x64\Tracker.exe
  3. %WinDir%\Microsoft.NET\Framework\v4.0.21006\Tracker.exe
  4. %ProgramFiles%\Microsoft SDKs\Windows\v7.0\bin\NETFX 4.0 Tools\Tracker.exe

Original comment by jaros...@gmail.com on 11 Jun 2013 at 5:41