search

imatix / gsl

iMatix GSL code generator
http://www.imatix.com
GNU General Public License v3.0
539 stars 107 forks source link

Loading GSL code broken in recent commit #74

Open bheatwole opened 9 years ago

bheatwole commented 9 years ago

Sometime since commit b26040196cc595cb5a554f14ca9a824e0c887721 the following script breaks: 

.#  lookup_code_generator.gsl
.#
.#  Generates code files for lookups from the messaging model
.#
.gsl from "/project/doc/common_functions.gsl"
.#
.#

The command passed to gsl is:

gsl -script:lookup_code_generator /project/doc/model/message_model.xml

The output is:

2015/03/03 13:20:58: GSL/4.1a/github Copyright (c) 1996-2011 iMatix Corporation
2015/03/03 13:20:58: gsl/4 I: Processing /project/doc/model/message_model.xml...
2015/03/03 13:20:59: (lookup_code_generator.gsl 5) File:  not found

Note that there are two spaces after File:. Looking at the commit log, I suspect the name of the file is being freed before it can be used, but I haven't tried to debug it.

Broken with: 91b9c6dbd15a1e43e6b45d40c5f68d3f1ddc22c0 Last known good commit: b26040196cc595cb5a554f14ca9a824e0c887721

bheatwole commented 9 years ago

After further testing, here is the status for the recent commits:

91b9c6dbd15a1e43e6b45d40c5f68d3f1ddc22c0 - broken 89c6b0659a35ae05664073e59137de133ff189ed - okay 86872fd2dce323af928d78fcb9cc88e414722802 - broken b26040196cc595cb5a554f14ca9a824e0c887721 - okay

jschultz commented 9 years ago

I can't replicate this problem - this code works as expected for me under Linux. Can you tell me what system you are working on?

Possibly the same bug as #73

jschultz commented 9 years ago

I've just given up trying to build GSL under Windows - I have MSVS 2013, which apparently is not supported. Tried using Gnu make with Makefile, but still have problems with pcre. Might be worth an option to build gsl without pcre support?

Maybe someone else can try finding the source of this problem - presumably you'll find that file_open (in sflfile.c) is being called with a null filename. If you can get a backtrace at that point we might be able to work out what has happened.

bheatwole commented 9 years ago

I actually compiled using a Dockerfile. This is the version that works:

FROM ubuntu:14.04

RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
    build-essential \
    libpcre3-dev \
    git

RUN cd /tmp && \
    git clone git://github.com/imatix/gsl && \
    cd gsl/src && \
    git checkout b26040196cc595cb5a554f14ca9a824e0c887721 && \
    make && \
    make install
RUN rm -rf /tmp/gsl

CMD ["/usr/local/bin/gsl"]

Take out the git checkout line to test the latest commit

The OS is CoreOS 604.0.0 running on Vagrant via VMWare Workstation. The underlying OS is Windows. I've been using this setup successfully for some months, but I re-ran the Dockerfile recently and that's when it broke.

evoskuil commented 9 years ago

The VS2013 build is only not supported because the PCRE NuGet package isn't available for VS2013 (vc120). Given that PCRE sources are included with GSL it might be better to just eventually include them in the Windows build directly. I think I tried that at one point, but don't remember the problem. Otherwise I could probably package a version of pcre_vc120 in my own NuGet repo. I just didn't see much reason for doing this since most people use GSL as a program and not a library.

VS2012 express is available as a free download from MSFT, this is how I use it (and VS2010).

evoskuil commented 9 years ago

Also, this break isn't recent, as I was using the NuGet packaged version of GSL that I made some time ago. I did update it locally just to confirm that the problem still exists.

jschultz commented 9 years ago

@bheatwole I don't have a problem getting a version that works, just one that reproduces the bug(s). I use Debian as my native OS, and those commands are pretty much equivalent to what I do anyway.

@evoskuil OK so it does seem to be a Windows problem. I'll try the free VS2012 version. But don't let me stop anyone else from producing that backtrace.

evoskuil commented 9 years ago

@jschultz I figured as much, otherwise there would be more complaining around here :).

bheatwole commented 9 years ago

I tried it on a pure linux system and all commits work there, so it probably is a Windows issue.

jschultz commented 9 years ago

Indeed. And the fact that it concerns file operations makes me suspect that it is related to #73 which also appears to only affect Windows.

evoskuil commented 9 years ago

Hmm, my last two comments pertain to #73. Forgot which issue thread I was on.

evoskuil commented 9 years ago

@bheatwole You can probably work around by naming the script inside of the XML. I do that on Windows and it works fine:

<?xml version = "1.0" ?>
<generate script="lookup_code_generator.gsl">
...
</generate>

and

gsl -q /project/doc/model/message_model.xml

I assume this would bypass the filesystem issue on Windows because the GSL command line is processing the files, not the GSL script processor.

jschultz commented 9 years ago

OK I'll resist the temptation to complain about taking longer to install VS2012 than it would take to install an entire Linux OS, and focus on the more immediate problem of why it still won't build gsl. I just get this:

Error   1   error : This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them.  For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is ..\..\..\..\..\..\nuget\pcre.redist.8.33.0.1\build\native\pcre.redist.targets. C:\Users\Jonathan\src\gsl\builds\msvc\vs2012\gsl\gsl.vcxproj    376 5   gsl

even when I enable nuGet. Can anyone help me here?

evoskuil commented 9 years ago

Open the solution, open the NuGet Package Manager (right click on project) and when the GUI finally opens you should see a band at the top of the Window with a notice to enable download of the missing package.

Otherwise you could just run builds\msvc\buildbuildall.bat, which will pull missing packages.

evoskuil commented 9 years ago

See also: https://github.com/imatix/gsl/blob/master/builds/msvc/README.md or more specifically: https://github.com/imatix/gsl/blob/master/builds/msvc/NUGET.md.

jschultz commented 9 years ago

Hmmm none of this seems to work for me. I've tried right-clicking everywhere I can but can't find anything called 'NuGet Package Manager' (maybe you can send me a screenshot?). Tried the buildall.bat script but it also fails instantly and when I look in the logfiles I see similar error message as when I tried building with the GUI. I tried following the README files and met similar dead-ends (manual download of NuGet package manager produces a file that the OS has no more idea than I do what to do with). What am I missing?

evoskuil commented 9 years ago

nuget-pm

Use the NuGet Package Manager to download and update packages.

evoskuil commented 9 years ago

vs-extension-mgr

Use the VS Extension Manager to install the NuGet Package Manager :).

jschultz commented 9 years ago 
4/03/2015 11:49:22 AM - Microsoft VSIX Installer
4/03/2015 11:49:22 AM - -------------------------------------------
4/03/2015 11:49:22 AM - Initializing Install...
4/03/2015 11:49:22 AM - Extension Details...
4/03/2015 11:49:22 AM -     Identifier      : NuPackToolsVsix.Microsoft.67e54e40-0ae3-42c5-a949-fddf5739e7a5
4/03/2015 11:49:22 AM -     Name            : NuGet Package Manager
4/03/2015 11:49:22 AM -     Author          : Microsoft Corporation
4/03/2015 11:49:22 AM -     Version         : 2.8.50926.602
4/03/2015 11:49:22 AM -     Description     : A collection of tools to automate the process of downloading, installing, upgrading, configuring, and removing packages from a VS Project.
4/03/2015 11:49:22 AM -     Locale          : en-US
4/03/2015 11:49:22 AM -     MoreInfoURL     : http://docs.nuget.org/
4/03/2015 11:49:22 AM -     InstalledByMSI  : False
4/03/2015 11:49:22 AM -     SupportedFrameworkVersionRange : [4.0,4.5]
4/03/2015 11:49:22 AM - 
4/03/2015 11:49:22 AM -     SignedBy        : Microsoft Corporation
4/03/2015 11:49:22 AM -     Certificate Info : [Subject]
  CN=Microsoft Corporation, OU=MOPR, OU=OPC, O=Microsoft Corporation, L=Redmond, S=Washington, C=US

[Issuer]
  CN=Microsoft Code Signing PCA, O=Microsoft Corporation, L=Redmond, S=Washington, C=US

[Serial Number]
  33000000CC776171D4B17BA1280001000000CC

[Not Before]
  23/04/2014 3:39:04 AM

[Not After]
  23/07/2015 3:39:04 AM

[Thumbprint]
  9DD77A79F7124D8DB128A526833CF3760FCB7794

4/03/2015 11:49:22 AM -     Supported Products : 
4/03/2015 11:49:22 AM -         Microsoft.VisualStudio.Pro
4/03/2015 11:49:22 AM -             Version : [10.0]
4/03/2015 11:49:22 AM -         Microsoft.VisualStudio.Pro
4/03/2015 11:49:22 AM -             Version : [11.0]
4/03/2015 11:49:22 AM -         Microsoft.VisualStudio.IntegratedShell
4/03/2015 11:49:22 AM -             Version : [10.0]
4/03/2015 11:49:22 AM -         Microsoft.VisualStudio.IntegratedShell
4/03/2015 11:49:22 AM -             Version : [11.0]
4/03/2015 11:49:22 AM -         Microsoft.VisualStudio.VWDExpress
4/03/2015 11:49:22 AM -             Version : [10.0]
4/03/2015 11:49:22 AM -         Microsoft.VisualStudio.VWDExpress
4/03/2015 11:49:22 AM -             Version : [11.0]
4/03/2015 11:49:22 AM -         Microsoft.VisualStudio.VPDExpress
4/03/2015 11:49:22 AM -             Version : [11.0]
4/03/2015 11:49:22 AM -         Microsoft.VisualStudio.VSWinExpress
4/03/2015 11:49:22 AM -             Version : [11.0]
4/03/2015 11:49:22 AM -         Microsoft.VisualStudio.VSWinDesktopExpress
4/03/2015 11:49:22 AM -             Version : [11.0]
4/03/2015 11:49:22 AM - 
4/03/2015 11:49:22 AM -     References      : 
4/03/2015 11:49:22 AM - 
4/03/2015 11:49:22 AM - Searching for applicable products...
4/03/2015 11:49:22 AM - Failed to initialize settings for Isolated Shell C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\ManagementStudio\ssms.exe: The system cannot find the path specified. (Exception from HRESULT: 0x80070003)
4/03/2015 11:49:22 AM - Found installed product - Microsoft Visual Studio Professional 2012
4/03/2015 11:49:23 AM - System.IO.DirectoryNotFoundException: The system cannot find the path specified. (Exception from HRESULT: 0x80070003)
  at System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32 errorCode, IntPtr errorInfo)
  at Microsoft.VisualStudio.Settings.ExternalSettingsManager.GetScopePaths(String applicationPath, String suffixOrName, String vsVersion, Boolean isLogged, Boolean isForIsolatedApplication)
  at Microsoft.VisualStudio.Settings.ExternalSettingsManager.CreateForApplication(String applicationPath)
  at VSIXInstaller.App.GetExtensionManager(SupportedVSSKU sku)
  at VSIXInstaller.App.GetExtensionManagerForApplicableSKU(SupportedVSSKU supportedSKU, IInstallableExtension installableExtension, List`1 applicableSKUs)
  at VSIXInstaller.App.InitializeInstall()
  at System.Threading.Tasks.Task.InnerInvoke()
  at System.Threading.Tasks.Task.Execute()
evoskuil commented 9 years ago

Yeah... don't know about that. Did you invoke it from the VS2012 Extension Manager UI?

evoskuil commented 9 years ago

You should be able to manually download the PCRE NuGet packages and extract them to the proper location. This avoids the need for the NuGet Package Manager Extension.

http://www.nuget.org/packages/pcre http://www.nuget.org/packages/pcre.redist

To do this you first need to install NuGet (not the extension).

jschultz commented 9 years ago

When I follow that link I just end up somewhere I've already been, ie here: http://visualstudiogallery.msdn.microsoft.com/27077b70-9dad-4c64-adcf-c7cf6bc9970c

jschultz commented 9 years ago

Perhaps a problem with having both VS2012 and VS 2013 installed?

evoskuil commented 9 years ago

No, that's not a problem. I use 2010, 2012 and 2013 concurrently.

evoskuil commented 9 years ago

http://docs.nuget.org/consume/installing-nuget (See bullet 5 - command line tool).

jschultz commented 9 years ago

OK so 'Manage NuGet packages' appeared on the pull-down menu, once. When I select it, I get an error box saying (FFS I can't even copy the text from the error box - every time I use Windows I am reminded how much I hate it) 'Operation failed The project 'gsl' is unsupported'.

Do you use Teamviewer? My Windows box is sandboxed anyway so I'm happy to let you play with it.

evoskuil commented 9 years ago

Sure... https://pgp.mit.edu/pks/lookup?search=eric%40voskuil.org&op=index

evoskuil commented 9 years ago

Also, I'm going to update the repository expectations in the GLS solutions so they use a local NuGet repo. Will make it easier, and given it's a standalone app the shared repo isn't so important.

jschultz commented 9 years ago

OK I've some weird things that may explain the strange behaviour around the file functions under Windows. Line 204 of sfldir.c:

dir-> file_name = dir-> _dir_entry.cFileName;

is not doing what it looks like it should do. Putting a (char *) cast on the assignment fixes that one.

Next gets even weirder. At line 483 of ggfile.c,

directory-> path      = xstrcpy (NULL, dirst-> dir_name, "/", NULL);

the arguments to xstrcpy (which are iterated with va_start), get lost inside xstrcpy, ie srcptr shows a value of 0xffffffff.

I'm afraid I'm going to have to put this down now, as I too many other calls on my time.

evoskuil commented 9 years ago

I've picked it up. Working through issues. Found another case of the char * assignment problem, which I don't yet have an explanation for. Also an improper truncation of the directory_open() directory (not just Windows).

jschultz commented 9 years ago

If there is a typing problem can we tweak the compiler options so that it gets picked up during compilation?

evoskuil commented 9 years ago

I'm not sure, it appears that it may be incorrect presentation in the debugger. It's not happening consistently, and can't make sense of it otherwise. The segment stripping is a real issue though.

evoskuil commented 9 years ago

@jschultz The string assignment problem you mention above occurs in at least one other place. It still happens with a (char *) cast or a &path[0] dereference. Sometimes it's fine without either. Definately not just debugger presentation, it causes a hard failure in later code. It's a Heisenbug.

evoskuil commented 9 years ago

I wasn't able to fully explain the string assignment issue, but I was able to eliminate it by disabling the UNICODE symbol definition. This affects a large number of definitions and probably somewhere the code is calling a UNICODE (W) vs. ANSI (A) API or similarly using an type defined for UNICODE. It would be better to fix that, but at least we know that's an issue.