search

mcaceresb / stata-gtools

Faster implementation of Stata's collapse, reshape, xtile, egen, isid, and more using C plugins
https://gtools.readthedocs.io
MIT License
182 stars 38 forks source link

gtools error occurred while loading _gtools_internal.ado (could not load plugin) #60

Open AnneShi opened 5 years ago

AnneShi commented 5 years ago

I have installed gtool for a while and the commands had worked well until 2 days ago, when our IT updated windows. The commands from gtool package no longer work. While running gcollapse, the program was interrupted with error message as:

(error occurred while loading _gtools_internal.ado) r(9998);

I am not able to find out what r(9998) means. I appreciate your help.

mcaceresb commented 5 years ago

Please give the output of

gtools
AnneShi commented 5 years ago

. . use S:\Forum\Ashi\Data\2017household\Stata\IPUMS\ps02_armenia, clear (Armenia person-household file with religion recoded)

. gcollapse hhsize [pw = perwt] (error occurred while loading _gtools_internal.ado) r(9998);

end of do-file

r(9998);

mcaceresb commented 5 years ago

Sorry, I meant the output of running

which gtools
gtools
AnneShi commented 5 years ago

. which gtools c:\ado\plus\g\gtools.ado ! version 1.5.3 04Apr2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com ! Program for managing the gtools package installation

. gtools Nothing to do. See help gtools or gtools, examples for usage. Version info: c:\ado\plus\g\gtools.ado ! version 1.5.3 04Apr2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com ! Program for managing the gtools package installation (error occurred while loading _gtools_internal.ado) (warning: gtools_plugin internal check failed)

mcaceresb commented 5 years ago

Can you tell me your OS and Stata versions?

AnneShi commented 5 years ago

Windows 8; Stata/MP 14.2 or Stata/SE 15.1

mcaceresb commented 5 years ago

Are you on a server or just a regular Windows 8 machine?

AnneShi commented 5 years ago

on a network

AnneShi commented 5 years ago

All commands from gtools worked till two days ago, when the IT updated the windows. I talked to our IT specialists and they are trying to figure out as well

mcaceresb commented 5 years ago

(Not 100% what you mean by "network").

Anyway, I wonder if it is related to this bug. It's a bit of a different situation since that was back when I still put the hashing library in a DLL (and I thought it was fixed).

Do you happen to have cygwin installed on your windows system? No worries if you don't.

AnneShi commented 5 years ago

Server, then. Yes I read the discussions and tried the suggested solutions therein. None works. No I do not have cygwin.

AnneShi commented 5 years ago

Out IT people said they might try to scale the windows security updates they installed a couple of days age -- to see whether those are the cause. Are the gtools plugins security sensitive?

mcaceresb commented 5 years ago

I do recall somebody else reporting that gtools stopped working after a Windows Server security update.

All gtools commands use temporary files, but none require any special permissions to run. I am really not sure why a security update would break gtools. Maybe it's a versioning issue? (i.e. I compile using a set of libraries that get updated and that breaks gtools). I don't have access to Windows server, however, so it's a bit difficult to debug.

When you say "none work" for the solutions in that thread, what did you try? I was going to try and compile a very basic plugin to see if whether the issue was gtools or Stata.

AnneShi commented 5 years ago

I un-installed and reinstalled gtools. I didn't do spooky.

mcaceresb commented 5 years ago

Can you download the zip file here and unzip to an empty directory, then run do test.do? It's for an unrelated issue but it will test what I want to double-check.

AnneShi commented 5 years ago

. do "C:\Users\ashi.PEW\Downloads\statatest\plugin\test.do"

. clear all

. program test1, plugin using(test1.plugin) file test1.plugin not found r(601);

mcaceresb commented 5 years ago

Sorry, you need to run it from that directory.

cd C:\Users\ashi.PEW\Downloads\statatest\plugin\
do test.do
AnneShi commented 5 years ago

. cd C:\Users\ashi.PEW\Downloads\statatest\plugin C:\Users\ashi.PEW\Downloads\statatest\plugin

. do test

. clear all

. program test1, plugin using(test1.plugin) r(9998);

end of do-file r(9998);

end of do-file

r(9998);

.

mcaceresb commented 5 years ago

Ok, that is interesting, because that is just a plain "Hello World" basic plugin example. So I think the issue is not gtools per se, but rather however I am compiling plugins is getting blocked... Let me check if I can update my Windows 10/Cygwin versions and re-compile to see if that helps. Will get to this over the weekend. Sorry I can't be more help right now.

AnneShi commented 5 years ago

Thank you so much! I need to finish something too. I have just reverted all gcollapse back to collapse in the codes I am running. It is painful, as I have a lot of codes using commands from the gtools package. I really like them.

mcaceresb commented 5 years ago

Try gtools, upgrade(develop), restart stata, and then try running gtools again?

AnneShi commented 5 years ago

Hi Mauricio,

Thank you.

I am on vacation these two days and will be back to work on Wednesday. I will test it when I am back.

Best,

Anne

From: Mauricio Caceres Bravo notifications@github.com Sent: Monday, May 6, 2019 8:23:41 AM To: mcaceresb/stata-gtools Cc: Anne Shi; Author Subject: Re: [mcaceresb/stata-gtools] gtool error (#60)

Try gtools, upgrade(develop), restart stata, and then try running gtools again?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmcaceresb%2Fstata-gtools%2Fissues%2F60%23issuecomment-489601446&data=02%7C01%7Cashi%40pewresearch.org%7C45fca3cc21214bfe2d0908d6d21dad13%7Cc96aea07e3234eaebc92c63cb98ba860%7C0%7C0%7C636927422228802868&sdata=uFzaqY93J9iG82GVd9oxuIBTH%2BF3ejq67BKDujmGmcs%3D&reserved=0, or mute the threadhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABVTJT2AGJACRIK3EGOK4STPUAPM3ANCNFSM4HKVUAEA&data=02%7C01%7Cashi%40pewresearch.org%7C45fca3cc21214bfe2d0908d6d21dad13%7Cc96aea07e3234eaebc92c63cb98ba860%7C0%7C0%7C636927422228802868&sdata=9qwkkyZeAGI80zMwjGeBDxRn2PSGY9enr7K5WsFqbYA%3D&reserved=0.

AnneShi commented 5 years ago

I tried gtools, upgrade(develop) and got this error message:

. gtools, upgrade(develop) option upgrade() incorrectly specified r(198);

mcaceresb commented 5 years ago

My bad

gtools, upgrade branch(develop)
AnneShi commented 5 years ago

The above codes to update gtools run successfully, but after I restarted the Stata and run the script using gcollapse, I got the same error message:

image

binyamink commented 5 years ago

Hi, first of all thanks a lot for the package. I have a similar issue as described above. In particular, I get the "Could not load plugin" error when trying to run the gtools commands. I have the same issue when running your "hello world" test. I have tried to replace gtools_windows_v2.plugin and gtools_windows_v3.plugin with the versions you've uploaded above.

Are there any more updated solutions than the ones found in this thread? Thanks a lot in any case!

mcaceresb commented 5 years ago

What's your OS?

binyamink commented 5 years ago

Sorry - windows 10 64-bit

mcaceresb commented 5 years ago

Is it a server?

binyamink commented 5 years ago

nope, a private machine

mcaceresb commented 5 years ago

Stata version?

binyamink commented 5 years ago

14

mcaceresb commented 5 years ago

14 or 14.1+?

binyamink commented 5 years ago

14.0

mcaceresb commented 5 years ago

And output from gtools and which gtools?

binyamink commented 5 years ago

gtools Nothing to do. See help gtools or gtools, examples for usage. Version info: C:\ado\personal\gtools\gtools.ado ! version 1.5.1 24Mar2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com ! Program for managing the gtools package installation Could not load plugin: C:\ado\personal\gtools\gtools_windows_v2.plugin (error occurred while loading _gtools_internal.ado) (warning: gtools_plugin internal check failed)

. which gtools C:\ado\personal\gtools\gtools.ado ! version 1.5.1 24Mar2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com ! Program for managing the gtools package installation

mcaceresb commented 5 years ago

Ok. I'll have to test this on a physical windows machine, I think. I'll keep you posted.

binyamink commented 5 years ago

Thanks a lot!

mcaceresb commented 5 years ago

@binyamink Can you try running the files here? Unzip into a folder and run test.do and let me know what happens.

binyamink commented 5 years ago

unfortunately same issue:

. program test1, plugin using(test1.plugin) Could not load plugin: .\test1.plugin r(9999);

drkrynstrng commented 5 years ago

I'm having the same issue on macOS Mojave 10.14.6 (64-bit) with Stata/SE 14.2. It actually worked fine when I first installed gtools, but the next day after rebooting I'm getting similar errors discussed here. See output below:

. which gtools
1_code/Stata/ado/plus/g/gtools.ado
*! version 1.6.2 25Aug2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com
*! Program for managing the gtools package installation

. gtools
Nothing to do. See help gtools or gtools, examples for usage. Version info:
1_code/Stata/ado/plus/g/gtools.ado
*! version 1.6.2 25Aug2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com
*! Program for managing the gtools package installation
class GtoolsResults undefined
(282 lines skipped)
(error occurred while loading _gtools_internal.ado)
(warning: gtools_plugin internal check failed)

and

. do test.do

. clear all

. program test1, plugin using(test1.plugin)
dlopen(./test1.plugin, 1): no suitable image found.  Did find:
                ./test1.plugin: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 
> 0x01 0x01 0x00
                /Downloads/plugin/test1.plugin: unknown file type, first eight by
> tes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00
Could not load plugin: ./test1.plugin
r(9999);

end of do-file

I tried re-installing, including from the develop branch, and it works if I stay in the same Stata session. However, if I clear and exit Stata and then start a new Stata session, it no longer works and produces the errors above. I also checked the file permissions and all of them are set to allow read, write, and execute. The same install and code works fine on a Linux server though, without these issues.

Based on the class GtoolsResults undefined output, I removed the GtoolsResults class definitions from the _gtools_internal.ado file and then everything runs fine even after restarting. So perhaps there is an error somewhere in those lines of code, which then prevents the plugin from being loaded.

mcaceresb commented 5 years ago

@binyamink Maybe run mata: mata mlib index and try again? If that doesn't work, what is the output of

which lgtools.mlib
findfile lgtools.mlib
mata: mata mlib query
drkrynstrng commented 5 years ago

Yes, that works. So I will include that line in do-files. I do this for ftools as well, by including the ftools command in do-files to create the ftools Mata library. Just curious, is there a reason these are not automatically indexed? In the do-files I'm working with I change the sysdir for ado-files, so perhaps that is why? I tested it and this appears to be the case. Thanks!

mcaceresb commented 5 years ago

@drkrynstrng I think you only need to run mata: mata mlib index once after you install packages, but I suppose I can't hurt to run it always?

Not sure why it happens. I've never had this issue but I have had this complaint before...

lffontes commented 4 years ago

I'm having the same issue on a Windows 10 pro (64-bits, private machine) with Stata/SE 14.0. See some outputs below.

. gtools Nothing to do. See help gtools or gtools, examples for usage. Version info: c:\ado\plus\g\gtools.ado ! version 1.6.2 25Aug2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com ! Program for managing the gtools package installation Could not load plugin: c:\ado\plus\g\gtools_windows_v2.plugin (error occurred while loading _gtools_internal.ado) (warning: gtools_plugin internal check failed)

. which gtools c:\ado\plus\g\gtools.ado ! version 1.6.2 25Aug2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com ! Program for managing the gtools package installation

. which lgtools.mlib c:\ado\plus\l\lgtools.mlib

. . findfile lgtools.mlib c:\ado\plus/l/lgtools.mlib

. . mata: mata mlib query .mlib libraries to be searched are lmatabase;lmataado;lmatafc;lmatagsem;lmatamcmc;lmataopt;lmatapath;lmatapostest;lmatapss;lmatasem;lftools;lgtools;l

moremata;lmoremata10;lmoremata11;lmoremata14;lsynth_mata_subr

mcaceresb commented 4 years ago

@lffontes does the ssc version fail as well? (i.e. if you install from ssc w/o updating, do you get the same issue?)

lffontes commented 4 years ago

@lffontes does the ssc version fail as well? (i.e. if you install from ssc w/o updating, do you get the same issue?)

Yes.

. gtools Nothing to do. See help gtools or gtools, examples for usage. Version info: c:\ado\plus\g\gtools.ado ! version 1.5.1 24Mar2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com ! Program for managing the gtools package installation Could not load plugin: c:\ado\plus\g\gtools_windows_v2.plugin (error occurred while loading _gtools_internal.ado) (warning: gtools_plugin internal check failed)

lffontes commented 4 years ago

Any idea on what is going on?

mcaceresb commented 4 years ago

Unfortunately I don't, I'm very sorry. My guess is that the version of windows I compile under (MinGW on regular Windows 10 but on a virtual machine) has some library that conflicts with Windows 10 Pro somehow.

I would hope that if the plugin were to compile on a Windows 10 pro machine it might work, but I don't have win 10 pro to test out my theory.

ast-code commented 4 years ago

I had the same error message. Found out that gtools works fine if I use the 64bit Stata version (tested for STATA/SE 15.1, Windows 10 pro build 17763).