VFPX / GoFish

GoFish is an advanced code search tool for fast searching and replacing of Visual FoxPro source code.
8 stars 7 forks source link

Greater searching speed. #226

Closed myearwood1 closed 4 months ago

myearwood1 commented 5 months ago

📝 Provide a description of the new feature

If GoFish would create text files of all parts of an existing application and recreate those that have changed since last search, all GoFish would have to do is search the text. When the text files are created, change their date time to match the original files'. Then it would be a quick thing to check for newer originals and create the text versions.

What is the expected behavior of the proposed feature? What is the scenario this would be used?

I have almost 20,000 files. If GoFish created a text-only version of my app for searching, the number of files would be reduced and search speed would improve. Keeping the text-only version in sync with my app as changes are made is a minor thing.


If you'd like to see this feature implemented, add a 👍 reaction to this post.

Jimrnelson commented 5 months ago

If GoFish created a text-only version of my app for searching, the number of files would be reduced and search speed would improve

Yes, if GF could search only text files, its search speed would definitely improve considerably.

However, GF is a search engine; that is to say, it is used to search your code files. It would be wildly inappropriate for it to try to manage your code files in any way.

Keeping the text-only version in sync with my app as changes are made is a minor thing.

Assuming you are correct with this, you should be able to perform this syncing yourself before calling GF.

myearwood1 commented 5 months ago

Foxpro is a search engine. I suggested you build an index from the source data, not manage the data. I see now that power corrupts. You called my comparison weak elsewhere. You don't dismiss things out of hand?

Jimrnelson commented 5 months ago

Mike --

First, note that since this issue is not closed, it was not dismissed out of hand.

You suggested that GF build a text only version of your app and then it keep the text-only version in sync with changes in your app.

I suggested that you can handle this yourself outside of GF.

That would allow GF to focus entirely on what it does best, which is to search, leaving the maintenance of what to search up to you.

I see no advantage at all of trying to incorporate the creation index of your source data into GF. You can handle that. GF can handle the search of whatever code base you provide it with.

myearwood1 commented 5 months ago

Sounds like dismissal out of hand to me. Dictators will be dictators.

On Thu, May 2, 2024 at 10:32 AM Jimrnelson @.***> wrote:

Mike --

First, note that since this issue is not closed, it was not dismissed out of hand.

You suggested that GF build a text only version of your app and then it keep the text-only version in sync with changes in your app.

I suggested that you can handle this yourself outside of GF.

That would allow GF to focus entirely on what it does best, which is to search, leaving the maintenance of what to search up to you.

I see no advantage at all of trying to incorporate the creation index of your source data into GF. You can handle that. GF can handle the search of whatever code base you provide it with.

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2090642662, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL36WYDORERVWPML7YP2UDZAJFALAVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJQGY2DENRWGI . You are receiving this because you authored the thread.Message ID: @.***>

Jimrnelson commented 5 months ago

Mike --

You haven't presented any reason for this to be incorporated into GF.

I'm quite sure it would indeed speed up searches considerably, but this could be accomplished without modifying GF at all.

In fact, it would seems to me that a distinct tool to manage this text-only version of your code space would make a lot of sense. I could see that as being a separate project altogether.

myearwood1 commented 5 months ago

I did indeed present a reason, which you obviously pay no mind to. GoFish searching on my project takes way too long. The fact that most "programmers" have tiny projects disqualifies their input on performance. As they also believe any optimization is evil, they are further disqualified. As it stands, I do not want to further clutter my development environment with more files. Anything that speeds up gofish is good, but avoiding looking for them is why it is as it is.

I will whip up something for myself and NOT try to give it to others for their benefit.

On Thu, May 2, 2024 at 11:14 AM Jimrnelson @.***> wrote:

Mike --

You haven't presented any reason for this to be incorporated into GF.

I'm quite sure it would indeed speed up searches considerably, but this could be accomplished without modifying GF at all.

In fact, it would seems to me that a distinct tool to manage this text-only version of your code space would make a lot of sense. I could see that as being a separate project altogether.

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2090781922, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL36WZ3BLIM6VPDCG2LAMLZAJJ6FAVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJQG44DCOJSGI . You are receiving this because you authored the thread.Message ID: @.***>

myearwood1 commented 5 months ago

I used FoxBinToPrg to generate a text-only version of my app. Instead of being 19,000+ files, it became 6837 files. Windows Powershell and the DOS findstr - which supports regex and can list all line numbers of the string found in the files - completely scans the code in 5.6 seconds.

$startTime = Get-Date findstr /s /n /c:"ORDER HAS BEEN SELECTED FOR BILLING" . $endTime = Get-Date $elapsedTime = $endTime - $startTime Write-Host "Elapsed time: $($elapsedTime.TotalSeconds) seconds"

GoFish takes 40 seconds to scan those same text files. That alone shows there is room for improvement. A regular GoFish of my project takes 51.8 seconds.

GoFish's abilities, such as navigating to the searched location in the files is well worth having, as is much more speed.

myearwood1 commented 5 months ago

We use SourceSafe here. It creates .??a files which can be used for diffing changes. The above findstr command in DOS takes almost 6 seconds. Can you use FindStr to produce a text file of found locations and use that text file to build up your display of matches? FindStr already supports RegEx and by using it to do the searches, GoFish would become significantly faster.

myearwood1 commented 5 months ago

FindStr /? will show you all the possibilities. You can make findstr do much of GoFish's work for you.

myearwood1 commented 5 months ago

This raises the Windows API PathFileExists function to FoxPro's language.

The following generates the missing text versions of the binary files using scctextx.prg MYGENERATETEXT.PRG use in (select(PROJECTNAME)) select 0 use PROJECTNAME again alias TheProject scan for not "PROJECTCLASSNAME"$upper(name) AND inlist(upper(justext(name)),"VCX","SCX","MNX","FRX","LBX") lcExt = chrtran(upper(justext(name)),"X","A") lcFile = strtran(upper(name),justext(upper(name)),m.lcExt) IF PATHFILEEXISTS(m.lcFile) = 0 STRTOFILE(m.lcFile,"mygenerated.txt",.T.)+CHR(13) DO (home()+"SccTextX") with upper(PROJECT.name) Endif endscan MODI FILE mygenerated.txt

This produces a list of all of the above filenames containing the search text. It takes about 5 seconds. lcCommand = 'cmd /c findstr /s /i /m /c:"ORDER HAS BEEN SELECTED FOR BILLING" c:\sourcefolder*.??a > c:\sourcefolder\output_file.txt' ShellExecute(0, "", "cmd.exe", m.lcCommand, "", 0)

I'll add a bit of code to generate the text versions only if the binaries are newer.

myearwood1 commented 5 months ago

Professor Donald Knuth is often quoted for saying Premature Optimization is wrong. If you read the original work, he means that penny-wise and pound-foolish programmers should not optimize, because they screw it up. He means that masterful programmers who write great code will only need to optimize 3% of the time. He means that programmers in general stupidly refuse even a 12% improvement when other more formal industries would take even 12% gladly. I show you a method to get 1000% improvement and all I hear are crickets and an excuse about managing the code of "programmers". If people are using source control, you already have this option in front of you. If they are not using source control you can generate the text files as an index, like Foxpro's own index command does and get stupendous speed. That is professional programming.

Jimrnelson commented 5 months ago

What's with the crickets? This is the middle of my work week.

If I have a chance, I will look into your suggestion this weekend.

myearwood1 commented 5 months ago

It's the middle of my work week too. I have repeatedly offered to assist in this project despite the gate-keeper mentality, lack of appreciation, rudeness and inability of the FoxPro community and been slammed for trying.

myearwood1 commented 5 months ago

By using FindStr you gain an additional ability. Instead of the filter feature, from Search A, send the resulting filenames to a file. Then do Search B on the results of Search A from said file, and so on, with the ability to go Back to Search A, refine search B and try again. I see no reason to save search results when GoFish can be so much more capable.

myearwood1 commented 5 months ago

Forget FindStr. It has limited regex abilities. Download and install cygwin's utilities at this link:

https://www.cygwin.com/setup-x86_64.exe

That will give us access to grep.exe.

The following command options mean -r recurse folders, -l list file name only, -i case insensitive, -P use Perl regex

lcCommand = 'cmd /c c:\cygwin64\bin\grep -r -l -i -P --include="*.??a" "\b(REPORT|REPO)\s+FORM\s\b" sourcefolder > destinationfolder\output_file.txt' ShellExecute(0, "", "cmd.exe", m.lcCommand, "", 0)

It takes about 9 seconds instead of 5, but does a better job of regex.

myearwood1 commented 5 months ago

FROM here: https://cygwin.com/faq.html

Is it free software?

  | Yes. Parts are GNU software (gcc, gas, ld, etc...), parts are covered by the standard X11 license, some of it is public domain, some of it was written by Red Hat (or the former Cygnus Solutions) and placed under the GPL. None of it is shareware. You don't have to pay anyone to use it but you should be sure to read the copyright section of the FAQ for more information on how the GNU General Public License may affect your use of these tools.Note that when we say "free" we mean freedom, not price. The goal of such freedom is that the people who use a given piece of software should be able to change it to fit their needs, learn from it, share it with their friends, etc. The GPL or LGPL licenses allows you those freedoms, so it is free software.

myearwood1 commented 5 months ago

Perhaps you can explain why I am prevented from doing this kind of modification? I gave to this community for years despite being harrassed by certain individuals who were not ostracized for their behavior. I regularly make massive improvements in the code of those who consider themselves programmers.

myearwood1 commented 5 months ago

I found out how to tell grep to exclude lines starting with asterisk, in other words, exclude comments. This takes 14 seconds rather than the 6000+ seconds I used to get. C:\cygwin64\bin\grep.exe -r -l -i -P --include='*.??a' '^(?!\s*\*).*THERE\s+IS\s+NO\s+SUCH\s' c:\myfolder image

Jimrnelson commented 5 months ago

Instead of the filter feature, from Search A, send the resulting filenames to a file. Then do Search B on the results of Search A from said file, and so on, with the ability to go Back to Search A, refine search B and try again.

I have taken this suggestion and implemented the following in today's release, GF 7.0.31:

There is a new entry "Results" in the Scope dropdown. When selected, the next search will be performed on the current results set -- that is, the files currently shown in the grid and Treeview (same list in both places), instead of those in a folder or project. This provides a fast and progressive sub-search of the current result set.

image

Jimrnelson commented 5 months ago

I have added a feature to GF in the current release (7.0.31) so that you can integrate your optimization techniques into GF and begin using them immediately.

The feature is known as "Custom UDF" and is explained in this FAQ

image

myearwood1 commented 5 months ago

That seems a reasonable compromise. I find the lack of thanks insulting, rude and abusive.

myearwood1 commented 5 months ago

Further to this. Since GoFish only runs in 9, you could use the Grep to output the line numbers of detected matches. You could then build your tree view on that information, and when the user wants to go to that location, you can use EDITSOURCE to navigate to the matching text in the form or whatever.

myearwood1 commented 5 months ago

I am sorry I find it hard to be thankful when this community is so full of fake know-it-alls and abusers that are allowed to be abusive FOR years. I appreciate what you did. I can use it already. I am dismayed that I can so easily find a way that is 400 times better than what you and the community think is good and continue to be ostracized.

myearwood1 commented 5 months ago

You take my ideas and make it seem you did the work by not crediting me. That must be against GitHub's concept of building good communities. I showed you how to make regex up to 7 times faster and I then showed how to make the entire product faster and in my case up to 400 times faster.

Jimrnelson commented 5 months ago

@myearwood1

I spent quite a few hours over a four day period investigating your suggestions here for optimization.

I was successful using the new scope option "Custom GF" to create a prototype using FindStr that applied all the applicable GF settings (which extensions to use, for instance), returned all the same results as GF, and was five to six times faster.

I was very close to being ready to install this as the default search (for folders only) in GF when I ran into a serious roadblock - I encountered a case where FindStr did not return correct results. Normal GF found matches in 817 files, yet this new technique using FindStr only found 809.

Below is the simplified demonstration of the problem. The top command shows the normal use of FindStr. Looking for matches in files "*.PRG", there is only one match. And yet, when executing the second command, it finds eight additional files, all with extension "PRG", so all of those found in the second set should have been found in the first set as well.

image

This tells me that FindStr is not 100% reliable and we do not understand the circumstances in which it fails.

Unfortunately, as I am only willing to use a new search algorithm that can exactly duplicate the current results from GF, this new search algorithm cannot be used as a replacement for the current algorithm.

Jimrnelson commented 5 months ago

Forget FindStr. It has limited regex abilities. Download and install cygwin's utilities at this link:

https://www.cygwin.com/setup-x86_64.exe

That will give us access to grep.exe.

As powerful as this might be, GF is not going to require installation of any third party software.

myearwood1 commented 5 months ago

I already told you to use grep instead of findstr.

On Thu, May 16, 2024, 7:11 p.m. Jimrnelson @.***> wrote:

Forget FindStr. It has limited regex abilities. Download and install cygwin's utilities at this link:

https://www.cygwin.com/setup-x86_64.exe

That will give us access to grep.exe.

As powerful as this might be, GF is not going to require installation of any third party software.

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2116352996, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL36W7Z6SZESLJ4A3MCIRTZCU4LFAVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJWGM2TEOJZGY . You are receiving this because you were mentioned.Message ID: @.***>

myearwood1 commented 5 months ago

You should let the users dictate what they want, not you.

On Thu, May 16, 2024, 7:11 p.m. Jimrnelson @.***> wrote:

Forget FindStr. It has limited regex abilities. Download and install cygwin's utilities at this link:

https://www.cygwin.com/setup-x86_64.exe

That will give us access to grep.exe.

As powerful as this might be, GF is not going to require installation of any third party software.

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2116352996, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL36W7Z6SZESLJ4A3MCIRTZCU4LFAVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJWGM2TEOJZGY . You are receiving this because you were mentioned.Message ID: @.***>

myearwood1 commented 5 months ago

You did include wwDotNetBridge.dll without asking users if they want it. You also include SF_RegExp.dll

grep is already located here: C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer\Git\usr\bin

Since you implemented my suggestion about speeding up RegEx, after carefully including only the files I need to search, I get 79.9 seconds. Grep does the same thing and gives the same results in 15.8 seconds. You don't have to force anyone to use it.

In case you're interested, here's the command I use to get grep to find the same regex that I gave GoFish 7.0.31. Note how nicely the file extensions are handled.

C:\cygwin64\bin\grep.exe -r -l -i -P --include=.{??a,prg} '^(?!\s*).*THERE\s+IS\s+NO\s+SUCH\s+APPRO' c:\myfolder

Jimrnelson commented 5 months ago

Using grep in Windows requires that you first have run a Windows installer for it.

That is not something that GF is going to get involved in.

myearwood1 commented 5 months ago

Prove it cannot be run just by putting it in a folder.

On Sat, May 18, 2024, 11:02 a.m. Jimrnelson @.***> wrote:

Using grep in Windows requires that you first have run a Windows installer for it.

That is not something that GF is going to get involved in.

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2118850569, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL36W334SSRSXYPQUYHGJDZC5UR7AVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYHA2TANJWHE . You are receiving this because you were mentioned.Message ID: @.***>

myearwood1 commented 5 months ago

Regardless, there is one that is trying to be the fastest grep ever, called hypergrep. Will you also gatekeep against it?

On Sat, May 18, 2024, 11:49 a.m. Mike Yearwood @.***> wrote:

Prove it cannot be run just by putting it in a folder.

On Sat, May 18, 2024, 11:02 a.m. Jimrnelson @.***> wrote:

Using grep in Windows requires that you first have run a Windows installer for it.

That is not something that GF is going to get involved in.

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2118850569, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL36W334SSRSXYPQUYHGJDZC5UR7AVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYHA2TANJWHE . You are receiving this because you were mentioned.Message ID: @.***>

myearwood1 commented 5 months ago

You can recommend or suggest people install it. The rest of the intermediates can live with their lowbrow solution.

On Sat, May 18, 2024, 11:51 a.m. Mike Yearwood @.***> wrote:

Regardless, there is one that is trying to be the fastest grep ever, called hypergrep. Will you also gatekeep against it?

On Sat, May 18, 2024, 11:49 a.m. Mike Yearwood @.***> wrote:

Prove it cannot be run just by putting it in a folder.

On Sat, May 18, 2024, 11:02 a.m. Jimrnelson @.***> wrote:

Using grep in Windows requires that you first have run a Windows installer for it.

That is not something that GF is going to get involved in.

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2118850569, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL36W334SSRSXYPQUYHGJDZC5UR7AVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYHA2TANJWHE . You are receiving this because you were mentioned.Message ID: @.***>

Jimrnelson commented 5 months ago

I did try it in a folder. It failed, requiring dlls.

If you can provide all the files needed so that the normal process of unzipping the GF files as maintained in GitHub will allow grep (or any variant) to work, I will certainly use them.

On Sat, May 18, 2024 at 9:50 AM Mike Yearwood @.***> wrote:

Prove it cannot be run just by putting it in a folder.

On Sat, May 18, 2024, 11:02 a.m. Jimrnelson @.***> wrote:

Using grep in Windows requires that you first have run a Windows installer for it.

That is not something that GF is going to get involved in.

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2118850569, or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABL36W334SSRSXYPQUYHGJDZC5UR7AVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYHA2TANJWHE>

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2118863731, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATLLFR7CWCHTRHCGW2FK7N3ZC52EPAVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYHA3DGNZTGE . You are receiving this because you commented.Message ID: @.***>

-- Jim Nelson

Jimrnelson commented 5 months ago

You can recommend or suggest people install it. The rest of the intermediates can live with their lowbrow solution.

Actually, YOU can recommend it. And, if so inclined, YOU can provide a Custom UDF that will take advantage of it.

The beauty of the new Custom UDF feature is that you (or anybody) can add this functionality for their own personal use without requiring any modification to GF at all.

myearwood1 commented 5 months ago

Why should I do that? You give no thanks. You make others' work and ideas appear to be yours.

On Sat, May 18, 2024, 12:01 p.m. Jimrnelson @.***> wrote:

I did try it in a folder. It failed, requiring dlls.

If you can provide all the files needed so that the normal process of unzipping the GF files as maintained in GitHub will allow grep (or any variant) to work, I will certainly use them.

On Sat, May 18, 2024 at 9:50 AM Mike Yearwood @.***> wrote:

Prove it cannot be run just by putting it in a folder.

On Sat, May 18, 2024, 11:02 a.m. Jimrnelson @.***> wrote:

Using grep in Windows requires that you first have run a Windows installer for it.

That is not something that GF is going to get involved in.

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2118850569, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ABL36W334SSRSXYPQUYHGJDZC5UR7AVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYHA2TANJWHE>

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2118863731, or unsubscribe < https://github.com/notifications/unsubscribe-auth/ATLLFR7CWCHTRHCGW2FK7N3ZC52EPAVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYHA3DGNZTGE>

. You are receiving this because you commented.Message ID: @.***>

-- Jim Nelson

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2118866339, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL36W756OKW4467Y3NLJJLZC53NNAVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYHA3DMMZTHE . You are receiving this because you were mentioned.Message ID: @.***>

myearwood1 commented 5 months ago

If I can determine if hypergrep is indeed faster than grep and since the code is freely given, I just have to do a build, then YOU can release that exe and YOU can decide when to update it. I used to give my time to this community, but since I am called names by the so-called leaders and since the performance of the code is 400 times slower than what I can do, which is how I define lousy, you continue to seem to want to make an enemy of me.

On Sat, May 18, 2024 at 12:55 PM Mike Yearwood @.***> wrote:

Why should I do that? You give no thanks. You make others' work and ideas appear to be yours.

On Sat, May 18, 2024, 12:01 p.m. Jimrnelson @.***> wrote:

I did try it in a folder. It failed, requiring dlls.

If you can provide all the files needed so that the normal process of unzipping the GF files as maintained in GitHub will allow grep (or any variant) to work, I will certainly use them.

On Sat, May 18, 2024 at 9:50 AM Mike Yearwood @.***> wrote:

Prove it cannot be run just by putting it in a folder.

On Sat, May 18, 2024, 11:02 a.m. Jimrnelson @.***> wrote:

Using grep in Windows requires that you first have run a Windows installer for it.

That is not something that GF is going to get involved in.

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2118850569, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ABL36W334SSRSXYPQUYHGJDZC5UR7AVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYHA2TANJWHE>

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2118863731, or unsubscribe < https://github.com/notifications/unsubscribe-auth/ATLLFR7CWCHTRHCGW2FK7N3ZC52EPAVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYHA3DGNZTGE>

. You are receiving this because you commented.Message ID: @.***>

-- Jim Nelson

— Reply to this email directly, view it on GitHub https://github.com/VFPX/GoFish/issues/226#issuecomment-2118866339, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL36W756OKW4467Y3NLJJLZC53NNAVCNFSM6AAAAABHCKGTZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYHA3DMMZTHE . You are receiving this because you were mentioned.Message ID: @.***>

myearwood1 commented 5 months ago

The image is of the files needed to get grep running without any install. I used ccleaner to ensure I had nothing dangling in the registry. I would like to explore hypergrep because it is based on Intel hyperscan, but the community should not have to wait on that. Personally, I think the users should install grep on their own, that way they can keep updating it and there will not be a GoFish copy and possibly their own on the machine.

image

To prove it worked without all the supporting files:

C:\mygrep\grep.exe -r -l -i -P --include=.{??a,prg} '^(?!\s*).*THERE\s+IS\s+NO\s+SUCH\s+APPRO' c:\mysource

It still took 15 seconds to scan all my source.

Jimrnelson commented 5 months ago

Since you already have those files, can you attach them here?

myearwood1 commented 5 months ago

mygrep.zip

I have been working on using CreateProcess to get the process ID once grep starts and then monitor it with WaitForSingleObject in a loop to show feedback and allow for interrupting it. That is less CPU intensive than a straight do while loop and it should react more quickly. Then when it finishes, use SED.exe to fix the backslashes in the file before importing it into gofish.

Jimrnelson commented 5 months ago

Thanks for the zip.

I have been working on using CreateProcess to get the process ID once grep starts and then monitor it with WaitForSingleObject in a loop to show feedback and allow for interrupting it. That is less CPU intensive than a straight do while loop and it should react more quickly.

What are you trying to accomplish here?

use SED.exe to fix the backslashes in the file before importing it into gofish.

That won't be necessary as I have other modifications to do as well.

myearwood1 commented 5 months ago

While Grep is running, we need to wait for it to complete the search. ShellExecute returns immediately without waiting. A do while loop for 15 seconds is not how I would do it. Grep outputs Linux style slashes, so that will have to be addressed.

myearwood1 commented 5 months ago

image image

Jimrnelson commented 5 months ago

While Grep is running, we need to wait for it to complete the search. ShellExecute returns immediately without waiting.

Since I'm not using ShellExecute, this is not a problem

Grep outputs Linux style slashes, so that will have to be addressed.

Thanks for noting this, I will have to fix that when created the cursor of file names.

myearwood1 commented 5 months ago

What will you use to launch grep? The FoxPro run command does not always work. I would prefer not seeing a DOS box, but as long as it works, I don't mind too much.

Jimrnelson commented 5 months ago

oShell = Createobject ('wscript.shell') m.oShell.Run (m.lcCommand, 7, .T.)

Jimrnelson commented 5 months ago

can you point to the command line documentation for grep parameters?

myearwood1 commented 5 months ago

Grep --help

myearwood1 commented 5 months ago

As an example: C:\mygrep\grep.exe -r -l -i -P --include=*.{??a,prg} '^(?!\s*\*).*THERE\s+IS\s+NO\s+SUCH\s+APPRO' c:\folder to read That is going to read all ??a files as well as all .prg files, it will avoid lines starting with * (comments) and find everything with THERE IS NO SUCH APPRO regardless of how many spaces are between the words. Some of the messages in this system are spaced out for emphasis, making them very hard to find.

myearwood1 commented 5 months ago

If you use the wscript shell, you'll have to preface the grep command with cmd.exe so you can pipe the output into a text file. "cmd.exe /C C:\mygrep\grep.exe -r -l -i -P --include=.{??a,prg} '^(?!\s*).*THERE\s+IS\s+NO\s+SUCH\s+APPRO' c:\search folder > grep_output.txt"