search

simsong / bulk_extractor

This is the development tree. Production downloads are at:
https://github.com/simsong/bulk_extractor/releases
Other
1.04k stars 183 forks source link

Cross-compiling for Windows using Fedora 36; CONFIGURE_FEDORA36_win64.bash errors out on libre2-dev #464

Open InfiniteBSOD opened 4 months ago

InfiniteBSOD commented 4 months ago

Host: Win 11 x64 23H2 (22631.3155) VM software: VMware Player 17 Guest OS: Fedora 36 x86-64 uname -a: Linux fedora 5.17.5-300.fc36.x86_64 #1 SMP PREEMPT Thu Apr 28 15:51:30 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Hello,

Trying to compile bulk_extractor for windows using these instructions.

However the script ("CONFIGURE_FEDORA36_win64.bash") stops with this error: checking for RE2... no configure: error: Could not find RE2 library. Please install libre2-dev or equivalent.

OK (I'm not that well versed in Linux): sudo dnf install libre2-dev returns:

No match for argument: libre2-dev
Error: Unable to find a match: libre2-dev

I tried installing re2 which worked (to install) but the script seems to depend on libre2-dev.

I am unsure if pulling the repo as the instructions tell me is correct since I read: Note: Currenlty bulk_extractor 2.1 does not build on windows, but 2.0 does. when viewing the release for 2.0 it refers to Fedora 18 in bulk_extractor-2.0.0\src_win\README_WINDOWS.md.

I tried to install Fedora 18 but it requires me to modify the URLs for yum since the URLs are outdated (updates are now located in "archive" and http instead of https) so I gave up on that route.

Appreciate the application though, would love to try a 2.x-version on Windows.

Best Regards

simsong commented 4 months ago

Thank you for the email. You are correct — there is no support for re2 under MinGW. I've been working on this very problem lately. The problem is that one of our users entered a regular expression [a-z]*@company.com and that expression basically causes the regular expression engine in both pcre and in std::regex to hang. Which means that if you don't have RE2 installed, and you use that regular expression, you will have a bad outcome.

How do you think this should be handled? Do you want to have bulk_extractor without scan_find - the ability to search for arbitrary regular expressions? or would you rather have it hang if you specify a bad regular expression?

or, would you like to port libre2 to MinGW?

simsong commented 4 months ago

Tagging @jonstewart

InfiniteBSOD commented 4 months ago

Thank you for the email. You are correct — there is no support for re2 under MinGW. I've been working on this very problem lately. The problem is that one of our users entered a regular expression [a-z]*@company.com and that expression basically causes the regular expression engine in both pcre and in std::regex to hang. Which means that if you don't have RE2 installed, and you use that regular expression, you will have a bad outcome.

How do you think this should be handled? Do you want to have bulk_extractor without scan_find - the ability to search for arbitrary regular expressions? or would you rather have it hang if you specify a bad regular expression?

or, would you like to port libre2 to MinGW?

Thank you so much for your quick reply.

I am thinking aloud here and I am not that well-versed in bulk_extractor so I might assume that something works a certain way that it doesn't:

Ok so one use-case I thought of with regex is to specify a regex that looks for strings that can be "seeds" for BTC. That doesn't seem to be covered in any available "scanner" although BTC-addresses are identified using the "accts"-scanner if I remember correctly.

I'm not a C/C++ programmer so developing a scanner or plugin to bulk_extractor is beyond my technical knowledge however to be able to specify custom regex to find things that the scanners don't or that is highly specific to a certain DFIR-investigation would be extremely helpful.

So without having any knowledge of how big a job it would be my 5 cents are to port libre2 to MinGW so that custom regex would work.

Thanks again for answering (and very promptly).

Appreciate your work and the software immensely.

simsong commented 4 months ago

libre2 depends on https://abseil.io Both need to be running under MinGW.

InfiniteBSOD commented 4 months ago

libre2 depends on https://abseil.io Both need to be running under MinGW.

Ah gotcha.

Saw this on their GitHub if it might be any help, the linked fix should solve this issue which was "Building Abseil 20230802.0 with MinGW-w64 fails".

Perhaps they (Abseil) fixed so it now compiles under MinGW :) ?

simsong commented 4 months ago

I've actually added support for pcre and re-added support for std::regex to be20_api, so we should have support back for Windows soon. It will just be really, really slow with bad regular expressions.

kieranjol commented 1 month ago

I misread the windows build warning and still tried to compile in Fedora 36 and 40. Glad to hear that a fix is in the works as i was running into crashing issues with the latest version on Windows and I figured there was no point in raising the issue as it was so far behind 2.1.

simsong commented 1 month ago

Hm. And there is this now: https://packages.msys2.org/package/mingw-w64-x86_64-re2

So perhaps I can get re2 running under mingw

On Thu, May 30, 2024 at 5:52 AM kieranjol @.***> wrote:

I misread the windows build warning and still tried to compile in Fedora 36 and 40. Glad to hear that a fix is in the works as i was running into crashing issues with the latest version on Windows and I figured there was no point in raising the issue as it was so far behind 2.1.

— Reply to this email directly, view it on GitHub https://github.com/simsong/bulk_extractor/issues/464#issuecomment-2139190600, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMFHLBAG2OGO43U35V6FK3ZE3ZFXAVCNFSM6AAAAABEJO4QEWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZZGE4TANRQGA . You are receiving this because you commented.Message ID: @.***>

kieranjol commented 1 month ago

Will that require a new configure script if using mingw or could an existing one be used? I know very little about compiling for windows!

On Thu 30 May 2024 at 11:45, Simson L. Garfinkel @.***> wrote:

Hm. And there is this now: https://packages.msys2.org/package/mingw-w64-x86_64-re2

So perhaps I can get re2 running under mingw

On Thu, May 30, 2024 at 5:52 AM kieranjol @.***> wrote:

I misread the windows build warning and still tried to compile in Fedora 36 and 40. Glad to hear that a fix is in the works as i was running into crashing issues with the latest version on Windows and I figured there was no point in raising the issue as it was so far behind 2.1.

— Reply to this email directly, view it on GitHub < https://github.com/simsong/bulk_extractor/issues/464#issuecomment-2139190600>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAMFHLBAG2OGO43U35V6FK3ZE3ZFXAVCNFSM6AAAAABEJO4QEWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZZGE4TANRQGA>

. You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/simsong/bulk_extractor/issues/464#issuecomment-2139279423, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAITFPUV3X3KZZPJM3JNVBLZE37MTAVCNFSM6AAAAABEJO4QEWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZZGI3TSNBSGM . You are receiving this because you commented.Message ID: @.***>