search

VectorCamp / vectorscan

A portable fork of the high-performance regular expression matching library
https://www.vectorcamp.gr/project/vectorscan/
Other
495 stars 53 forks source link

Evaluate removing 32-bit architectures support in general (was Intel 32-bit) #195

Open markos opened 10 months ago

markos commented 10 months ago

We are evaluating removing Intel 32-bit support to simplify the codebase. Currently the code builds but it is unclear if anyone is using vectorscan on older 32-bit CPUs. For those older ports, there is always the existing Hyperscan project which works just fine there.

a16bitsysop commented 9 months ago

it was disabled on x86 on alpine as it still has the sse4.2 minimum requirement on x86

victorjulien commented 9 months ago

If Hyperscan is closing up, as mentioned here https://github.com/VectorCamp/vectorscan/issues/139#issuecomment-1821176637, it might be good to support the broadest possible ARCH set in Vectorscan. I understand this can be a significant burden, but for a project like Suricata the limited architecture support of Hyperscan has meant we've needed to maintain alternative pattern matchers for other archs and not used some of it's more advanced features like streaming mode.

If vectorscan can support all archs we (really) care about (Intel, Arm and in the future perhaps RiscV), it would be great. Then we can work on fully supporting all the great features much more easily.

Whether that means Intel 32 bit is important is questionable. I don't think we see any serious 32 bit deployments of Suricata anymore, neither for ARM or Intel.

So I guess my feedback is more about the general goal of the project: is it targeting a wide range of archs and OS' or a more limited "modern and current" set?

markos commented 9 months ago

@victorjulien it's funny, your reply came right at the moment when I was looking at how much more effort it would be to enable armhf and powerpc 32-bit as well :)

As it turns out, I'm almost finished working on a SIMDe backend. This won't replace the existing SIMD backends, but it will be nice to compare performance, with the bonus feature that SIMDe has a scalar backend- so it can run on all architectures without a supported SIMD unit. So far all tests -but 3- pass and this feature will go in the next version. It will obviously be slower, but it does mean there can be a single library for all platforms supported.

Yes, you are right, it's worth adding more platform support. It will not happen immediately of course.

markos commented 9 months ago

With regards to RISC-V, I'm still waiting for a decent platform to buy that has working SIMD/RVV support.

markos commented 9 months ago

Also, we have decided to add support for OpenWrt in one of the next releases, depending on how difficult it would be.

markos commented 9 months ago

I still haven't answered your question though:

So I guess my feedback is more about the general goal of the project: is it targeting a wide range of archs and OS' or a more limited "modern and current" set?

As it turns out we have limited resources and our foremost goal is to support a few selected architectures as best we can. These include: a) aarch64, b) x86_64, c) ppc64le. Next in line are: Loongson, Mips64el (with MSA), Risc-V with RVV and for all other architectures, a SIMDe backend. Now these should cover most if not all modern architectures. The real question for us is if it's worth keeping support for 32-bit arches in general, as it pollutes the codebase with tons of #ifdefs 32/64-bit. The biggest problem though is that we do not test any of the 32-bit codepaths. We test 100s of configurations in our CI, but none 32-bit. So it's an honest question, because it's not really about diversity of existing hardware platforms -which we care about- but supporting really ancient hardware that is unlikely to be supported in the future by current distributions.

victorjulien commented 9 months ago

One comment I received offline about 32bit support, is that Raspberry PI OS still defaults to 32bit even if the hardware is 64bit capable. A 64bit version is available but the 32bit version is still listed first in their downloads.

markos commented 9 months ago

One comment I received offline about 32bit support, is that Raspberry PI OS still defaults to 32bit even if the hardware is 64bit capable. A 64bit version is available but the 32bit version is still listed first in their downloads.

The source of my concern is that I know many distros -including Debian- are considering dropping armel/armhf support in the near future. Perhaps because of RISC-V keeping 32-bit might make sense.