Closed billowqiu closed 1 year ago
Hi @billowqiu,
Thank you for your interest in BlazingMQ! BlazingMQ is feature-rich, battle-tested message queue that supports several different network topologies and message routing strategies, among much other functionality that we have found very important to our use cases. With this level of functionality necessarily comes a larger code footprint.
Some of what you're seeing is generated files for our protocol; when we exclude those, the size shrinks significantly:
src $ cloc --not-match-f="(_messages\\.|\\.t\\.cpp)" .
865 text files.
788 unique files.
77 files ignored.
github.com/AlDanial/cloc v 1.98 T=0.55 s (1434.4 files/s, 609002.7 lines/s)
--------------------------------------------------------------------------------
Language files blank comment code
--------------------------------------------------------------------------------
C++ 360 25858 25877 114699
C/C++ Header 353 26594 63323 70271
XSD 6 407 124 3908
Text 36 248 0 1383
Markdown 10 74 0 539
JSON 9 0 0 465
CMake 11 73 142 224
yacc 1 22 15 127
lex 1 32 4 118
Bourne Again Shell 1 8 5 29
--------------------------------------------------------------------------------
SUM: 788 53316 89490 191763
--------------------------------------------------------------------------------
Another part of this is, as you say, because BlazingMQ is a C++03 codebase that depends on very few third-party libraries, we have needed to implement some of the basic, non-message-queue functionality that projects written in other languages might be able to pull in as a dependency more easily than we were able when BlazingMQ was first being written years ago. These components are contained within src/groups/mwc/
, and represent a bit over one fifth of the codebase:
src $ cloc --not-match-f="(_messages\\.|\\.t\\.cpp)" groups/mwc/
267 text files.
242 unique files.
25 files ignored.
github.com/AlDanial/cloc v 1.98 T=0.16 s (1507.9 files/s, 471296.9 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
C/C++ Header 112 8082 20127 25810
C++ 112 3318 3656 13592
Text 13 76 0 507
Markdown 3 28 0 236
XSD 1 11 1 161
CMake 1 5 9 18
-------------------------------------------------------------------------------
SUM: 242 11520 23793 40324
-------------------------------------------------------------------------------
This repository also includes both the client C++ SDK libbmq
, the broker bmqbrkr
, and tools like bmqtool
to interact with the broker outside of an application.
We don't believe this is too much to be worried about. Our peer open-source message queue solutions with comparable levels of functionality seem to be in the same order of magnitude of codebase size as us. A very quick check for Kafka:
kafka $ cloc *
5642 text files.
5501 unique files.
198 files ignored.
github.com/AlDanial/cloc v 1.98 T=4.57 s (1203.6 files/s, 268274.4 lines/s)
--------------------------------------------------------------------------------
Language files blank comment code
--------------------------------------------------------------------------------
Java 4282 116548 166894 632634
Scala 633 30973 32410 173700
HTML 60 2495 1158 21282
Python 134 3809 5332 14968
JSON 209 216 0 9182
Gradle 4 507 308 2756
XML 11 300 350 1680
Bourne Shell 56 293 1006 1274
Markdown 12 409 0 1129
Properties 56 329 1263 709
Bourne Again Shell 2 66 112 497
DOS Batch 35 79 520 266
Maven 3 27 50 221
XSLT 1 26 27 153
Dockerfile 1 16 32 72
JavaScript 1 3 15 6
Text 1 0 0 1
--------------------------------------------------------------------------------
SUM: 5501 156096 209477 860530
--------------------------------------------------------------------------------
And RabbitMQ (written in a less verbose language than us):
rabbitmq-server $ cloc *
3104 text files.
2540 unique files.
636 files ignored.
github.com/AlDanial/cloc v 1.98 T=1.23 s (2071.4 files/s, 411121.5 lines/s)
------------------------------------------------------------------------------------
Language files blank comment code
------------------------------------------------------------------------------------
Erlang 1096 37380 32964 222644
JSON 57 65 0 31399
Elixir 415 7958 2977 27974
JavaScript 70 4493 4710 23345
Markdown 250 12176 14 17150
Starlark 129 1880 1812 17035
make 82 1957 759 10014
EJS 58 249 6 6218
Text 69 1191 0 5345
Java 27 884 427 4603
Python 48 1073 929 4521
Bourne Shell 64 646 629 3283
HTML 14 192 4 2985
YAML 21 47 138 2284
DOS Batch 17 326 163 1390
Bourne Again Shell 10 144 5 802
XML 17 80 300 770
Maven 7 91 9 743
AsciiDoc 4 176 0 705
HCL 9 104 33 536
F# 1 71 12 476
C# 13 92 40 365
CSS 3 76 2 326
Dockerfile 5 32 70 262
SVG 2 0 2 244
Ruby 18 64 19 217
Kotlin 3 26 41 151
MSBuild script 3 5 7 149
PHP 5 24 63 105
Bazel 1 31 0 103
Perl 6 9 4 69
Properties 7 12 0 54
Visual Studio Solution 2 2 2 46
INI 3 12 0 44
CSV 1 2 0 24
diff 1 2 6 7
Groovy 1 2 2 6
ASP.NET 1 0 0 1
------------------------------------------------------------------------------------
SUM: 2540 71574 46149 386395
------------------------------------------------------------------------------------
These might be overcounts for these projects, so I would only take these numbers as indicative that BlazingMQ is not significantly larger than other open-source message queues. There's simply a certain amount of inherent complexity that any enterprise-grade message queue must contain.
That said, if you're interested in learning more about BlazingMQ or perhaps contributing to the project, this does make it a bit intimidating. I would recommend looking at our Getting Started guide, our C++ library API docs, and our (work-in-progress) protocol guide. Additionally, you may also be interested in our Java SDK (and its API docs), which is much smaller, at the cost of reduced functionality exposed to users.
@pniedzielski Thank you for your wonderful reply.
@billowqiu Please feel free to reopen if you have any additional questions. Thanks!
No third-party libraries included,blazingmq codebase use cloc output.