search

gigablast / open-source-search-engine

Nov 20 2017 -- A distributed open source search engine and spider/crawler written in C/C++ for Linux on Intel/AMD. From gigablast dot com, which has binaries for download. See the README.md file at the very bottom of this page for instructions.
Apache License 2.0
1.54k stars 441 forks source link

Feature Request: Modularize the Gigablast Source #133

Open martinvahi opened 7 years ago

martinvahi commented 7 years ago

Currently it's a huge pile of files, but if the source were modularized, people could contribute by working on a specific area. For example, if the database layer were abstracted away,
then people might be able to offer different database backends as pull requests. The same with spidering and indexing. There might be different spidering strategies.

The whole source should also be debugged with the [Valgrind Memcheck (http://valgrind.org/docs/manual/mc-manual.html).

According to my, very subjective, understanding(archival copy), dividing a project to sub-modules is a kind of work that only the original author of the project, the architect, must do.

I mean, imagine, how hard it would be to contribute to the Linux kernel project, if the Linux kernel were a huge pile of files that does not have clearly defined sub-areas that people can work on. The open source development of a project depends on modularity, because in the case of non-modular architecture people start to "step on each others toes" very quickly.

A question: I have noticed that there is some development going on at the Gigablast online service side of the Gigablast project, but given that as of 2017_05 there are no updates in the GitHub repository for years, then has the open source development of the Gigablast project stopped? As I said, modularizing the project and making the project Valgrind Memcheck passaable would be a necessary step for making the Gigablast project ready for public, open source, development.

Thank You for reading my comment.

martinvahi commented 7 years ago

Due to some helpful hints from the Wild-Wild-Web I came to know the following Gigablast upgrade project:

https://github.com/privacore/open-source-search-engine