Physical redesign

[thejohnfreeman]

There are several changes I'd like to see made to our physical source layout:

• Move third-party dependencies (secp256k1, ed25519) from src/ to external/. Cleanly separate first-party code from third-party. Make it clear that first-party code is under active development in this project.
• Move CMake scripts to cmake/. Follow convention.[^3]
• Separate public headers from sources and private headers. Put public headers under include/{name}. name can be ripple to start, to minimize impact on #include directives[^1]. Put sources and private headers under src/rippled[^2]. The separation will help consumers easily browse just the installed definitions.
• Separate libxrpl from rippled. Move sources and private headers for the libxrpl target into src/libxrpl from src/rippled. This will help easily implement one aspect of levelization: no source in libxrpl should depend on any source in rippled.
• The build rules should use globs to install all public headers and to compile all sources. This relieves contributors from having to remember to edit sources lists in the CMake.
• Remove impl directories. Should be no need for this layer once public headers are separated.

[^1]: Eventually it should be renamed to xrpl, to further draw the line of separation between the XRPL and Ripple. [^2]: Which should eventually be renamed to src/xrpld for the same reason. [^3]: The rest of the Builds/ directory should eventually disappear: container code should move to a separate project like rippled-docker; levelization code should be obviated by good physical design; and the rest is likely bit-rotted, unused for several years, and should just be deleted.

These recommendations are based on my research into project structure, looking at standardization attempts, major projects, and build system conventions and defaults. The result of these changes will look something like this:

/
|- conanfile.py
|- CMakeLists.txt
|- cmake/
|- external/
|  |- ed25519-donna/
|  |- secp256k1/
|  |- snappy/
|  `- soci/
|- include/
|  `- ripple/
`- src/
   |- libxrpl/
   |- xrpld/
   `- tests/

These changes are pretty easy to implement, taking maybe two days, but will have a very large "who moved my cheese?" effect. It will be more disruptive than the formatting commit from a few years ago. I think the best way to make it happen is with a script that makes all the changes (file moves and include fixes). Run that script on the develop branch and push the result as commit C. Let developers merge their work-in-progress branch with parent(C), run the script, commit the result, and then merge with C.

See related: #4593

[thejohnfreeman]

Moving this discussion elsewhere. Leaving the issue open just to indicate it is an open problem.

[thejohnfreeman]

Some things I've run into while working on the script:

• Nearly every source (.cpp) in basics, json, crypto, and protocol are linked in libxrpl. One exception is a selection of files in basics/impl that are linked into rippled instead. In my testing, they can be moved to libxrpl with no changes, and I think they should.
• Most beast headers are installed, but there are 30 exceptions across beast/asio, beast/container, beast/core, beast/test, and beast/utility. I think they should all be installed.
• There are a few files that appear useless that I would like to remove:
  • json/README.md
  • json/TODO.md: probably obsolete in the face of Boost.Json migration, and should be in issues anyway
  • json/impl/LICENSE: should be moved to the top-level LICENSE file if not removed
  • json/impl/version: almost certainly out-of-date, and not used anywhere
• There is one header (basics/PerfLog.h) that is installed as part of libxrpl but cannot be imported because it includes headers not installed as part of rippled. I think it should not be installed.