Prepared internal structure for new architectures.

[ralfbiedert]

Note that Cargo.toml still points to a relative repo since the other crates haven't merged yet. Providing pull request for more structured review.

[AdamNiederer]

Tests and perf look great. Thanks a bunch! I'll fix up the dependencies once merged.