[Design Task] Guidelines and tooling support for Move code portability

[sblackshear]

Move compatibility in general. There were already some compatibility challenges due to differences in frameworks (e.g., libraries that depend on DiemAccount aren't compatible with ones that depend on StarcoinAccount), and Sui Move's data model that differs from core Move's global storage will throw another wrench in the works. Some incompatibility is to be expected given the nature of Move (a platform-independent language that gives platform designers a lot of flexibility to customize both the Move runtime and libraries for their platform). But of course, we should still strive for as much compatibility as possible and suggest best practices that will help with this.

Some early thoughts on this:

• Compatibility should be a source/package level concern. Striving for compatibility at the bytecode level across different networks is a much harder task, and it's not clear that it's needed (and thus, I think is a task that we should not attempt). There are packages that can be 100% compatible at the source level, but incompatible at the bytecode level due to hardcoded addresses, address length, and in the future, possibly other factors like varying the serialization format. However, none of these are blockers for source compatibility, at least not in principle.
• An obvious best practice is to suggest not writing "non-portable" code that depends on address length or serialization format. This is very similar to (e.g.) not writing C code that depends on the pointer size.
• We can suggest "tiering" modules based on how likely they are to be reusable, and encourage packages to only group modules in the same tier. Otherwise, someone might want to depend on some reusable module in a package, but be blocked by some incompatible (but unused) module in the same package. Some natural tiers that come to mind are:
  1. Modules that only define utility functions and do not define any types (e.g., a Math.move module). They must (transitively) only depend on other tier 1 packages. These are fully generic and can be used by any platform.
  2. Modules that define utility functions and expose utility types (e.g., ASCII::String in the stdlib). They must (transitively) only depend on other tier 1-2 packages. These can be used by any platform in principle, but there's nothing that stops two platforms from (e.g.) defining and reusing type-incompatible versions of UTF8::String. We should try to avoid this by working as a Move community to build canonical libraries of these types.
  3. Same as (2), but also has some native functions. They must (transitively) only depend on other tier 1-3 packages. These can be used by any platform in principle, but some platforms might not want to use the native functions defined by some other platform because native functions can violate the rules of core Move (e.g., by creating addresses from scratch).
  4. Same as (3), but also has entrypoints. Different platforms can (and will) choose different rules for entrypoints that may not be compatible. These must (transitively) only depend on other tier 1-4 packages. These are only reusable across platforms with the same entrypoint structure.
  5. Same as (4), but also uses global storage (e.g., borrow_global, Table). These must (transitively) only depend on other tier 1-5 packages. These are only reusable across platforms with the same entrypoint structure and global storage.
  6. Same as (5), but also uses platform-specific packages/types (e.g. accounts, transactions, time, validator set management, ...). These are not likely to be compatible with other packages. Perhaps over time, we can define source-level "interfaces" for some of these concepts to allow compatibility for similar platforms, but I wouldn't want to rush into this. Some level of incompatibility is always going to be expected here.

I'm not sure whether this tiering system should just be an informal guideline, or also something that we try to enforce programmatically in the package system and/or linters. It's worth noting that packages are not organized according to this tiering system today, although some are close (e.g., there's very little global storage used in the Move stdlib). But the overwhelming tendency is toward monolithic tier 6 packages, which I think we should change.

Once we have reorganized packages using these principles, I think there will be a lot more boxes in your diagram above, but the tiering system will make it easier to figure out where we can/should draw dependency arrows and where we shouldn't.

(copied from original discussion in https://github.com/diem/move/issues/91)

[tnowacki]

I think you might want to flip tier 4-5? I feel like you might want to itnroduce storage concepts before you do entry points

I think the tier system definitely seems expressive, but maybe overkill for a package system. Thoughts on just splitting it into: "Library", "X-Library" where X is some instantiation of Move (Sui, Diem, etc), or "X-Application" I think

• "Library" tier 1-2
• "X-Library" tier 3+5
• "X-Application" tier whatever's left

I the short term we could at least enforce a "library" annotation in the package system, that checks against natives (outside of those in the stdlib?), globals, entries, etc.

[jolestar]

How to define the tier is a challenge.

Option 1: Like Rust, a project is a workspace, which corresponds to an address in Move, and there are many crates in a project which correspond to packages in Move. A project can depend on another project's sub-crates.

Option 2: Like Solidity, we distinguish between contract and library, and the library can only include stateless modules and will be embedded in the binary of contract when packaging. This can solve the dependency problem of pure util library(tire 1), but the disadvantage is that duplicate binary wastes state space on the chain.

Before finding a good tier solution, we can figure out how to share stdlib. Currently, Move stdlib is embedded in the x-framework in different projects and shares the 0x1 address with the framework.

Maybe stdlib can use the reserved address 0x0 and load it directly from the VM. Using the 0x0 address also can avoid module name conflicts between stdlib and x-framework.