kurtlawrence commented 5 years ago

The ability to load items/stmts/crates from a file (or other source).

Thinking that the source can be parsed using current technology and added into the current mod.

mcandre commented 4 years ago

Yes, something like gore's :import or similar would really unlock the potential of this REPL!

kurtlawrence commented 4 years ago

Papyrus File Loading Prototyping

Goal

Handling of 'static' code, which does not need to constantly written to the lib.rs file each REPL cycle. Each 'file' can be written once and referenced in the lib file. This may help incremental compilation as well.

Storage

Storage would be done on the ReplData. Instead of storing the contents of the code (as this would be static once written to disk) a hash of the contents is taken. As external crates may be referenced, a list of CrateType is required. The writing path is required, this is required to write the file (under ./src), and to remove an entry if required. The path can be used as a key.

struct StaticFile {
    path: PathBuf,
    codehash: Vec<u8>,
    crates: Vec<CrateType>,
}

// storage on ReplData
type StaticFiles = HashSet<StaticFile>;

Notes:

StaticFile is keyed on path for hashing/eq

codehash can be any hash should use blake2

path needs to be validated, it must be relative and must resolve to something under ./src

Adding

A function on ReplData can be used add a static file.

impl ReplData {
    fn add_static_file(&mut self, path: PathBuf, code: &str) -> Result<bool, AddingStaticFileError>
    {
    // validate `path`

    // hash `code`

    // compare `path` in set

    // if non-existent or hashes differ
    // - parse for crates
    // - write to disk, without the crates, under ./src

    // return if updated or unchanged
    }
}

Notes:

code is hashed with the crates inside, but written without.

multiple failure points, hence the return type.

Adding will write the file contents but will not trigger recompilation.

Removing

Removing is done through a function on ReplData. It should only require the path. Removal will remove the entry and remove the file from disk.

Path validation

A path must resolve to something under ./src, adjacent to where the lib.rs file sits. Validation can be done by stepping through the path components and only allowing basic file or folder components (no relative '.' or '..' or root slashes. Names also need to be valid as module names, so have to be checked for -, or not starting with correct characters, or including whitespace.

Crate parsing

Crates can be specified using the extern crate crate_name; syntax at the top of the file. There will be an explicit parsing implementation which requires crate declaration to be at the top. The first n lines that parse as a crate will be ignored when writing code to disk. Each crate will be added to a vector to be added to the toml and lib file on REPL cycles.

Linking in REPL cycle

Additions of static files is done on each REPL cycle. The crates are injected into the Cargo.toml creation, and the lib/root level mods are added to the source code. Rather than ask users to define the mod name (as paths and names would need to match), paths are parsed for the lib/root mod addition. There are no checks done if a static file is defined along a path that won't be included in compilation. An example of this could be defining a path foo/bar.rs. The way to reference this would be to add another static file foo/mod.rs with the contents pub mod bar;. The parser will only add mods if they are at the root level. This means paths foo.rs and foo/mod.rs would be considered legitimate and be included as mod foo; Documentation will have to stress the implementation details to ensure users name things correctly.

kurtlawrence / papyrus

Load from file/string #14