[Feature] Preserve/Consistent Directory Ordering

[gluax]

It would be excellent for determinism if the order would always be the same when you iterated through an included directory, from when run to another. Could make use of the IndexMap crate to assist with this.

[Michael-F-Bryan]

All entries in a directory are stored in alphabetic order, so as long as the set of items in the included directory don't change it should always be the same.

https://github.com/Michael-F-Bryan/include_dir/blob/06303ec208189a7f71ee2d2cf9d742b540f0d9d8/macros/src/lib.rs#L141

[gluax]

@Michael-F-Bryan was this added in the latest version? At least currently in my application, it was different every time. For reference this for module in INCLUDED_DIR.find("directory/*.ext").unwrap() resulted in a different order every time.

[Michael-F-Bryan]

Yes, this was included in v0.7.0.

Can you upgrade and see if the problem persists?

[gluax]

@Michael-F-Bryan the problem does indeed remain.

The latter is iterated over in a different order every time.

let prelude = STDLIB.get_dir("prelude").unwrap();

for module in prelude.files() {}

[Michael-F-Bryan]

@gluax, can you provide a minimal reproducible example for this?

In particular,

• Is the order changing between calls to files() (or dirs() or entries()) within the same run of the program, or does the order only change when you recompile?
• Can you show a couple orders that are being returned?

I tried it locally and was unable to reproduce the issue.

This is how I set the project up:

$ rustc --version --verbose
rustc 1.58.0-nightly (bd41e09da 2021-10-18)
binary: rustc
commit-hash: bd41e09da334697c0f993b36685cb599061d9faa
commit-date: 2021-10-18
host: x86_64-unknown-linux-gnu
release: 1.58.0-nightly
LLVM version: 13.0.0

$ cd /tmp
$ cargo new repro && cd "$_"
$ mkdir assets
$ touch assets/{foo,bar,baz}{1,2,3}.{rs,md}
$ tree
.
├── assets
│   ├── bar1.md
│   ├── bar1.rs
│   ├── bar2.md
│   ├── bar2.rs
│   ├── bar3.md
│   ├── bar3.rs
│   ├── baz1.md
│   ├── baz1.rs
│   ├── baz2.md
│   ├── baz2.rs
│   ├── baz3.md
│   ├── baz3.rs
│   ├── foo1.md
│   ├── foo1.rs
│   ├── foo2.md
│   ├── foo2.rs
│   ├── foo3.md
│   └── foo3.rs
├── Cargo.toml
└── src
    └── main.rs

2 directories, 20 files

$ cargo add include_dir
    Updating 'https://github.com/rust-lang/crates.io-index' index
      Adding include_dir v0.7.1 to dependencies

The contents of src/main.rs:

use include_dir::{include_dir, Dir};

static ASSETS: Dir<'_> = include_dir!("$CARGO_MANIFEST_DIR/assets");

fn main() {
    for file in ASSETS.files() {
        println!("{}", file.path().display());
    }
}

I compiled it once and ran the same binary several times.

$ cargo build
$ ./target/debug/repro
bar1.md
bar1.rs
bar2.md
bar2.rs
bar3.md
bar3.rs
baz1.md
baz1.rs
baz2.md
baz2.rs
baz3.md
baz3.rs
foo1.md
foo1.rs
foo2.md
foo2.rs
foo3.md
foo3.rs

$  ./target/debug/repro
bar1.md
bar1.rs
bar2.md
bar2.rs
bar3.md
bar3.rs
baz1.md
baz1.rs
baz2.md
baz2.rs
baz3.md
baz3.rs
foo1.md
foo1.rs
foo2.md
foo2.rs
foo3.md
foo3.rs

I also did several runs where I recompiled from scratch:

$ cargo clean && cargo run --quiet
bar1.md
bar1.rs
bar2.md
bar2.rs
bar3.md
bar3.rs
baz1.md
baz1.rs
baz2.md
baz2.rs
baz3.md
baz3.rs
foo1.md
foo1.rs
foo2.md
foo2.rs
foo3.md
foo3.rs

$ cargo clean && cargo run --quiet
bar1.md
bar1.rs
bar2.md
bar2.rs
bar3.md
bar3.rs
baz1.md
baz1.rs
baz2.md
baz2.rs
baz3.md
baz3.rs
foo1.md
foo1.rs
foo2.md
foo2.rs
foo3.md
foo3.rs