global cache doesn't have a global mutex

[yegor256]

Currently, all our Mojos assume that they are the solo owners of the global cache. There is not synchronization, which may soon lead to problems, if two projects will try to compile at the same machine. Let's introduce a global file-based semaphore.

[levBagryansky]

@yegor256 This approach with global semaphore will lead to the fact that parallel compilations will always be performed in fact sequentially. As I understand cargo has local cache somewhere in target/debug since removing of the directory increases compilation time a lot.

tardis3@R:~/rust_trash$ cargo build
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
tardis3@R:~/rust_trash$ rm -r target/debug/build/
tardis3@R:~/rust_trash$ cargo build
   Compiling libc v0.2.147
   Compiling proc-macro2 v1.0.63
   Compiling memchr v2.5.0
   Compiling quote v1.0.29
   Compiling indexmap v1.9.3
   Compiling clang-sys v1.6.1
   Compiling bindgen v0.60.1
   Compiling libffi-sys v2.3.0
   Compiling memoffset v0.6.5
   Compiling thiserror v1.0.40
   Compiling mpi v0.6.0
   Compiling combine v4.6.6
   Compiling aho-corasick v1.0.2
   Compiling nom v7.1.3
   Compiling atty v0.2.14
   Compiling which v4.4.0
   Compiling rand v0.4.6
   Compiling clap v3.2.25
   Compiling syn v2.0.23
   Compiling regex v1.8.4
   Compiling cexpr v0.6.0
   Compiling env_logger v0.9.3
   Compiling thiserror-impl v1.0.40
   Compiling jni v0.21.1
   Compiling mpi-sys v0.2.0
   Compiling libffi v3.2.0
   Compiling rust_trash v0.1.0 (/home/tardis3/rust_trash)
    Finished dev [unoptimized + debuginfo] target(s) in 15.06s

[yegor256]

@levBagryansky yes, that's true. But it's better to be slow instead of being incorrect.

[levBagryansky]

@maxonfjvipon could you please assign me

[levBagryansky]

@yegor256 So do we synchronize every file separately? File should be accessed for reading by multiple processes but locked for all other processes when one is editing it.

[levBagryansky]

@yegor256 This task is more difficult than it might seem: 1) The cache is accessed in different parts of the code, and using different classes. For example, here's how OptCached works with the cache:

        final String name = xml.xpath("/program/@name").get(0);
        try {
            final XML optimized;
            if (this.contains(name)) {      // Here we making sure that the cached object exists and is valid. 
                optimized = new XMLDocument(this.cached(name));    // Here we use the assumption above, but it can be invalid already. Such cases can be too much.
            } else {
...
    private boolean contains(final String name) throws IOException {
        final Path cache = this.cached(name);
        final boolean res;
        if (Files.exists(cache)) {
            res = !Files
                .getLastModifiedTime(cache)
                .toInstant()
                .isBefore(
                    Files.getLastModifiedTime(this.path)
                        .toInstant()
                    );
        } else {
            res = false;
        }
        return res;
    }

Accessing the cache occurs at different points in the program, when working with the same file, You can't just lock the file with any access to it, you need to correctly find the boundaries of working with it. It turns out that to solve the problem, you need to insert a lock in all places where the cache is used, and when modifying the code, keep an eye on this. It's complicated and inelegant. 2) The cache usage cases are too blurry. Now it contains both what really should be in the global cache and what would be more correct to store in the local cache:

tardis3@R:~$ cd .eo
tardis3@R:~/.eo$ ls verified/3b62157/org/eolang/
abcd-tests.xmir    heap-tests.xmir               ram.xmir
as-phi-tests.xmir  heap.xmir                     runtime-tests.xmir
as-phi.xmir        int-tests.xmir                rust-tests.xmir
bool-tests.xmir    int.xmir                      rust.xmir
bool.xmir          io                            seq-tests.xmir
bytes-tests.xmir   memory-tests.xmir             seq.xmir
bytes.xmir         memory.xmir                   snippets
cage-tests.xmir    nan-tests.xmir                string-tests.xmir
cage.xmir          nan.xmir                      string.xmir
cti-test.xmir      negative-infinity-tests.xmir  switch-tests.xmir
cti.xmir           negative-infinity.xmir        switch.xmir
error.xmir         nop-tests.xmir                try-tests.xmir
float-tests.xmir   nop.xmir                      try.xmir
float.xmir         positive-infinity-tests.xmir  tuple-tests.xmir
goto-tests.xmir    positive-infinity.xmir        tuple.xmir
goto.xmir          ram-tests.xmir                unit-tests.xmir
tardis3@R:~/.eo$ 

I added the eo-runtime/src/test/eo/org/eolang/abcd-tests.eo file to my local repository and started mvn clean install after that , a file appeared in the cache .eo/verified/3b62157/org/eolang/abcd-tests.xmir. It seems to me that only downloadable dependencies should be added to this cache, and a local cache would be enough for user objects. It seems to me that we are using the cache for the wrong purposes, which makes it difficult to synchronize. By the way, if the cache could only be changed by adding new files there- for example, objects of next versions, then it would not need to be synchronized at all.

[yegor256]

@maxonfjvipon since you worked with the caching, maybe you can fix this one too