All tests successful on my machine, but FAIL or TIMEOUT on the website.

[iago-lito]

This happens on my 4th iteration to Forth in the Rust track: https://exercism.org/tracks/rust/exercises/forth/solutions/iago-lito

What could be wrong?

[iHiD]

I imagine your solution takes longer than 15s to run, so we time it out.

[iago-lito]

Well, this is not the case either on my machine, although I agree this is more difficult to establish formally :)

$ cargo clean
$ time cargo test
   Compiling forth v1.7.0 (/home/iago-lito/snap/exercism/5/exercism/rust/forth)
    Finished test [unoptimized + debuginfo] target(s) in 1.39s
     Running unittests (target/debug/deps/forth-2b917c8ababc0bca)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/forth.rs (target/debug/deps/forth-dca3b30c7b8757e2)

running 45 tests
test addition_error ... ok
test addition_and_subtraction ... ok
test calling_non_existing_word ... ok
test can_add_two_numbers ... ok
test can_consist_of_built_in_words ... ok
test can_define_word_that_uses_word_with_the_same_name ... ok
test can_multiply_two_numbers ... ok
test can_divide_two_numbers ... ok
test can_subtract_two_numbers ... ok
test defining_a_number ... ok
test can_use_different_words_with_the_same_name ... ok
test definitions_after_ops ... ok
test definitions_are_case_insensitive ... ok
test division_error ... ok
test drop ... ok
test drop_case_insensitive ... ok
test drop_error ... ok
test drop_with_two ... ok
test dup ... ok
test dup_case_insensitive ... ok
test dup_error ... ok
test dup_top_value_only ... ok
test errors_if_dividing_by_zero ... ok
test execute_in_the_right_order ... ok
test malformed_word_definition ... ok
test multiple_definitions ... ok
test multiplication_and_division ... ok
test multiplication_error ... ok
test no_input_no_stack ... ok
test numbers_just_get_pushed_onto_the_stack ... ok
test over ... ok
test over_case_insensitive ... ok
test over_error ... ok
test over_with_three ... ok
test performs_integer_division ... ok
test redefining_a_built_in_operator ... ok
test redefine_an_existing_word_with_another_existing_word ... ok
test redefining_an_existing_built_in_word ... ok
test redefining_an_existing_word ... ok
test subtraction_error ... ok
test swap ... ok
test swap_case_insensitive ... ok
test swap_error ... ok
test swap_with_three ... ok
test user_defined_words_are_case_insensitive ... ok

test result: ok. 45 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests forth

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

real    1,60s
user    2,31s
sys 0,38s

My concern also stems from the fact that the trouble maybe started when I switched browser to access exercism.org and had to log again.. maybe while this test was still running on the previous browser.. so maybe I broke something?

[masahx]

this happened to me again with Java... sometimes first build / test takes longer locally but after that it is fast enough

it is working solution

[iHiD]

The tests don't run in the browser, so that won't be related.

This is the raw output (I'm not sure why it then times out).

   Compiling forth v1.7.0 (/mnt/exercism-iteration)
    Finished test [unoptimized + debuginfo] target(s) in 3.39s
     Running unittests (target/debug/deps/forth-2b917c8ababc0bca)
     Running tests/alloc-attack.rs (target/debug/deps/alloc_attack-15d39a8ca4ebef4b)
error: test failed, to rerun pass '--test alloc-attack'

[iHiD]

@ErikSchierboom Could you run this locally and see what happens pls?

It's Rust/Forth. Code is:

use std::{collections::HashMap, mem};

pub type Value = i32;
pub type ForthResult = Result<(), Error>;

#[derive(Default)]
pub struct Forth {
    // Store the language manipulable stack.
    stack: Vec<Value>,
    // And the list of user-defined words: Word ↦ Definition (list of words)
    dict: HashMap<String, Vec<String>>,
    // Choose not to keep references to the program input
    // because it might be transient: copy the needed data to own it.
}

#[derive(Debug, PartialEq)]
pub enum Error {
    DivisionByZero,
    StackUnderflow,
    UnknownWord,
    InvalidWord,
}

// Very basic automaton state.
#[derive(PartialEq)]
enum ParseMode {
    Execution,       // Executing instructions.
    UserDefinedWord, // Expecting new user-defined word.
    Definition,      // Reading definition of user-defined word.
}
struct ParseState {
    mode: ParseMode,
    // Move user-defined words and their definition into these temporary variables.
    // Before actually recording them to the dict.
    user_defined_word: String,
    definition: Vec<String>,
}

impl Forth {
    pub fn new() -> Forth {
        Forth {
            stack: Vec::new(),
            dict: HashMap::new(),
        }
    }

    pub fn stack(&self) -> &[Value] {
        self.stack.as_ref()
    }

    /// # Errors
    ///
    /// Returns error on parsing or evaluation failure.
    pub fn eval(&mut self, input: &str) -> ForthResult {
        // Start with a low-level process lexing input character by character
        // to construct the successive tokens of the language.
        // When a new token is constructed, it is immediately sent to the `parse` method
        // responsible for interpreting them.

        // The parser state is regularly updated by the `parse` method.
        let mut parser = ParseState {
            mode: ParseMode::Execution,
            user_defined_word: String::new(),
            definition: Vec::new(),
        };

        // Construct current token in this variable.
        let mut token = String::new();
        for c in input.chars() {
            match (c, token.is_empty()) {
                (' ', false) => {
                    // One token is complete, pass it to the parsing level.
                    self.parse(&token, &mut parser)?;
                    // And consume.
                    token.clear();
                }
                (' ', true) => {
                    // Ignore additional blank space.
                }
                _ => {
                    // Just keep constructing the new token.
                    token.extend(c.to_lowercase());
                }
            }
        }
        // Process last token if there is any left.
        if !token.is_empty() {
            self.parse(&token, &mut parser)?;
        }
        // Check parser state consistency:
        if parser.mode == ParseMode::Execution {
            Ok(())
        } else {
            Err(Error::InvalidWord)
        }
    }

    // This second stage interpret tokens with a small finite state machine
    // and is responsible for handling user-defined words definitions.
    // When there is nothing left to define,
    // it passes the remaining tokens to the actual execution process
    // in the `execute` method.
    fn parse(
        &mut self,
        token: &str,
        ParseState {
            ref mut mode,
            ref mut user_defined_word,
            ref mut definition,
        }: &mut ParseState,
    ) -> ForthResult {
        use ParseMode::{Definition, Execution, UserDefinedWord};
        match (token, &mode) {
            (":", Execution) => {
                // Prepare for receiving a user-defined word.
                *mode = UserDefinedWord;
            }
            (user_word, UserDefinedWord) => {
                // The user-defined word is complete, start recording definition.
                if user_word.parse::<Value>().is_ok() {
                    // The user defined word cannot be a number.
                    return Err(Error::InvalidWord);
                }
                *user_defined_word = String::from(user_word);
                *mode = Definition;
            }
            (";", Definition) => {
                // The definition is completed, record and move on.
                self.dict
                    .insert(mem::take(user_defined_word), mem::take(definition));
                *mode = Execution;
            }
            (def_word, Definition) => {
                // One new word is added to current definition.
                // Words in the definition right-hand side seem to be expanded
                // at the time of definition and not at the time of execution.
                // So expand them now before actually recording the definition.
                // ASSUME that words in the RHS are always defined prior to the definition,
                // so only one replacement pass is needed.
                if let Some(replace) = self.dict.get(def_word) {
                    definition.extend_from_slice(replace);
                } else {
                    definition.push(String::from(def_word));
                }
            }
            (word, Execution) => {
                // This token should now have a meaning to update the language state.
                self.execute(word)?;
            }
        }
        Ok(())
    }

    // This higher-level method processes every *word* one by one
    // to update the language stack.
    fn execute(&mut self, word: &str) -> ForthResult {
        use Error::{DivisionByZero, StackUnderflow, UnknownWord};

        // Is this word customly defined by user?
        if let Some(definition) = self.dict.get(word) {
            // Recurse to use the definition instead.
            let copy = definition.clone(); // Clone to ignore redefinitions during recursion.
            for w in &copy {
                self.execute(w)?;
            }
        } else {
            macro_rules! binary_operator {
                ( $a:ident $b:ident : $eval:expr $(; $extra:stmt)? ) => {
                    // Consume two operands from the stack.
                    if let (Some($a), Some($b)) = (self.stack.pop(), self.stack.pop()) {
                        $($extra)?
                        // Compute and push result on the stack.
                        self.stack.push($eval);
                    } else {
                        return Err(StackUnderflow);
                    }
                };
            }

            match word {
                "+" => binary_operator!(a b : b + a),
                "-" => binary_operator!(a b : b - a),
                "*" => binary_operator!(a b : b * a),
                "/" => binary_operator!(a b : b / a; if a == 0 { return Err(DivisionByZero); }),
                "dup" => {
                    // Duplicate last element on the stack.
                    if let Some(&top) = self.stack.last() {
                        self.stack.push(top);
                    } else {
                        return Err(StackUnderflow);
                    }
                }
                "drop" => {
                    // Drop last element in the stack.
                    if self.stack.pop().is_none() {
                        return Err(StackUnderflow);
                    }
                }
                "swap" => {
                    // Swap the two top elements in the stack.
                    let l = self.stack.len();
                    if l >= 2 {
                        self.stack.swap(l - 1, l - 2);
                    } else {
                        return Err(StackUnderflow);
                    }
                }
                "over" => {
                    // Duplicate the element before last in the stack.
                    let l = self.stack.len();
                    if l >= 2 {
                        self.stack.push(self.stack[l - 2]);
                    } else {
                        return Err(StackUnderflow);
                    }
                }
                _ => {
                    if let Ok(value) = word.parse::<Value>() {
                        // If the word is valid number, just push it on the stack.
                        self.stack.push(value);
                    } else {
                        // Otherwise it's unsure what to do with it.
                        return Err(UnknownWord);
                    }
                }
            }
        }
        Ok(())
    }
}

[iHiD]

@iago-lito Are you using any external crates in this solution?

[ErikSchierboom]

I've tried running this locally on my machine both with and without Docker. Without Docker it runs smoothly in 2.5 seconds. With Docker it times out after a minute or so. I'm transferring this to exercism/rust-test-runner, as it seems to be related to the test runner itself.

[iago-lito]

Are you using any external crates in this solution?

No I'm not. Thank you for investigating this :) For some reason I didn't get the github notification about progress on this issue, I'll try to keep updated. Feel free to ask for more detail if you need.

[senekor]

I have tested the solution locally. It does indeed time out. This exercise is special, it has two separate test files. The test timing out is in the non-standard file. It is also ignored by default, as most tests. So my guess is that this specific test was not un-ignored locally.

@iago-lito this issue is kinda old, but the special test is actually important because it forces the solution be algorithmically efficient. So, if you're still up for the challenge... ;-)

Closing due to age, but feel free to ping if you have any other questions.

[iago-lito]

@senekor Thank you for reviving / closing this ;) How should I've been able to tell that this special test file existed since cargo test did not tell?

[senekor]

Yeah, it's difficult to tell. I think I was confused myself when I first solved the exercise. cargo test technically does tell you how many tests were skipped. But it's visually very hard to notice. The output of cargo-nextest is better, if you're interested in that.

We might consider adding a note about this in the exercise instructions.