Tests seem to cause side effects that other tests depend on

[CharlieTap]

Global.wast has the following functions defined:

  (func (export "get-z1") (result i32) (global.get $z1))
  (func (export "get-z2") (result i64) (global.get $z2))

These point at imports from the global script host module:

  (global (import "spectest" "global_i32") i32)
  (global (import "spectest" "global_i64") i64)

The tests assert values that are set as part of other wast files

(assert_return (invoke "get-z1") (i32.const 666))
(assert_return (invoke "get-z2") (i64.const 666))

This causes this test to be:

• dependent on the order in which the other tests run
• non parallelisable
• incompatible with incremental compilers of the test suite
• fragile and hard to grep

A quick fix could be resetting any globals between wast scripts?

[CharlieTap]

I realised that these are the default values for the globals:

(global $global_i32 (export "global_i32") (mut i32) (i32.const 666)) (global $global_i64 (export "global_i64") (mut i64) (i64.const 666)) (global $global_f32 (export "global_f32") (mut f32) (f32.const 666.6)) (global $global_f64 (export "global_f64") (mut f64) (f64.const 666.6))

It might be worth updating the docs with this as it would have saved me a fair bit of trial and error 😬

[rossberg]

Yeah, it's on my wish/todo list for long time to get rid of spectest altogether. FWIW, there reference interpreter's implementation of the spectest imports is here.

That said, any test for anything related to linking and cross-module semantics will necessarily be order-dependent and generally require cross-module side effects.