Testing sutil app

[marcingolenia]

Hi there! Thank You for great work on Sutil. I wanted to develop small app using beta version but I struggle to setup tests :( I am hoping to get some help.

1. I've stolen TestFramework.fs from Sutil repository.
2. The test looks like this;

   
   module Tests.Tests

open TestFramework open Sutil

let tests = testList "Sutil.DOM" [

// Simplest case
testCase "Hello World" <| fun () ->
    let app =
        Html.div "Hello World"

    mountTestApp app

    Expect.queryText "div" "Hello World"
    Expect.areEqual(1, 1)

]

3. Commands I use (stolen from sutil repo):

"tests-console": "mocha dist/tests -r esm",
"watch:tests": "dotnet fable watch tests -o dist/tests --runWatch mocha dist/tests -r esm",
"tests": "dotnet fable tests --run webpack --config webpack.config.tests.js",
"start:tests": "dotnet fable watch tests --run webpack serve --config webpack.config.tests.js "

The best what I get after running `start:tests` is attached on the screen:
![image](https://user-images.githubusercontent.com/5363851/136210491-6215487c-ed5d-4369-88f0-6735ab67b7ad.png)
So it is empty. When I run tests:watch I get:
```bash
> watch:tests
> dotnet fable watch tests -o dist/tests --runWatch mocha dist/tests -r esm

Fable: F# to JS compiler 3.2.9
Thanks to the contributor! @JacobChang
tests> dotnet restore Tests.fsproj
  Determining projects to restore...
  All projects are up-to-date for restore.
Parsing tests/Tests.fsproj...
Initializing F# compiler...
Compiling tests/Tests.fsproj...
F# compilation finished in 5621ms
Fable compilation finished in 1392ms
.> node_modules/.bin/mocha dist/tests -r esm
Watching .
logging:init defaults

  0 passing (1ms)

When I run test-console I get;

> tests-console
> mocha dist/tests -r esm

logging:init defaults

  0 passing (0ms)

my webpack.config.tests.js looks like this;

var path = require("path");

module.exports = {
    mode: "development",
    entry: "./tests/Tests.fs.js",
    output: {
        path: path.join(__dirname, "./tests/public"),
        filename: "test-bundle.js",
    },
    devServer: {
        contentBase: "./tests/public",
        port: 8080,
    },
}

I have the repo here: https://github.com/marcingolenia/kabanos if You need more information.

Can You give me some hints on how to move forward with this? Thanks in advance :)

[AngelMunoz]

Looks like you might be missing the call to these tests, check tests/AllTests.fs

let main() =
    runTests
        [ Test.Binding.tests
          Test.Store.tests
          Test.Observable.tests
          Test.DOM.tests ]

main()

you might need to add these into your app, I'll try to take a look later but that would be my first guess without looking too much into it

[marcingolenia]

:D YES! That did it. The command start:tests works. To be 100% happy I need to run this tests in console as well. When I run tests-console I get this;

npm run tests-console

> tests-console
> mocha dist/tests -r esm

logging:init defaults
Running tests
Test Suite

/home/mgolenia/projects/kabanos/app/dist/tests/TestFramework.js:1
ReferenceError: document is not defined
    at log (/home/mgolenia/projects/kabanos/app/dist/tests/TestFramework.js:85:20)
    at logH (/home/mgolenia/projects/kabanos/app/dist/tests/TestFramework.js:113:5)
    at runTests (/home/mgolenia/projects/kabanos/app/dist/tests/TestFramework.js:215:5)
    at main (/home/mgolenia/projects/kabanos/app/dist/tests/Run.js:9:5)
    at Object.<anonymous> (/home/mgolenia/projects/kabanos/app/dist/tests/Run.js:12:1)
    at Generator.next (<anonymous>)
    at Object.exports.requireOrImport (/home/mgolenia/projects/kabanos/app/node_modules/mocha/lib/esm-utils.js:42:12)
    at Object.exports.loadFilesAsync (/home/mgolenia/projects/kabanos/app/node_modules/mocha/lib/esm-utils.js:55:34)
    at Mocha.loadFilesAsync (/home/mgolenia/projects/kabanos/app/node_modules/mocha/lib/mocha.js:473:19)
    at singleRun (/home/mgolenia/projects/kabanos/app/node_modules/mocha/lib/cli/run-helpers.js:125:15)
    at exports.runMocha (/home/mgolenia/projects/kabanos/app/node_modules/mocha/lib/cli/run-helpers.js:190:10)
    at Object.exports.handler (/home/mgolenia/projects/kabanos/app/node_modules/mocha/lib/cli/run.js:362:11)
    at /home/mgolenia/projects/kabanos/app/node_modules/mocha/node_modules/yargs/build/index.cjs:443:71
    at process.runNextTicks [as _tickCallback] (internal/process/task_queues.js:62:5)
    at internal/main/run_main_module.js:17:47

Do You have more ideas ? :)

[davedawkins]

I started out with console tests, and found that they weren't rich enough for me to check the DOM-related functionality, so I developed a second set of tests that run in the browser.

This means that the Mocha tests are probably out of date. I will check them out later and see if I can bring everything back together under Mocha, perhaps with a headless browser.

I need my tests to run in GH, so this is worth doing.

[AngelMunoz]

you might hook up or add the missing bits and pieces to adapt @web/test-runner like Lit.Test it precisely uses a hedless browser https://github.com/fable-compiler/Fable.Lit/tree/main/src/Lit.Test

cc @alfonsogarciacaro might have some insight

[alfonsogarciacaro]

Yes, I'm trying to improve the experience of testing Fable apps in a headless browser with Lit.Test, you can read about it here: https://fable.io/Fable.Lit/docs/testing.html

For now, Lit.Test has a dependency on Lit, but you can use it to test anything on the DOM. For example, here with the browser bindings: https://github.com/fable-compiler/fable-browser/blob/master/test/EventTest.fs

[marcingolenia]

Thanks a lot. This looks good! I will try to bend this to my will ;)

[davedawkins]

@alfonsogarciacaro does the headless testing part actually require Lit? From that file, it would appear not.

[alfonsogarciacaro]

@davedawkins Lit is only required for the methods that renders HTML and puts it in a disposable container in the DOM. Maybe we could just extract those methods and have a Fable.Expect library that it's independent of Lit: https://github.com/fable-compiler/Fable.Lit/blob/e5f24e282160dbd8a7fd69e79b58592192c76aaf/src/Lit.Test/Expect.Dom.fs#L97-L115

Note that the Expect.XXX modules are only utilities for assertion and dealing with the DOM or Elmish. The headless part is done by Web Test Runner. This file contains bindings for working with WTR (like declaring the tests or doing snapshots): https://github.com/fable-compiler/Fable.Lit/blob/e5f24e282160dbd8a7fd69e79b58592192c76aaf/src/Lit.Test/WebTestRunner.fs

[davedawkins]

Maybe we could just extract those methods and have a Fable.Expect library that it's independent of Lit

Exactly what I was thinking.

I'm right now looking at WebTestRunner.fs and @web/test-runner.

[alfonsogarciacaro]

Exactly what I was thinking.

Here you go :) https://github.com/fable-compiler/Fable.Expect

[davedawkins]

Amazing, thank you!

[marcingolenia]

Sorry for bothering You again. I am total noob with Fable stuff. So, I have added this script; "test": "dotnet fable tests -o build/tests --run web-test-runner build/tests/*Test.js --node-resolve", installed two dependencies;

    "@web/test-runner": "^0.13.18",
    "@web/test-runner-commands": "^0.5.13",

Created a whatever test:

module Tests

open Expect
open Expect.Dom
open Sutil
open WebTestRunner

describe "LitElement" <| fun () ->
    it "whatever works" <| fun () -> promise {
        use container = Container.New()
        container.El.innerHTML <- "whatever works."
        container.El |> Expect.innerText "whatever works."
    }

run the test and it passed :)

Fable compilation finished in 1112ms
.> node_modules/.bin/web-test-runner build/tests/FirstTest.js --node-resolve

Chrome: |██████████████████████████████| 1/1 test files | 1 passed, 0 failed

Finished running tests in 0.4s, all tests passed! 🎉

Now I try to mount the sutil app.

module Tests

open Expect
open Expect.Dom
open Sutil
open Browser
open WebTestRunner

let createContainer (tagName: string) =
    let el = document.createElement(tagName)
    document.body.appendChild(el) |> ignore
    { new Container with
        member _.El = el
        member _.Dispose() = document.body.removeChild(el) |> ignore }

describe "LitElement" <| fun () ->    
    it "element renders" <| fun () -> promise {
        use container = createContainer("""<div id="sutil-app"></div>""")
        // Program.mountElement "sutil-app" App.app
        Program.mountElementOnDocument container.El.ownerDocument "sutil-app" App.app
        container.El |> Expect.innerText "Hello World from sutil."
    }

Probably everything in the test is wrong :) I tried different combinations and I am not sure how to mount sutil app into the container. Current error:

 🚧 Browser logs:
      logging:init defaults
      TypeError: Cannot read properties of null (reading 'ownerDocument')

Any further hints appreciated! Thank You for Fable.Expect :) If You need more info, the repo is here; https://github.com/marcingolenia/kabanos Is it possible to mount sutil on HTMLElement, not document? I didn't find anything like that.

[davedawkins]

Give me an hour or so and I'll update, here's the result of my test, looking good. I'll push a new Sutil with the test-headless folder, and hopefully that will get you going

[davedawkins]

@marcingolenia I've pushed these changes:

https://github.com/davedawkins/Sutil/tree/main/tests-headless

• Only the HelloWorldTest.fs is there
• The original tests have not been converted yet
• I'm still not sure what @alfonsogarciacaro 's intention is for testing DOM structure, but we'll figure it out. You can use container.El.getSelector for simple tests, since it will throw if not found, but it doesn't look explicitly like an "Expect.XX" test. We can suggest additions to Expect.Dom, but there's enough there to get started.

To run these tests:

$ cd tests-headless
$ npm install
$ npm run test

I'll convert the existing tests over next. Ideally I want to be able to switch between head-less and head-full browsers.

[alfonsogarciacaro]

The idea is to use accessible queries so by writing the tests you also make sure your UI is accessible: https://github.com/fable-compiler/Fable.Expect#accessible-queries

Accessible queries will throw if they don't find anything although not an Assertion error. If you want to turn them into assertions you can use Expect.success.

container.El
|> Expect.success "new todo found" (fun el -> el.getByText(newTodo))

You can use Expect.successAnd to chain assertions.

container.El
|> Expect.successAnd "value display" (fun el -> el.getByText("value))
|> Expect.innerText "Value: Foo"

If you just want to make sure your UI doesn't change unexpectedly you can use a snapshot test:

do! container.El |> Expect.matchHtmlSnapshot "new-todo"

[davedawkins]

Yes, and that's what app developers should be doing for sure. This is probably what @marcingolenia needs.

As a framework developer, I need to make sure that the DSL and bindings have the right side effects in the DOM. The accessible queries will help up to a certain level of detail, but not quite enough. (For example, ordering of elements etc).

In any case, this logged issue, and your assistance, is aimed at the "end-app-developer", so we're in good shape in that regard.

[alfonsogarciacaro]

Well, at the end, container.El is just and HTMLElement (the accessible query methods are implemented as extensions) so you can still use .querySelector or any other native method to inspect the DOM :)

[marcingolenia]

Hi! :) So the tests like the one You've posted @davedawkins works in my solution as well. The problem is when I try to tests a Sutil Component which comes from another project. The tests looks like this now; Tests.fsproj ->

    it "element renders" <| fun () -> promise {
        use container = Container.New()
        DOM.mountOn (App.app()) container.El |> ignore
        container.El |> Expect.innerText "Hello World from sutil."
    }

The application: App.fsproj ->

module App

open Sutil

let app() =
    Html.div "Hello World from sutil."

So complex stuff :D If I run the tests I get;

build/tests/FirstTest.js:

 🚧 Browser logs:
      logging:init defaults
      TypeError: Cannot read properties of null (reading 'ownerDocument')
        at makeContext (build/tests/.fable/Sutil.1.0.0-beta-011/DOM.fs.js:1677:36)
        at mountOn (build/tests/.fable/Sutil.1.0.0-beta-011/DOM.fs.js:2241:23)
        at MountPoint__Mount (build/tests/.fable/Sutil.1.0.0-beta-011/Program.fs.js:23:12)
        at mountElementOnDocument (build/tests/.fable/Sutil.1.0.0-beta-011/Program.fs.js:41:10)
        at mountElement (build/tests/.fable/Sutil.1.0.0-beta-011/Program.fs.js:45:5)
        at build/tests/src/App.js:9:1

the command: "test": "dotnet fable tests -o build/tests --run web-test-runner build/tests/*Test.js --node-resolve",

[marcingolenia]

So if a copy the "html" from the App module like this;

    it "element renders" <| fun () -> promise {
        use container = Container.New()
        DOM.mountOn (Html.div "Hello World from sutil.") container.El |> ignore
        container.El |> Expect.innerText "Hello World from sutil."
    }

I get:

build/tests/FirstTest.js:

 🚧 Browser logs:
      logging:init defaults

Chrome: |██████████████████████████████| 1/1 test files | 2 passed, 0 failed

Finished running tests in 0.6s, all tests passed! 🎉

We are close!

[davedawkins]

Let me take a look. Can you submit a project repository link that gives me a test case to work with?

[marcingolenia]

Sure; https://github.com/marcingolenia/kabanos. It's almost empty; the file with tests: FirstTest.fs.

[davedawkins]

I've reproduced it with your repo, so looking into it now

[davedawkins]

The line that's failing is where you mount the app in App.fs:

app() |> Program.mountElement "sutil-app"

In the test environment, the node with ID "sutil-app" doesn't exist and so we get the error. If you comment that out, your test works.

Going forward, you might want to consider a couple of ways to arrange your project, so that running the tests doesn't try to "run" (mount) the main application.

1. Test for existence of "sutil-app" - use this as a check to decide if you're running as a test or not

2. Separate your app components into a separate project, and run the tests against that. You can then have a minimal "App" project that instantiates and mounts the main App() class.

I hope this helps!

[marcingolenia]

omg ;p I didn't think that that the Program.mountElement will actually run - I wanted to use App.app only but yup - this is a module and the code is actually being executed.. My bad! Thank You Dave.

I have moved the "app" component to separate file, and introduced Program.fs which executes the mounting + changed the entry in webpack; entry: "./src/Program.fs.js",.

Let me play a little bit with Sutil and Fable.Expect now, I feel obliged to make PR with "Testing" to the sutil documentation page.

[marcingolenia]

Now this is a good start for an app guided by tests!

module Tests

open Expect
open Expect.Dom
open WebTestRunner

let render _component =
    let container = Container.New()
    Sutil.DOM.mountOn _component container.El |> ignore
    container

describe "Testing!" <| fun () ->
    it "element renders" <| fun () -> promise {
        use sut = render App.app
        sut.El |> Expect.innerText "Hello World from sutil."
    }

I know this is nothing yet, but feels good already.

[davedawkins]

Well done Marcin! This has been good for me too, I finally have headless tests, thanks to you and @alfonsogarciacaro.