NorfairKing / sydtest

A modern testing framework for Haskell with good defaults and advanced testing features.
115 stars 25 forks source link

sydtest-webdriver test fails because /dev/null doesn't exist? #51

Closed L7R7 closed 2 years ago

L7R7 commented 2 years ago

I'm trying to get a minimal example of a webdriver to work. However, my test fails with the following error message:

/dev/null: withBinaryFile: does not exist (No such file or directory)

I can reproduce that in my CI here: https://github.com/L7R7/haskell-incubator/runs/6631633650?check_suite_focus=true#step:5:12960 My local setup is similar to the one in CI (Ubuntu 22.04), and I can't really see why the test fails. I'm aware that the tests should fail in CI because there's no selenium installed. On my machine I have it installed and get the same error, so I would say that this is not the root cause.

My guess would be that the error comes from here: https://github.com/NorfairKing/sydtest/blob/master/sydtest-webdriver/src/Test/Syd/Webdriver.hs#L217 I'm confused. Am I missing something?

NorfairKing commented 2 years ago

I have no clue to be honest. Can you show me your example? Is it reproducible?

L7R7 commented 2 years ago

Yes, it's reproducible. My code is here: https://github.com/L7R7/haskell-incubator/blob/webdriver/polysemy-servant/test/HibWebSpec.hs I just tired running the webdriver tests in sydtest-webdriver and there it fails complaining about missing selenium-webdriver. So it's at least different. The only difference between my code and the code in sydtest-webdriver is that my application is wrapped inside an IO action

L7R7 commented 2 years ago

With a quick

applicationSetupFunc' :: IO Application -> SetupFunc PortNumber
applicationSetupFunc' application = SetupFunc $ \func ->
  Warp.testWithApplication (application) $ \p ->
    func (fromIntegral p)

I don't get this error message anymore, but instead it just hangs. I'll keep digging...

NorfairKing commented 2 years ago

@L7R7 You probably want to use sydtest-wai...

L7R7 commented 2 years ago

Yes, I do want that. The tests in systest-webdriver use this: https://github.com/NorfairKing/sydtest/blob/master/sydtest-wai/src/Test/Syd/Wai/Def.hs#L89-L93

What I posted above is exactly that but takes an IO Application instead of Application (should this be part of sydtest-wai?)

I currently believe that the issue is somewhere around the config of selenium-server. I'll try and get some debug logging out of that soon...

NorfairKing commented 2 years ago

What I posted above is exactly that but takes an IO Application instead of Application (should this be part of sydtest-wai?)

No because SetupFuncs already compose.

I currently believe that the issue is somewhere around the config of selenium-server. I'll try and get some debug logging out of that soon...

Check the nix shell and the nix overlay for hints :D

L7R7 commented 2 years ago

I'm closing this for now as it's not an issue with sydtest but with my setup. I gave up for now and will come back to it when I either have more time and patience or when I know how to nix.

For future reference, this might be related: https://github.com/kallisti-dev/hs-webdriver/issues/186

disconsis commented 2 months ago

I was getting the same error when selenium-server was not available. Fixed it by installing. The error mentioning /dev/null is really weird.