search

ccxvii / mujs

An embeddable Javascript interpreter in C.
http://mujs.com/
ISC License
812 stars 98 forks source link

Test harness doesn't work when multiple testcases are run #147

Closed yurivict closed 3 years ago

yurivict commented 3 years ago

I am trying to follow this method to run tests: https://github.com/ccxvii/mujs/issues/49#issuecomment-416935229

Individual cases run:

$ (echo test/suite/ch15/15.3/S15.3_A3_T5.js) | mujs ../mujs-harness.js
test/suite/ch15/15.3/S15.3_A3_T5.js

Total: 1 
Pass: 1 
Failed: 0
$ (echo test/suite/ch15/15.3/S15.3_A3_T6.js) | mujs ../mujs-harness.js
test/suite/ch15/15.3/S15.3_A3_T6.js

Total: 1 
Pass: 1 
Failed: 0

but they fail when both cases are run sequentially:

$ (echo test/suite/ch15/15.3/S15.3_A3_T5.js ; echo test/suite/ch15/15.3/S15.3_A3_T6.js) | mujs ../mujs-harness.js
test/suite/ch15/15.3/S15.3_A3_T5.js
test/suite/ch15/15.3/S15.3_A3_T6.js
ERROR: test/suite/ch15/15.3/S15.3_A3_T6.js 
    #1: 

Total: 2 
Pass: 1 
Failed: 1

Is there an updated test harness, or what is wrong with the test harness?

avih commented 3 years ago

what is wrong with the test harness?

It's probably bitrotten and doesn't work anymore with current mujs.

Is there an updated test harness?

See https://github.com/ccxvii/mujs/issues/49#issuecomment-821925719

yurivict commented 3 years ago

Would it be better to not apply special patches to MuJS but run the mujs process separately for each test?

avih commented 3 years ago

I'm not sure what you're saying/suggesting/implying. The patchset which I posted at the other issue do run each test in a new mujs instance because otherwise (like it was initially with one mujs instance for all tests) it causes more issues.

yurivict commented 3 years ago

Your patch includes patch to main.c. If tests are run in individual sessions there should be no need to patch C, no?

Test harness and instructions how to run it should be committed into the repository.

avih commented 3 years ago

Sorry, I don't really understand what you're trying to say. Are you trying to clarify something for yourself? are you saying the patch has an issue or problem? are you making some suggestion?

Here's what I suggest:

And then if you have questions, or suggestions, or you notice issues, then please ask again, clearly.

yurivict commented 3 years ago

Please do not be condescending.

I am sorry that your low level of intellect doesn't allow you an oppotunity to comprehend more advanced matters.

Perhaps you should quit commenting here and change to something more accessible to you.

avih commented 3 years ago

Glad you could vent off. When you find the time to ask a question clearly to help me understand better what you want, I'll try to answer as best as I can.