search

tmaslen / jasminejsx

A port of Jasmine 2.2.0 that is compatible with Adobe ExtendScript
Apache License 2.0
10 stars 3 forks source link

Is there a limit to the number of specs that can be run with jasmine.js in ExtendScript? #3

Closed shawnpyle closed 5 years ago

shawnpyle commented 5 years ago

Hello @tmaslen, Thanks for your work on this project! It's been helpful getting up and running with ExtendScript.

I'm working on a new Adobe CEP module and I'd love to use this library to test my host JSX. However, I'm seeing something odd and was hoping you might be able to point me in the right direction, or at this point, any direction.

You can see my updates at https://github.com/shawnpyle/jasminejsx. Essentially, I've removed the need for the Toolkit by using Apple JavaScript for automation and using a custom LogReporter (slight modifications to your JsxrReporter. However, I'm seeing that if I have a number of specs (31, see spec/SpecLength.jsx for more details) output fails to be logged.

Results looks like this:

spyle@409 jasminejsx|master$ ./specRunner.sh
Starting ExtendScript specs...
Be sure you are signed into the Creative Cloud app or you'll get no output from the test suite.
1|2019-09-04T22:52:59|INFO|***************************
2|2019-09-04T22:52:59|INFO|Jasmine ExtendScript Runner
3|2019-09-04T22:52:59|INFO|***************************
Time to complete: 1.668 seconds

but I would expect it to look like this:

spyle@409 jasminejsx|master$ ./specRunner.sh
Starting ExtendScript specs...
Be sure you are signed into the Creative Cloud app or you'll get no output from the test suite.
1|2019-09-04T22:53:99|INFO|***************************
2|2019-09-04T22:53:00|INFO|Jasmine ExtendScript Runner
3|2019-09-04T22:53:00|INFO|***************************
4|2019-09-04T22:53:09|INFO|SpecLength jasmine-2.2.0 it1
5|2019-09-04T22:53:18|INFO|  Failed
6|2019-09-04T22:53:27|INFO|
7|2019-09-04T22:53:27|INFO|SpecLength jasmine-2.2.0 it2
8|2019-09-04T22:53:27|INFO|  Failed
9|2019-09-04T22:53:28|INFO|
10|2019-09-04T22:53:28|INFO|SpecLength jasmine-2.2.0 it3
11|2019-09-04T22:53:28|INFO|  Failed
12|2019-09-04T22:53:28|INFO|
13|2019-09-04T22:53:29|INFO|SpecLength jasmine-2.2.0 it4
14|2019-09-04T22:53:29|INFO|  Failed
15|2019-09-04T22:53:29|INFO|
16|2019-09-04T22:53:29|INFO|SpecLength jasmine-2.2.0 it5
17|2019-09-04T22:53:30|INFO|  Failed
18|2019-09-04T22:53:30|INFO|
19|2019-09-04T22:53:30|INFO|SpecLength jasmine-2.2.0 it6
20|2019-09-04T22:53:30|INFO|  Failed
21|2019-09-04T22:53:31|INFO|
22|2019-09-04T22:53:31|INFO|SpecLength jasmine-2.2.0 it7
23|2019-09-04T22:53:31|INFO|  Failed
24|2019-09-04T22:53:31|INFO|
25|2019-09-04T22:53:31|INFO|SpecLength jasmine-2.2.0 it8
26|2019-09-04T22:53:32|INFO|  Failed
27|2019-09-04T22:53:32|INFO|
28|2019-09-04T22:53:32|INFO|SpecLength jasmine-2.2.0 it9
29|2019-09-04T22:53:32|INFO|  Failed
30|2019-09-04T22:53:33|INFO|
31|2019-09-04T22:53:33|INFO|SpecLength jasmine-2.2.0 it10
32|2019-09-04T22:53:33|INFO|  Failed
33|2019-09-04T22:53:34|INFO|
34|2019-09-04T22:53:34|INFO|SpecLength jasmine-2.2.0 it11
35|2019-09-04T22:53:34|INFO|  Failed
36|2019-09-04T22:53:34|INFO|
37|2019-09-04T22:53:34|INFO|SpecLength jasmine-2.2.0 it12
38|2019-09-04T22:53:35|INFO|  Failed
39|2019-09-04T22:53:35|INFO|
40|2019-09-04T22:53:35|INFO|SpecLength jasmine-2.2.0 it13
41|2019-09-04T22:53:35|INFO|  Failed
42|2019-09-04T22:53:36|INFO|
43|2019-09-04T22:53:36|INFO|SpecLength jasmine-2.2.0 it14
44|2019-09-04T22:53:36|INFO|  Failed
45|2019-09-04T22:53:36|INFO|
46|2019-09-04T22:53:37|INFO|SpecLength jasmine-2.2.0 it15
47|2019-09-04T22:53:37|INFO|  Failed
48|2019-09-04T22:53:37|INFO|
49|2019-09-04T22:53:38|INFO|SpecLength jasmine-2.2.0 it16
50|2019-09-04T22:53:38|INFO|  Failed
51|2019-09-04T22:53:38|INFO|
52|2019-09-04T22:53:38|INFO|SpecLength jasmine-2.2.0 it17
53|2019-09-04T22:53:38|INFO|  Failed
54|2019-09-04T22:53:39|INFO|
55|2019-09-04T22:53:39|INFO|SpecLength jasmine-2.2.0 it18
56|2019-09-04T22:53:39|INFO|  Failed
57|2019-09-04T22:53:39|INFO|
58|2019-09-04T22:53:40|INFO|SpecLength jasmine-2.2.0 it19
59|2019-09-04T22:53:40|INFO|  Failed
60|2019-09-04T22:53:40|INFO|
61|2019-09-04T22:53:40|INFO|SpecLength jasmine-2.2.0 it20
62|2019-09-04T22:53:41|INFO|  Failed
63|2019-09-04T22:53:41|INFO|
64|2019-09-04T22:53:41|INFO|SpecLength jasmine-2.2.0 it21
65|2019-09-04T22:53:41|INFO|  Failed
66|2019-09-04T22:53:41|INFO|
67|2019-09-04T22:53:42|INFO|SpecLength jasmine-2.2.0 it22
68|2019-09-04T22:53:42|INFO|  Failed
69|2019-09-04T22:53:42|INFO|
70|2019-09-04T22:53:43|INFO|SpecLength jasmine-2.2.0 it23
71|2019-09-04T22:53:43|INFO|  Failed
72|2019-09-04T22:53:43|INFO|
73|2019-09-04T22:53:43|INFO|SpecLength jasmine-2.2.0 it24
74|2019-09-04T22:53:44|INFO|  Failed
75|2019-09-04T22:53:44|INFO|
76|2019-09-04T22:53:44|INFO|SpecLength jasmine-2.2.0 it25
77|2019-09-04T22:53:44|INFO|  Failed
78|2019-09-04T22:53:45|INFO|
79|2019-09-04T22:53:45|INFO|SpecLength jasmine-2.2.0 it26
80|2019-09-04T22:53:45|INFO|  Failed
81|2019-09-04T22:53:45|INFO|
82|2019-09-04T22:53:46|INFO|SpecLength jasmine-2.2.0 it27
83|2019-09-04T22:53:46|INFO|  Failed
84|2019-09-04T22:53:46|INFO|
85|2019-09-04T22:53:46|INFO|SpecLength jasmine-2.2.0 it28
86|2019-09-04T22:53:46|INFO|  Failed
87|2019-09-04T22:53:47|INFO|
88|2019-09-04T22:53:47|INFO|SpecLength jasmine-2.2.0 it29
89|2019-09-04T22:53:47|INFO|  Failed
90|2019-09-04T22:53:47|INFO|
91|2019-09-04T22:53:48|INFO|SpecLength jasmine-2.2.0 it30
92|2019-09-04T22:53:48|INFO|  Failed
93|2019-09-04T22:53:48|INFO|
94|2019-09-04T22:53:48|INFO|30 specs, 30 failures
95|2019-09-04T22:53:49|INFO|Finished in 0.478 seconds
Time to complete: 1.756 seconds

My questions are thus:

  1. Do you know of any ExtendScript limitation to the number of functions that can be in a certain scope?
  2. Did you experience this when using the Toolkit?
  3. Do you have any idea as to why this might be happening?
  4. Are there other ExtendScript testing frameworks you use or have seen? (They seem non-existent).

Thanks in advance for any help you can provide.

tmaslen commented 5 years ago

Hi Shawn,

I'm really sorry I'm not in a position to help you with this.

I last looked at this project 5 years ago and my knowledge of it all has left me.

When I was working on this I spent a lot of time in the adobe help boards, each adobe product had a pretty decent community of developers working on it. Hopefully you can find something there.

Good luck buddy.

/t

shawnpyle commented 5 years ago

Thanks for the update!

shawnpyle commented 5 years ago

For posterity, it was jasmine's asynchronous spec running that was causing problems. Disabling that functionality allowed me to run many tests. theasci/jasminejsx has been updated accordingly. 

spyle@437 jasminejsx|master$ ./specRunner.sh
Starting ExtendScript specs...
Be sure you are signed into the Creative Cloud app or you'll get no output from the test suite.
1|2019-09-05T12:48:15|INFO|***************************
2|2019-09-05T12:48:15|INFO|Jasmine ExtendScript Runner
3|2019-09-05T12:48:15|INFO|***************************
4|2019-09-05T12:48:40|INFO|51 specs, 0 failures, 51 pending specs
5|2019-09-05T12:48:41|INFO|Finished in 0.221 seconds
Time to complete: 1.443 seconds