Open joanhey opened 3 years ago
NateBrady23
commented
3 years ago @msmith-techempower Let's take a look at this and confirm. On the one hand, I don't want to give platforms an edge by forcing frameworks to do this while they don't; on the other hand, it does make sense that the approach isn't "realistic" for frameworks.
joanhey
commented
2 years ago OK You can permit it in platforms only or platforms and raw db. Or never permited. But never in micro or full frameworks with ORM.
So we will have extra info from use ORM and template.
But will be in the requirements where it's possible to by pass the template engine.
masfahru
commented
1 year ago Hello @nbrady-techempower, some of web frameworks which have built-in hydration (for example NextJS, NuxtJS, and SvelteKit) add script(s) in the html response, and it fails the test.
--------------------------------------------------------------------------------
VERIFYING FORTUNE
--------------------------------------------------------------------------------
Accessing URL http://tfb-server:8080/fortunes:
fortune: Oh no! I compared <!doctype html><html>
fortune: <head><title>Fortunes</title></head>
fortune: <body><table>
fortune: <tr><th>id</th><th>message</th></tr>
fortune: <tr><td>11</td><td><script>alert("This should not be displayed in a browser alert box.");</script></td></tr>
fortune: <tr><td>4</td><td>A bad random number generator: 1, 1, 1, 1, 1, 4.33e+67, 1, 1, 1</td></tr>
fortune: <tr><td>5</td><td>A computer program does what you tell it to do, not what you want it to do.</td></tr>
fortune: <tr><td>2</td><td>A computer scientist is someone who fixes things that aren't broken.</td></tr>
fortune: <tr><td>8</td><td>A list is only as strong as its weakest link. — Donald Knuth</td></tr>
fortune: <tr><td>0</td><td>Additional fortune added at request time.</td></tr>
fortune: <tr><td>3</td><td>After enough decimal places, nobody gives a damn.</td></tr>
fortune: <tr><td>7</td><td>Any program that runs right is obsolete.</td></tr>
fortune: <tr><td>10</td><td>Computers make very fast, very accurate mistakes.</td></tr>
fortune: <tr><td>6</td><td>Emacs is a nice operating system, but I prefer UNIX. — Tom Christaensen</td></tr>
fortune: <tr><td>9</td><td>Feature: A bug with seniority.</td></tr>
fortune: <tr><td>1</td><td>fortune: No such file or directory</td></tr>
fortune: <tr><td>12</td><td>フレームワークのベンチマーク</td></tr>
fortune: </table></body></html>
fortune: to
fortune: <!doctype html><html>
fortune: <head><title>Fortunes</title></head>
fortune: <body><table>
fortune: <tr><th>id</th><th>message</th></tr>
fortune: <tr><td>11</td><td><script>alert("This should not be displayed in a browser alert box.");</script></td></tr>
fortune: <tr><td>4</td><td>A bad random number generator: 1, 1, 1, 1, 1, 4.33e+67, 1, 1, 1</td></tr>
fortune: <tr><td>5</td><td>A computer program does what you tell it to do, not what you want it to do.</td></tr>
fortune: <tr><td>2</td><td>A computer scientist is someone who fixes things that aren't broken.</td></tr>
fortune: <tr><td>8</td><td>A list is only as strong as its weakest link. — Donald Knuth</td></tr>
fortune: <tr><td>0</td><td>Additional fortune added at request time.</td></tr>
fortune: <tr><td>3</td><td>After enough decimal places, nobody gives a damn.</td></tr>
fortune: <tr><td>7</td><td>Any program that runs right is obsolete.</td></tr>
fortune: <tr><td>10</td><td>Computers make very fast, very accurate mistakes.</td></tr>
fortune: <tr><td>6</td><td>Emacs is a nice operating system, but I prefer UNIX. — Tom Christaensen</td></tr>
fortune: <tr><td>9</td><td>Feature: A bug with seniority.</td></tr>
fortune: <tr><td>1</td><td>fortune: No such file or directory</td></tr>
fortune: <tr><td>12</td><td>フレームワークのベンチマーク</td></tr>
fortune: <script>
fortune: {
fortune: __sveltekit_e078m7 = {
fortune: base: new URL(".", location).pathname.slice(0, -1),
fortune: env: {}
fortune: };
fortune: const element = document.currentScript.parentElement;
fortune: const data = [null,{"type":"data","data":{fortunes:[{id:11,message:"\u003Cscript>alert(\"This should not be displayed in a browser alert box.\");\u003C/script>"},{id:4,message:"A bad random number generator: 1, 1, 1, 1, 1, 4.33e+67, 1, 1, 1"},{id:5,message:"A computer program does what you tell it to do, not what you want it to do."},{id:2,message:"A computer scientist is someone who fixes things that aren't broken."},{id:8,message:"A list is only as strong as its weakest link. — Donald Knuth"},{id:0,message:"Additional fortune added at request time."},{id:3,message:"After enough decimal places, nobody gives a damn."},{id:7,message:"Any program that runs right is obsolete."},{id:10,message:"Computers make very fast, very accurate mistakes."},{id:6,message:"Emacs is a nice operating system, but I prefer UNIX. — Tom Christaensen"},{id:9,message:"Feature: A bug with seniority."},{id:1,message:"fortune: No such file or directory"},{id:12,message:"フレームワークのベンチマーク"}]},"uses":{}}];
fortune: Promise.all([
fortune: import("./_app/immutable/entry/start.d4d83278.js"),
fortune: import("./_app/immutable/entry/app.86ba13de.js")
fortune: ]).then(([kit, app]) => {
fortune: kit.start(app, element, {
fortune: node_ids: [0, 3],
fortune: data,
fortune: form: null,
fortune: error: null
fortune: });
fortune: });
fortune: }
fortune: </script></table></body></html>
fortune: Fortune invalid. Diff following:
fortune: --- Valid
fortune: +++ Response
fortune: @@ -18 +18,24 @@
fortune: -</table></body></html>
fortune: +<script>
fortune: + {
fortune: + __sveltekit_e078m7 = {
fortune: + base: new URL(".", location).pathname.slice(0, -1),
fortune: + env: {}
fortune: + };
fortune: +
fortune: + const element = document.currentScript.parentElement;
fortune: +
fortune: + const data = [null,{"type":"data","data":{fortunes:[{id:11,message:"\u003Cscript>alert(\"This should not be displayed in a browser alert box.\");\u003C/script>"},{id:4,message:"A bad random number generator: 1, 1, 1, 1, 1, 4.33e+67, 1, 1, 1"},{id:5,message:"A computer program does what you tell it to do, not what you want it to do."},{id:2,message:"A computer scientist is someone who fixes things that aren't broken."},{id:8,message:"A list is only as strong as its weakest link. — Donald Knuth"},{id:0,message:"Additional fortune added at request time."},{id:3,message:"After enough decimal places, nobody gives a damn."},{id:7,message:"Any program that runs right is obsolete."},{id:10,message:"Computers make very fast, very accurate mistakes."},{id:6,message:"Emacs is a nice operating system, but I prefer UNIX. — Tom Christaensen"},{id:9,message:"Feature: A bug with seniority."},{id:1,message:"fortune: No such file or directory"},{id:12,message:"フレームワークのベンチマーク"}]},"uses":{}}];
fortune: +
fortune: + Promise.all([
fortune: + import("./_app/immutable/entry/start.d4d83278.js"),
fortune: + import("./_app/immutable/entry/app.86ba13de.js")
fortune: + ]).then(([kit, app]) => {
fortune: + kit.start(app, element, {
fortune: + node_ids: [0, 3],
fortune: + data,
fortune: + form: null,
fortune: + error: null
fortune: + });
fortune: + });
fortune: + }
fortune: + </script></table></body></html>
FAIL for http://tfb-server:8080/fortunes
Invalid according to FortuneHTMLParser
See https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview#specific-test-requirementscan a response like that be acceptable in the future ? because i think bypassing the template system for those web frameworks deviates their purpose.
Some frameworks bypass the template engine.
I understand it in platforms. But in micro and full frameworks they have a template engine. If they don't use it for the fortunes, it isn't a realistic approach. Nobody will place the full output in the controller for an html response like that.
We need to clarify it, or all frameworks will change it to bypass the template engine.