search

scalafiddle / scalafiddle-core

Source code for the scalafiddle.io compilation service
https://scalafiddle.io
Apache License 2.0
101 stars 37 forks source link

Getting 414 Request-URI Too Long when compiling a fiddle. #20

Closed pishen closed 7 years ago

pishen commented 7 years ago

As this fiddle https://scalafiddle.io/sf/mGyqlop/0 has shown, we can only get an "ERROR" on the right side after compiling. Going into the browser console, the error message is as below:

GET https://embed.scalafiddle.io/compile?opt=fast&source=H4sIAAEAAAAA_-2dR68kR3…b3bbv7tt192-6-bXfftrtv29237e7bdvdtu__3bbv4I8BnL3988n9P_g6s53urgHAAAA%3D%3D 414 ()
XMLHttpRequest cannot load https://embed.scalafiddle.io/compile?opt=fast&source=H4sIAAEAAAAA_-2dR68kR3…b3bbv7tt192-6-bXfftrtv29237e7bdvdtu__3bbv4I8BnL3988n9P_g6s53urgHAAAA%3D%3D. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://scalafiddle.io' is therefore not allowed access. The response had HTTP status code 414.
ochrons commented 7 years ago

The source code is simply too long. ScalaFiddle uses GET requests and places the source code into the query string (gzipped and base64-coded) to enable CloudFlare caching of compilation results.

The size of GET requests is, however, limited to a few dozen kB.

mkotsbak commented 7 years ago

What about using POST instead?

ochrons commented 7 years ago

POST responses cannot be cached by CDNs.

pishen commented 7 years ago

@ochrons How if we do a GET to the CDN with the hash of the source code in the URL instead of the source code itself? And if CDN doesn't have the cache, our server will return 404 to the client. After seeing 404, the client will do a POST to the compile API with the source code in its body. Then, after compilation, instead of returning the result directly, the server will cache the result for a while, and redirect the client to an URL with the hash of the source code in it at the same time. Since our server cache the result for a while, it should be able to provide the result for the CDN to cache it now.

ochrons commented 7 years ago

Believe me, I've already considered these options :)

With that strategy, every change to source code results in slower compilation since it has to make two roundtrips instead of one. Perhaps this strategy could be adopted only for large source files, but it makes things more complicated without much benefit. ScalaFiddle.io is not intended for writing large blocks of code, but for providing embedded small example code snippets in documentations and for quick testing.

BTW, the server already caches compilation results, since CDN caching is distributed and will generate more requests to the backend when accessed from different locations. To really get benefit out of the hashing approach, the server would need to cache all compilations in a persistent fashion.