search

mschwartz / SilkJS

V8 Based JavaScript Swiss Army Knife (and HTTP Server!)
https://github.com/decafjs/decaf
Other
323 stars 37 forks source link

Silkjs broke static contents #39

Closed dhanielo closed 9 years ago

dhanielo commented 9 years ago

I created a MVVM architecture with AngularJS and SilkJS but in several occurrences silkjs don't render a full js content or keeping last html, example: Original:

'use strict';

var lightApp = angular.module('lightApp', [
'ngRoute',
'lightControllers'
]);

lightApp.config(['$routeProvider',
  function($routeProvider) {
    $routeProvider.
      when('/tasks', {
        templateUrl: 'tasks.html',
        controller: 'tasksController'
      });
  }]);

Rendered:

'use strict';

var lightApp = angular.module('lightApp', [
'ngRoute',
'lightControllers'
]);

lightApp.config(['$routeProvider',
  function($routeProvider) {
    $routeProvider.
      when('/dashboard', {
        templateUrl: 'dashboard.html',
        controller: 'dashboardController'

Sometimes on update html page (ex.: index.html), SilkJS don't render the new code but return the last index.html even when the server has been restarted.

Some idea for fix the bug?

Thanks all!

mschwartz commented 9 years ago

Please see https://github.com/decafjs/decaf and https://github.com/decafjs

dorgan commented 9 years ago

So is this project defunct??

On Tue, Oct 7, 2014 at 2:14 PM, Michael Schwartz notifications@github.com wrote:

Please see https://github.com/decafjs/decaf and https://github.com/decafjs

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58232441.

mschwartz commented 9 years ago

It's not defunct in the sense that I'm happy to support people already using it. My energies are being spent on decafjs these days, however.

On Tue, Oct 7, 2014 at 11:17 AM, dorgan@donaldorgan.com < notifications@github.com> wrote:

So is this project defunct??

On Tue, Oct 7, 2014 at 2:14 PM, Michael Schwartz notifications@github.com

wrote:

Please see https://github.com/decafjs/decaf and https://github.com/decafjs

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58232441.

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58232942.

dhanielo commented 9 years ago

Due to a peculiarity (I'm writing a short paper for my graduate degree) need to find a solution to this bug.

mschwartz commented 9 years ago

I don't see that bug in SilkJS programs I run here. I cannot diagnose what's going on without seeing how you are trying to use it.

On Tue, Oct 7, 2014 at 11:29 AM, Daniel Oliveira notifications@github.com wrote:

Due to a peculiarity (I'm writing a short paper for my graduate degree) need to find a solution to this bug.

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58235031.

mschwartz commented 9 years ago

See attached images 2014-10-07 at 11 34 am 2014-10-07 at 11 36 am 2014-10-07 at 11 37 am

jaygarcia commented 9 years ago

Some things to think about:

Is this happening with your local environment or remote server? Is this something that can be replicated across different environments? Is SILK throwing exceptions or is this somehow failing silently.

I've never run into this particular issue, but am very curious to see if somehow I can help find a solution.

Jay Garcia

CTO, Modus Create Inc. Ext JS in Action, Author Sencha Touch in Action, Author (coming soon)

web :: moduscreate.com twitter :: @ModusJesus mobile :: 301.785.6030

On Tue, Oct 7, 2014 at 1:47 PM, Daniel Oliveira notifications@github.com wrote:

I created a MVVM architecture with AngularJS and SilkJS but in several occurrences silkjs don't render a full js content or keeping last html, example: Original:

'use strict';

var lightApp = angular.module('lightApp', [ 'ngRoute', 'lightControllers' ]);

lightApp.config(['$routeProvider', function($routeProvider) { $routeProvider. when('/tasks', { templateUrl: 'tasks.html', controller: 'tasksController' }); }]);

Rendered:

'use strict';

var lightApp = angular.module('lightApp', [ 'ngRoute', 'lightControllers' ]);

lightApp.config(['$routeProvider', function($routeProvider) { $routeProvider. when('/dashboard', { templateUrl: 'dashboard.html', controller: 'dashboardController'

Sometimes on update html page (ex.: index.html), SilkJS don't render the new code but return the last index.html even when the server has been restarted.

Some idea for fix the bug?

Thanks all!

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39.

mschwartz commented 9 years ago

You are likely right, Jay. It seems as if he's got some proxy or cache in between his server and his browser. There's no way SilkJS would cache an HTML file and keep sending it if it is changed on disk. Especially after a restart. However, if the old HTML file is cached somewhere in between, it could be served by that cache over and over again.

On Tue, Oct 7, 2014 at 11:40 AM, Jay Garcia notifications@github.com wrote:

Some things to think about:

Is this happening with your local environment or remote server? Is this something that can be replicated across different environments? Is SILK throwing exceptions or is this somehow failing silently.

I've never run into this particular issue, but am very curious to see if somehow I can help find a solution.

Jay Garcia

CTO, Modus Create Inc. Ext JS in Action, Author Sencha Touch in Action, Author (coming soon)

web :: moduscreate.com twitter :: @ModusJesus mobile :: 301.785.6030

On Tue, Oct 7, 2014 at 1:47 PM, Daniel Oliveira notifications@github.com

wrote:

I created a MVVM architecture with AngularJS and SilkJS but in several occurrences silkjs don't render a full js content or keeping last html, example: Original:

'use strict';

var lightApp = angular.module('lightApp', [ 'ngRoute', 'lightControllers' ]);

lightApp.config(['$routeProvider', function($routeProvider) { $routeProvider. when('/tasks', { templateUrl: 'tasks.html', controller: 'tasksController' }); }]);

Rendered:

'use strict';

var lightApp = angular.module('lightApp', [ 'ngRoute', 'lightControllers' ]);

lightApp.config(['$routeProvider', function($routeProvider) { $routeProvider. when('/dashboard', { templateUrl: 'dashboard.html', controller: 'dashboardController'

Sometimes on update html page (ex.: index.html), SilkJS don't render the new code but return the last index.html even when the server has been restarted.

Some idea for fix the bug?

Thanks all!

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39.

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58236982.

dhanielo commented 9 years ago

Yes folks, do not discount the chance of an adjacent problem, I'm on a windows machine running vagrant with ubuntu 12.04.

The complete code of the application is:

https://github.com/rill-l33t/light-linux.git

It's pretty simple because I started to develop a little time.

note: I believe it is not the browser cache, tested in another browser, but I think yes: somewhere a cache is created, but can not find.

mschwartz commented 9 years ago

Try remove LogFile line from your config.js. It fails to run for me because server/logs directory does not exist.

On Tue, Oct 7, 2014 at 11:49 AM, Daniel Oliveira notifications@github.com wrote:

Yes folks, do not discount the chance of an adjacent problem, I'm on a windows machine running vagrant with ubuntu 12.04.

The complete code of the application is:

https://github.com/rill-l33t/light-linux.git

It's pretty simple because I started to develop a little time.

note: I believe it is not the browser cache, tested in another browser, but I think yes: somewhere a cache is created, but can not find.

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58238734.

dhanielo commented 9 years ago

I'm sorry, please create directory "logs" in "server" because I add this folder in .gitignore.

mschwartz commented 9 years ago

When I create the directory, this is what I see:

2014-10-07 at 11 55 am

dhanielo commented 9 years ago

Access: http://localhost:9090/#/tasks, see source page and open js/app.js

mschwartz commented 9 years ago

I get a bug in your angular js program:

ReferenceError: $location is not defined at new (http://localhost:9090/js/controllers.js:7:23) at e ( http://localhost:9090/bower_components/angular/angular.min.js:36:215) at Object.instantiate ( http://localhost:9090/bower_components/angular/angular.min.js:36:344) at http://localhost:9090/bower_components/angular/angular.min.js:73:365 at link ( http://localhost:9090/bower_components/angular-route/angular-route.js:939:26 ) at U ( http://localhost:9090/bower_components/angular/angular.min.js:68:177) at C ( http://localhost:9090/bower_components/angular/angular.min.js:57:359) at g ( http://localhost:9090/bower_components/angular/angular.min.js:50:19) at http://localhost:9090/bower_components/angular/angular.min.js:49:127 at http://localhost:9090/bower_components/angular/angular.min.js:50:508

angular.js:10734 On Tue, Oct 7, 2014 at 12:00 PM, Daniel Oliveira notifications@github.com wrote: > Access: http://localhost:9090/#/tasks, see source page and open js/app.js > > — > Reply to this email directly or view it on GitHub > https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58240761.
dhanielo commented 9 years ago

Please paste the js/app.js rendered in your browser.

dhanielo commented 9 years ago

I fixed this, update the project and try again.

Or replace this (js/controllers.js):

function ($scope, $http, $location) {
        $http.get('http://'+$location+'/getTaskMockup').success(function(data) {

For this:

function ($scope, $http) {
        $http.get('http://'+$location+'/getTaskMockup').success(function(data) {
mschwartz commented 9 years ago

I did a git pull and new errors occur:

  1. Request URL: http://undefined/getTaskMockup
  2. Request Headers 1. Provisional headers are shown
    1. Accept: application/json, text/plain, /
    2. Origin: http://localhost:9090
    3. Referer: http://localhost:9090/
    4. User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36
    5. X-DevTools-Emulate-Network-Conditions-Client-Id: 74CCED65-3B8F-0A21-DF5F-C12E625E6F02

GET http://undefined/getTaskMockup net::ERR_NAME_NOT_RESOLVED angular.js:9118 GET http://undefined/getTaskMockup net::ERR_NAME_NOT_RESOLVED angular.js:9118

On Tue, Oct 7, 2014 at 12:13 PM, Daniel Oliveira notifications@github.com wrote:

I fixed this, update the project and try again.

Or replace this (js/controllers.js):

function ($scope, $http, $location) { $http.get('http://'+$location+'/getTaskMockup').success(function(data) {

For this:

function ($scope, $http) { $http.get('http://'+$location+'/getTaskMockup').success(function(data) {

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58242955.

dhanielo commented 9 years ago

$location is angularJS service, for some reason it is not being recognized but this isn't main problem. I change $location for "localhost:9090" to continue testing, try again a pull.

mschwartz commented 9 years ago

I don't know what else to do for you. I run your app and it seems to work. You have bugs, but we all do :)

This is what I see:

2014-10-07 at 12 36 pm

It is likely you have some software like Norton or something similar that is trying to "help" your web browser, but it is getting in the way of your WWW development.

dhanielo commented 9 years ago

Yes, you get the result I expected (I got at the beginning), the problem is something here in the company machine (we use Trend Micro), had not thought of that, I'll do some tests to find out the real source.

dhanielo commented 9 years ago

Not... the problem is not related with security softwares, I downloaded the file ('app.js') through the "links" on "vagrant" and he came in half.

'use strict';

var lightApp = angular.module('lightApp', [
'ngRoute',
'lightControllers'
]);

lightApp.config(['$routeProvider',
  function($routeProvider) {
    $routeProvider.
      when('/dashboard', {
        templateUrl: 'dashboard.html',
        controller: 'dashboardController'
jaygarcia commented 9 years ago

Sounds like the VM itself might be the issue? I have trie hammering Silk in the past and have not run into this at all. =(

Does it occur After you change a file or is it something that is persistent after rebooting the VM? I've seen a similar issue with something called sendfile http://man7.org/linux/man-pages/man2/sendfile.2.html() using a different technology (VertX).

@Mike, what are your thoughts on this?

Jay Garcia

CTO, Modus Create Inc. Ext JS in Action, Author Sencha Touch in Action, Author (coming soon)

web :: moduscreate.com twitter :: @ModusJesus mobile :: 301.785.6030

On Tue, Oct 7, 2014 at 4:01 PM, Daniel Oliveira notifications@github.com wrote:

Not... the problem is not related with security softwares, I downloaded the file ('app.js') through the "links" on "vagrant" and he came in half.

'use strict';

var lightApp = angular.module('lightApp', [ 'ngRoute', 'lightControllers' ]);

lightApp.config(['$routeProvider', function($routeProvider) { $routeProvider. when('/dashboard', { templateUrl: 'dashboard.html', controller: 'dashboardController'

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58250971.

mschwartz commented 9 years ago

SilkJS uses sendfile(2) to deliver files. However, there's no way that sendfile() is caching previous file contents. All it does is copy from one fd to another in kernel space - more optimal to copy that way than to read into user space memory then write through the kernel to the socket. In other words, sendfile reads from the file on disk (fd) and writes to the socket (browser, fd).

I'm not sure if he's got SilkJS installed per the instructions? How he's running it?

On Tue, Oct 7, 2014 at 2:16 PM, Jay Garcia notifications@github.com wrote:

Sounds like the VM itself might be the issue? I have trie hammering Silk in the past and have not run into this at all. =(

Does it occur After you change a file or is it something that is persistent after rebooting the VM? I've seen a similar issue with something called sendfile http://man7.org/linux/man-pages/man2/sendfile.2.html() using a different technology (VertX).

@Mike, what are your thoughts on this?

Jay Garcia

CTO, Modus Create Inc. Ext JS in Action, Author Sencha Touch in Action, Author (coming soon)

web :: moduscreate.com twitter :: @ModusJesus mobile :: 301.785.6030

On Tue, Oct 7, 2014 at 4:01 PM, Daniel Oliveira notifications@github.com

wrote:

Not... the problem is not related with security softwares, I downloaded the file ('app.js') through the "links" on "vagrant" and he came in half.

'use strict';

var lightApp = angular.module('lightApp', [ 'ngRoute', 'lightControllers' ]);

lightApp.config(['$routeProvider', function($routeProvider) { $routeProvider. when('/dashboard', { templateUrl: 'dashboard.html', controller: 'dashboardController'

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58250971.

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58263743.

obscurerichard commented 9 years ago

Oh dear. Virtualbox and the vboxsf file system and sendfile do not get along. Vagrant usually uses Virtualbox as its virtualization container and uses vboxsf as the default shared file system to share files between the host and the guest. See:

https://www.virtualbox.org/ticket/819 https://www.virtualbox.org/ticket/9069 https://www.virtualbox.org/ticket/12597

The workaround is well known, to turn off sendfile support in whatever web server you are using:

http://www.mabishu.com/blog/2013/05/07/solving-caching-issues-with-vagrant-on-vboxsf/

I'd recommend that SilkJS add a configuration option to turn off sendfile(2) support for delivering files to work around this issue.

mschwartz commented 9 years ago

Can you install Chrome or Firefox in your virtual box instance?

On Tue, Oct 7, 2014 at 5:46 PM, Richard Bullington-McGuire < notifications@github.com> wrote:

Oh dear. Vagrant and sendfile do not get along.

https://www.virtualbox.org/ticket/819

The workaround is well known, to turn off sendfile support in whatever web server you are using:

http://www.mabishu.com/blog/2013/05/07/solving-caching-issues-with-vagrant-on-vboxsf/

I'd recommend that SilkJS add a configuration option to turn off sendfile(2) support for delivering files to work around this issue.

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58287837.

dhanielo commented 9 years ago

@mschwartz I can't install chrome or fx in vbox because vagrant don't create xserver in vm. The solution of @obscurerichard seems to me the most right at the moment. Is it possible disable sendfile() in silkjs?

mschwartz commented 9 years ago

You can install X and desktop with vagrant. I use it all the time.

On Wed, Oct 8, 2014 at 4:35 AM, Daniel Oliveira notifications@github.com wrote:

@mschwartz https://github.com/mschwartz I can't install chrome or fx in vbox because vagrant don't create xserver in vm. The solution of @obscurerichard https://github.com/obscurerichard seems to me the most right at the moment. Is it possible disable sendfile() in silkjs?

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58344577.

dhanielo commented 9 years ago

Yes, I can install x on vagrant, but is really necessary for the test? I used LINKS browser before and got the same result.

mschwartz commented 9 years ago

https://github.com/mschwartz/SilkJS/blob/master/httpd/response.js#L193

You can change the code to not call sendfile here.

On Wed, Oct 8, 2014 at 5:42 AM, Daniel Oliveira notifications@github.com wrote:

Yes, I can install x on vagrant, but is really necessary for the test? I used LINKS browser before and got the same result.

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58351197.

dhanielo commented 9 years ago

nice! I will try it.

jaygarcia commented 9 years ago

Just try with wget or curl on the Linux machine. That will tell you if the send file on flagrant is the culprit (my strong feeling).

Also, to fix this easily, you can front silks with Ngnix on your vm.

JG

301.785.6030 :: @moduscreate

:: sent from my mobile device ::

On Oct 8, 2014, at 08:46, Daniel Oliveira notifications@github.com wrote:

nice! I will try it.

— Reply to this email directly or view it on GitHub.

mschwartz commented 9 years ago

It is a bug in VirtualBox, Jay.

I don't think there's a fix, other than to replace sendfile() call with open(), read()/write() loop, close()

The beauty of the situation is that the logic is written in JavaScript and this is an open source project so people are free to hack on it to suit their needs :)

On Wed, Oct 8, 2014 at 6:17 AM, Jay Garcia notifications@github.com wrote:

Just try with wget or curl on the Linux machine. That will tell you if the send file on flagrant is the culprit (my strong feeling).

Also, to fix this easily, you can front silks with Ngnix on your vm.

JG

301.785.6030 :: @moduscreate

:: sent from my mobile device ::

On Oct 8, 2014, at 08:46, Daniel Oliveira notifications@github.com wrote:

nice! I will try it.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58355400.

moduscreate commented 9 years ago

We resolved this with a project that used VirtualBox, Flagrant, and VertX by fronting the with VertX stack nginx. That's a viable, cheap and quick solution to the problem. No need to hack/modify Silk.

Jay Garcia -------------------------------------------- CTO & Co-founder, Modus Create Inc. Author of Ext JS & Sencha Touch in Action

moduscreate.com @ModusCreate 301.785.6030

This electronic mail transmission contains information from Modus Create, Inc. which may be privileged and/or confidential. The information contained herein is for the exclusive use of the addressee named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this information is prohibited. If you have received this electronic mail transmission in error, please notify us by telephone *(301-785-6030) or by electronic mail (*contact@moduscreate.com contact@moduscreate.com ).

On Wed, Oct 8, 2014 at 9:20 AM, Michael Schwartz notifications@github.com wrote:

It is a bug in VirtualBox, Jay.

I don't think there's a fix, other than to replace sendfile() call with open(), read()/write() loop, close()

The beauty of the situation is that the logic is written in JavaScript and this is an open source project so people are free to hack on it to suit their needs :)

On Wed, Oct 8, 2014 at 6:17 AM, Jay Garcia notifications@github.com wrote:

Just try with wget or curl on the Linux machine. That will tell you if the send file on flagrant is the culprit (my strong feeling).

Also, to fix this easily, you can front silks with Ngnix on your vm.

JG

301.785.6030 :: @moduscreate

:: sent from my mobile device ::

On Oct 8, 2014, at 08:46, Daniel Oliveira notifications@github.com wrote:

nice! I will try it.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58355400.

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58355686.

obscurerichard commented 9 years ago

If silk is using sendfile to read files that it sends out to the network, and the source filesystem is vboxsf, this will affect it. Sure, you could use a different web server to serve up some static assets to work around the problem too. You'll have to configure that server to avoid using sendfile.

dhanielo commented 9 years ago

@mschwartz, @jaygarcia , @obscurerichard, @moduscreate Hi guys! thanks all! I fix it, I rewrite a code snippet responsible for send file to sock as you recommended.

if (Config.sendFile || Config.sendFile == undefined) {

   net.sendFile(res.sock, fn, 0, size); // (FileSystem.readfile64(fn));
} else {
     var file = fs.open(fn, fs.O_RDONLY);
      var content = net.read(file, size, size);
      net.write(res.sock, content, size);
      fs.close(file);                                  
}
moduscreate commented 9 years ago

What was the resolution? :)

Jay Garcia -------------------------------------------- CTO & Co-founder, Modus Create Inc. Author of Ext JS & Sencha Touch in Action

moduscreate.com @ModusCreate 301.785.6030

This electronic mail transmission contains information from Modus Create, Inc. which may be privileged and/or confidential. The information contained herein is for the exclusive use of the addressee named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this information is prohibited. If you have received this electronic mail transmission in error, please notify us by telephone *(301-785-6030) or by electronic mail (*contact@moduscreate.com contact@moduscreate.com ).

On Wed, Oct 8, 2014 at 11:46 AM, Daniel Oliveira notifications@github.com wrote:

@mschwartz https://github.com/mschwartz, @jaygarcia https://github.com/jaygarcia , @obscurerichard https://github.com/obscurerichard, @moduscreate https://github.com/moduscreate Hi guys! thanks all! I fix it, I rewrite a code snippet responsible for send file to sock as you recommended.

— Reply to this email directly or view it on GitHub https://github.com/mschwartz/SilkJS/issues/39#issuecomment-58379122.

dhanielo commented 9 years ago

I change sendFile() for open(), read(), write(), close() and I created option for change modes according to need.

https://github.com/mschwartz/SilkJS/pull/40

obscurerichard commented 9 years ago

@rill-l33t could you submit a pull request with your changes?

Also instead of reading the entire file into memory, it might be better to read and write it in chunks of say, 64K at a time to avoid exhausting memory if it attempts to serve a large file.

mschwartz commented 9 years ago

He did submit a pull request which has been merged.

obscurerichard commented 9 years ago

Awesome! I missed that.

Richard Bullington-McGuire +1 571 236 0938

On Oct 13, 2014, at 4:42 PM, Michael Schwartz notifications@github.com wrote:

He did submit a pull request which has been merged.

— Reply to this email directly or view it on GitHub.

dhanielo commented 9 years ago

@obscurerichard you saw my changes? you agree with them? https://github.com/mschwartz/SilkJS/pull/40

obscurerichard commented 9 years ago

I did. The pull request looks good. It might be better not to read in the entire file before writing it out, though.

dhanielo commented 9 years ago

I understand, but do not know make how different. how read 64k in 64k?