url rewriting optional? github.io compatibility?

[laurelnaiad]

It looks like most of what you're getting from node as a server and apache is url rewriting. If that's true, would docs generated by docular be github.io compatible if hash mode was forced? Or is there more to it than that?

[gitsome]

@stu-salsbury , great question. I am not very familiar with github.io. I will have to take a look, but you are right that the node server script and the .htaccess file for apache really only does some rewriting. Without it, a refresh in the middle of a documentation page would end up in a 404. The site does however work fine if loaded from the root and then navigate. I suspect this is what would occur in github.io, but I will have to verify. I am currently working on adding a plugin architecture that requires additional logic in the url rewriting so controllers can be hit via /controller/. Well do some more research and get back.

[laurelnaiad]

Thanks, @gitsome.

github.io, to the best of my knowledge, is ultimately a static webserver. Now that I'm not reading on my phone, I looked at the source.

I was wondering what would happen if this line that sets html5Mode

https://github.com/gitsome/docular/blob/master/lib/webapp/resources/js/docs_module_end.js#L5

        $locationProvider.html5Mode(true).hashPrefix('!');

were commented out (or in real life could be switched on/off via configuration). Would the site work in hash mode? I guess I can try it once I get up and running, but was curious as to whether I'd need to run my own server or not.

[gitsome]

great @stu-salsbury, I gave it a quick shot.. it looks like there would need to be a few code changes to get this working correctly, but this would be a great feature to get working as running this in github.io I could see being a very common and relevant use case. I've marked this as an enhancement to track.

[laurelnaiad]

Yeah, I think docular and github.io would go together like peanut butter and chocolate. I'll look forward to it!

This is one minor thing I found when checking out how github.io works... not a big deal, though there may be other gotchas. It describes how to disable "jekyll", which I imagine one would want to do: https://help.github.com/articles/files-that-start-with-an-underscore-are-missing

[laurelnaiad]

Just in case this helps... when the time comes... here's a little code that I got from a stackoverflow answer about switching between html5 mode and hash mode. It might be useful to you when the time comes.

I've found that this makes it possible to author all of the paths as if there are no hashbangs and to get Angular to insert them on the fly. Basically, it lies to angular and says the browser doesn't support HTML5 mode, which causes angular to put hashes in at runtime. This way no URLs in the templates/etc. need to conditionally require hashes.

If the flag doesn't return true-ish, then html5Mode works normally. If the flag is true-ish, then angular runs in a mode that adds #s to paths automagically.

//assumes this.hashMode is the flag that forces hash mode where the default is html5Mode
  myModule.config([ '$locationProvider', '$provide',
    function($locationProvider, $provide) {
      if (this.hashMode) {
        $provide.decorator('$sniffer', function($delegate) {
          $delegate.history = false;
          return $delegate;
        });
      }
      $locationProvider.html5Mode(true);
      //...

[gitsome]

Nice! A true Reeses Cup full of info! thanks @stu-salsbury

[laurelnaiad]

I started playing with this.

I applied the change as I quoted above. Then I messed with the main index.html file. I wrapped all the paths I could find in the path function that already exists, like this:

 path('resources/css/bootstrap.min.css')

(some of the paths weren't wrapped in the path function)

I changed that function prepend baseUr like this:

            var path = function(name) {
                if (production) {
                    return baseUrl + name.replace(/\.js$/, '.min.js');
                } else {
                    return baseUrl + name;
                }
            };

I took out the direct assignment of baseURL, which seemed to be confused with baseUrl -- note the capitalization difference and made one assignment for both cases:

baseURL = baseUrl = location.pathname;

I copied the whole webapp to separate loctation and ran it from a static web server under "localhost/mydocs".

A bunch of things work. There are broken links to images -- I'm sure that would be fixable.

The biggest issue I notice is with source code. I haven't looked into it yet, but there are all these "C" files where the source code is pointing, and the URL it tries to access is trying to append a bunch of local file system stuff to point right back at the source files themselves.... this part seems like it might be problematic -- I have yet to figure out what's going on there...

I'm going to knock off for the night, but just wanted to give a heads up... anyway, that's why I was asking about baseUrl in the other issue.

[laurelnaiad]

With some tweaking to the following files I have a lot of docular-generated docs working from a static web server in a subdirectory of the server with hash mode.

So, http://localhost:3006 is run by a "dumb" node web server, and the generated webapp directory is mapped to http://localhost:3006/docs/ and I can access the documentation at places like http://localhost:3006/docs/#/documentation/docular/basics/index and such. Great!

The files I modified were:

• lib/resources/templates/docular_partial_footer.html
• lib/resources/templates/docular_partial_navigation.html
• lib/resources/templates/index.html
• lib/webapp/resources/js/doc_module_end.js

At this stage, it's pretty hack-ish, but I just wanted to see how far I could get by blasting away at it.

However, images seem to be a problem.

I'm fairly confused about file paths and what is part of the docular install versus what is part of generated docs vs what is part of the angular and docular documentation that is generated.

Is docular/lib supposed to be part of docular? Is docular/lib/webapp supposed to be the output of grunt-docular? Or is the input?

I'm having trouble relating the angular/anguar.js/docs directory to what I'm seeing in docular -- I can see that angular/angular.js/docs/img seems to be related to gitsome/docular/lib/webapp/resources/img, but perhaps the latter also includes docular images?

To be short: I can't tell how much of the file paths problems that I'm encountering are because of the way that docular and angular docs are authored, and how much is related to the assumptions in the docular generation process about relative paths.

In addition to wanting to understand the structure of docular and how it relates to the documentation for angular and docular, one other thing is sort of on my mind... do you want docular webapps to work without a smart server? Or is your intention to keep developing the server as an integral part of docular? I ask because it sounds like you're working on a node express version, which makes it seem like your long run plan is to have/require a smart one! :) If that's the case, it would be great to know -- please don't take this the wrong way, but I'm pretty sure that I want to have documentation that doesn't require an intelligent backend server, and I might need to look elsewhere if "dumb/static" web server deployments are not a primary goal.

[gitsome]

Okay, nice to see you made some progress. I think I follow some of the issues you are running into.

1) Is docular/lib supposed to be part of docular? Is docular/lib/webapp supposed to be the output of grunt-docular? Or is the input?

I look at Docular as a plugin that contains the index.js file, a directory for tests (which still need a ton of work), and a directory for scripts and resources within the "lib" directory needed by or affected by the index.js file.

Currently the "output" of Docular is the "webapp" directory within the "lib" directory. I had envisioned that the lib directory would hold the scaffolding of the "webapp" and then copies of those files plus the generated docs would be exported to where the user wanted. This feature hasn't been fully realized, so all dynamically generated docs and supporting files get injected into the "webapp" directory that lives within the "lib" directory.

Within "webapp", there are files that should be there for all instances of the Docular output, as well as dynamically generated supporting files which includes the documentation partials.

It is a bit cludgy right now so I can understand the confusion.

So, the AngularJS and Docular source and documentation is within the "lib" directory as a resource anyone can use for examples. However, if someone wanted to use Docular to parse documentation for their own project, their source files would live outside of Docular and then configure the Gruntfile.js file to point to those source files (this would be relative to wherever Grunt was installed).

So for example... suppose I was making a project called myProject. My directory tree would look like this:

| - myProject
____| node_modules
_________| grunt-docular
______________| node_modules
___________________| docular
________________________| lib
_____________________________| webapp
_____| + mySourceCode
_____| Gruntfile.js

* So the "mySourceCode" directory would hold all the files I want parsed and shown by Docular.
* I would include that in the configuration in the Gruntfile.js file at the root of my app
* When I run "grunt docular", the source files are parsed and documentation is generated and stuck within the webapp directory as shown in the directory tree.
* The documentation webapp IS the webapp directory up above. The "grunt docular-server" task runs the "server.js" file but changes the current working directory (cwd) to be the "webapp" directory. Similarly, if you copy and paste the contents of the webapp directory and jam them into an apache server, the .htaccess file would assume that the root of your app is the "webapp" directory.

So I think this is why it is so confusing (especially if you are contributing to the plugin). If you don't think of grunt-docular as a "black box" and want to get at the webapp files, things get hairy.

I think this is where the enhancements of providing a target directory would be beneficial. This is going to be a challenge because I know I would want to edit the files that should not be changed within the webapp. When you re-export those changes would be overriden... another topic for another issue :-)

2) I can't tell how much of the file paths problems that I'm encountering are because of the way that docular and angular docs are authored, and how much is related to the assumptions in the docular generation process about relative paths.

So I think the answer to question 1 above may cover this, but if any asset like an image is referenced in the documentation, then it should be relative to the root of the documentation "webapp".  So that would require adding a directory yourself within the "webapp" directory with those images. The images for the docular tutorial source are stored within the webapp (but the references need to be updated) as an example.

3) do you want docular webapps to work without a smart server?

Yes, I think most people would like to have the app run in a static "non-smart" server. I checked in the code for the plugin architecture that may require the use of some custom rewrite/request handling, but it is not mandatory unless you absolutely need to use that plugin that requires it.

I hope that helps. I think you are exposing some key weaknesses of the current setup that when resolved will really make Docular much easier to use and to understand. 

[laurelnaiad]

Thanks, @gitsome. I've continued to poke around and realized that part of my confusion is how you combined the docs for docular and angular. I've given up trying to figure out how the paths work for the docs specific to docular and angular, and fallen back to making some documentation of my own work in a subdirectory on github. I'm hacking away trying to get something to completely work.

Here's where I'm at (you don't really want to see the hacking I've done to get there, so I won't trouble you with it... I'm going to need to go through my changes and figure out which ones are really useful and then I can show you where I tweaked things.

For now, if you have time, would you take a look at this?

http://afterglowtech.github.io/agt-dynamicRoutingShared/

Here are some things that are a little sketchy (so far): 1) there seems to be an assumption that the source files are living somewhere near the webapp so source code is totally off limits when I take webapp and run it outside of its normal location? -- I'd really like to be able to copy the source into the webapp to keep this feature 2) I can't get my class to load on github.io -- it works on my dev box. Because of that, I'm wondering whether this problem is also related to #1 (i.e. it wants to see the source code file where the @ documentation lives. This URL should actually display the documentation for class Common (it has a function documented) http://afterglowtech.github.io/agt-dynamicRoutingShared/#/documentation/dynamicRoutingShared/api/common.class:Common -- but it seems I'm getting 404 on http://afterglowtech.github.io/agt-dynamicRoutingShared/documentation/partials/dynamicRoutingShared/api/common.class:Common.html ... there is indeed no such file, but when I run the app locally I don't get an error and I see my class' documentation. 3) I'm still a bit unsure about how I'm supposed to tag things with ampersands and the whole syntax -- for instance I had to put "@link dynamicRoutingShared/api" in /docs/section-api.doc and I wasn't really sure why. 4) I noticed a file in my webapp that looks like it snuck into the docular-source directory: documentation/docular-source/dynamicRoutingShared/api. (you can switch to the gh-pages of my respository to see the webapp that I've generated.

So here's my actual source tree (a branch specific to testing docular) https://github.com/afterglowtech/agt-dynamicRoutingShared/tree/tryDocular And here's my gh-pages where I've posted the generated webapp https://github.com/afterglowtech/agt-dynamicRoutingShared/tree/gh-pages

I'm glad to hear that you're aiming for static web server support, still a bit confused about when you mention plugins that won't be supported, but I won't freak out or anything.

If you have time to take a look at this, that'd be awesome.

[laurelnaiad]

This is the mess I've made of docular so far: https://github.com/stu-salsbury/docular/tree/upgradeAngular . Like I said, I've hacked it up... the only manual step after grunt docular that I'm doing is editing the base href to <base href="/agt-dynamicRoutingShared/'/>

[gitsome]

Hacking is always a good first step :-) I'll have to bring this down and try it out. So is the "baseUrl" config in the Gruntfile.js file not working. I don't know that I thoroughly tested that, but i should propagate through to the webapp. The docular-configuration.js file that is generated should be creating that tag... not sure though.

[laurelnaiad]

I don't think it was generating that, but I can't remember for sure -- I got confused and just forced it to work the way I needed it to with the intent to revist it. :)

If you're going to try to run my hacked version of docular, be aware that right now, you can't configure going between html5 and hash modes -- so, you you wanted to run in html5 mode with grunt docular-server, you'd need to

1. manually change https://github.com/stu-salsbury/docular/blob/upgradeAngular/lib/webapp/resources/js/docs_module_end.js#L5 ...(comment out the call to $provide.decorator() altogether
2. change base href to <base href="/"/>

But to check out my second problem above (not getting the documentation for my class) you could just debug my github.io pages app in your browser if that's comfortable.

[gitsome]

Sweet! Great to see it working on the github.io pages!

1) I see your point. Your goal is to have the source code within the webapp correct? The webapp generally is built to display the generated documentation so I am not sure that having the source within the webapp would suppor the general use case. Is it that you want the source to be available for viewing within github along with the documentation that is generated? I think I am confused here "I'd really like to be able to copy the source into the webapp to keep this feature".

2) I looked at your source and everything looks good. It is interesting that it works locally and not on your github.io pages. How are you getting the webapp over to your github.io page? Are you running docular first locally to generate the webapp, and then copying the webapp contents over to your github.io page?

If this were the case then the file that is coming up with a 404 should be exactly where it should be. So my question is how is that file not within your github.io pages?

3) You are perfect on your @'s and documentation blocks. The {@link is a special sort of enhancement for linking between documentation. So I would consider that completely separate form the general @ symbols used to generate key value pairs for the documentation object. The {@link 's are found after the fact in a separate step during the actual parsing of the description section of the doc.

4) That is really interesting that your other documentation ended up in the docular-source directory. This is a little mind boggling. If you remove those directories/files manually and run the documentation generation again do they reappear? If so, there must be a bug when parsing documentation groups.

Thanks again for all your time and details on these issues! I will review how you got it working on the github.io pages and then try to get it working myself.. hopefully we can get that feature in soon.

[laurelnaiad]

on source in the docs... In an ideal world, I could link to the source from each function's documentation straight to the github.com branch down to the line number, e.g. https://github.com/afterglowtech/agt-dynamicRoutingShared/blob/master/src/common.js#L3 for the function that I documented in my test... but that's for another day!

For starters I figured I'd try to get it to work the way it was intended to work -- and I thought that the intention of the view source button was to show the source code that is being documented... but now I'm not so sure. What is that button supposed to do?

on the documentation for the class doing a 404 on github.io... Yes, I grunt grunt docular locally and then copy the contents of webapp into my gh-pages branch.

The href for the class is /documentation/partials/dynamicRoutingShared/api/common.class:Common.html``. That file isn't present locally or on github.io. This is what's in the api directory: https://github.com/afterglowtech/agt-dynamicRoutingShared/tree/gh-pages/documentation/partials/dynamicRoutingShared/api . So it seems like that path isn't pointing to an actual file -- which makes me think something in docular is interpreting the path and looking for something else.

Can you point me to the place in the generated webapp code that handles the showing of documentation for a given class? I haven't been able to figure out how it works locally, thus don't even know what to look for that might be different on github.io.

on generated files ending up in docular-source Given that when I generate the code, git is reporting that the file is different than the one previously checked in, something's being modified.

I'm starting to wonder whether there needs to be a "clean" operation on the webapp in order to get it back to pristine state when you're rebuilding the docs -- right now if I understand correctly anything generated hangs around if you regenerate.

I think it would be super-helpful, instead of trying to clean the webapp, if the generation took place in a separate location -- grab the pristine webapp and then generate the documentation into a copy of it (hopefully outside the docular tree altogether).

[laurelnaiad]

Ok, I did some more testing -- is it possible that that path is relying on a feature of node webservers? Just a thought.

[thenanox]

I wanted to share some info that ive been researching regarding what @stu-salsbury commented before.

I was able to merge pull request gitsome/docular/pull/35 into my local docular and able to copy outputted documentation into my nginx local instalation. After some testing in my computer i had the problem that there were some files (the ones with ":" ) that werent working in my nginx. This also happens to @stu-salsbury like he says here:

"on the documentation for the class doing a 404 on github.io... Yes, I grunt grunt docular locally and then copy the contents of webapp into my gh-pages branch.

The href for the class is /documentation/partials/dynamicRoutingShared/api/common.class:Common.html. That file isn't present locally or on github.io"

This is due to the OS presented in the computer. Windows is not able to create files with the character ":", but it has a feature to create streams inside file with that character. So phisically the documentation its inside the file common.class in windows, but its not in a separate file. Read more about this in here: http://www.howtogeek.com/howto/windows-vista/stupid-geek-tricks-hide-data-in-a-secret-text-file-compartment/

Node webservers are able to read from sources inside files, thats why in node it works even when windows create this secret streams inside files, but not in any other webserver. In linux this is not happening because its able to create files with ":" so when you copy outtputed documentation it works.

So, what its the proper solution? im worried because this project seems dead since two months ago and i really like this project. Also @gitsome its kind of dissapeared. I think this should need a proper solution because right now documentation outputted its not working in the right way.

Thank you beforehand.

[laurelnaiad]

Thanks @thenanox! I was indeed running express on windows. I have a linux machine with nginx set up, which should doubly-support the ":" character behavior... when I get around to picking up my documentation tasks again...

[gitsome]

thanks @thenanox, So there is an incompatibility with the ":" within the filename of the partial generated on windows. So it seems that we should be able to simply replace that character in the filename and update the list the UI receives to pull in that partial. I think the syntax within the docs can stay with ":". Perhaps switch to "_" ? I'll fire up my windows vm and get a fix ready.

I apologize for the delay! If anyone is really serious about this project and feels comfortable with the code base, shoot me a message and we can look into adding access to the main repo.

[laurelnaiad]

I'm serious about the project, not entirely comfortable with the code base but could be, and don't have the time to help very much.... that said, this issue is really so mixed up with #11 that I feel compelled to reference the issues together... so I just did.

[gitsome]

Thanks @stu-salsbury, I am using this project myself to document another project (which is why I have been away for a while), so rest assured that it will keep going... but probably in spurts on Labor Day type weekends :-) Thanks for the added documentation for linking #11

[jmdobry]

:+1: for being able to deploy docular-generated documentation to github.io. GitHub Pages are the goto solution for hosting documentation for OSS projects which are hosted on GitHub.

[stephenliberty]

The semicolon problem has been fixed in the beta. Using html5mode is now optional, which should allow for much easier hosting.