search

JordanMartinez / purescript-jordans-reference

Learn PureScript with this "clone and play" repository
https://jordanmartinez.github.io/purescript-jordans-reference-site/
585 stars 73 forks source link

Add in-repo project: ToC-generating program for this project #222

Closed JordanMartinez closed 5 years ago

JordanMartinez commented 5 years ago

43 has the idea of a single file that acts like a clickable ToC that will open up the corresponding file, making navigation much easier overall.

I'd like to target this concept as the next in-repo project to work on. I think the libraries it should cover are:

Thus, I will write a program that generates outputs/updates a file with an up-to-date Table of Contents of this entire repo that, when an entry is clicked, will open the corresponding file in GitHub on the latestRelease branch.

While it will produce a GitHub-friendly version by default, it should be written with the option of generating a ToC using links for your own local computer.

I also want it to include all section headers, so that one can literally see the overall structure of this repo in one page. If I get there, I'll add "sections" in the Syntax folder so that those can be viewed, too.

I'm not sure what the final look of the ToC should be since there will be many intermediate folder names involved and I'd like to show the structure of things without it getting too long.

JordanMartinez commented 5 years ago

Purpose

Principles

Outcome 1

The format of the ToC could appear in two different ways:

The format of the ToC should look like this:

Table of Contents

Top-Level Folder

An example using current project setup

Table of Contents

Getting Started

Build Tools

...

Outcome 2

Brainwrite

JordanMartinez commented 5 years ago

Update on this project:

First, I've learned how to use yargs, node-path, node-fs-aff, and string-parsers. The are still a few things with string-parsers that I don't understand. Since I didn't need to use them AFAIK, I'm not going to cover them further. Also, I did not figure out how to produce great parsing errors, so sometimes it was difficult to track down a bug/issue in the parser code itself.

Second, I realized that the nested nature of these headers can get pretty complicated to parse when one includes multi-line comments. For example,

------------
-- These are not an issue...

-- ## 1 - This is a header

-- ### 1.1 - This is a header

-- ### 1.2 - This is a header

------------
-- These possibilities do present an issue

-- ## 2 - This is a header

{-

### 2.1 - This is a header

-}

{- ### 2.2 - This is a header -}

{-
#### 2.2.1 - This is a header

#### 2.2.2 - This is a header

-}

-- #### 2.2.3 - This is a header

Third, there are situations where I might include '#' in a comment as a reference to the 'applyFlipped' function name: arg # function. I believe that occurs sometimes in a multi-line comment, so it makes it much harder to distinguish a header prefix from a regular function.

Thus, to keep things simple (though this could be improved in the future), headers in PS files will look like this:

-- ####... Header Text on regular single-line comment

{-
-- ####... Header Text inside a block-level / multi-line comment: make it look like a single-line comment
-}

{- -- ###... this is not a header -}
JordanMartinez commented 5 years ago

Another update on this:

JordanMartinez commented 5 years ago

Update Summary

The program works!

Things I've fixed since the last update:

Things I still have to do:

Other issues I found:

Current Output

You can see the output in the temporary table-of-contents.md file (link below). Be aware of these things when using that link:

The temporary table-of-contents.md file in the development branch.

JordanMartinez commented 5 years ago

Latest update:

Things left to do in this issue:

JordanMartinez commented 5 years ago

Latest update on this. I've refactored the code a bit in an attempt to verify that the file's URLs are valid. I encountered a 'Maximum Call Stack Size Exceeded' error, and it took some time to realize that the problem was caused by referring to the wrong variable.

However, there was some good news from this:

Overall, I don't think I like the program logic of my current not-yet-pushed design. On one hand, I'm trying to make it fast. On the other hand, that desire might make the code much harder for a new learner to understand.

JordanMartinez commented 5 years ago

Latest update on this.

In my initial design of this program, I walked the file tree once to parse the content and I walked the resulting structure a second time to render it. After having a better understanding of the design space, I saw that that was largely wasteful in CPU cycles. In the current implementation (now pushed to the development branch), I walk the file system once, parse and render files, and render directories as I return to its parent directory. The previous approach was more like OO in FP syntax whereas this second approach is more like the FP declarative programming style via traverse. (I guess that's why they say that Traversable is the answer to everything...) On top of that, I made it run in parallel thanks to Parallel and Aff. (Actually, running it in parallel means my logger is now incorrect.... Ha! Whoops!) I am also checking the file's URLs for a 200 HTTP code before I render them, so that if it fails, then I just render a plain text entry for that item in the resulting file. If it succeeds, I'll render it as a hyperlink using that file.

The code could still be cleaned up in a few ways:

Before making a new release, Travis CI should now run this program and fail if a single link is considered invalid

On top of that, there's a few other issues relating to new people reading this for the first time:

JordanMartinez commented 5 years ago

Rather than delaying a release, I'll just work on this more in separate issues.

JordanMartinez commented 5 years ago

Closed by #278