support nested lists

[wkeese]

According to http://stackoverflow.com/editing-help/#advanced-lists, markdown supports nested lists by indenting four spaces.

However, it doesn't work in our doc, for example dojox/string/BidiEngine.js:

//      As of February 2011, Bidi engine has following limitations:
//
//          1. No support for following numeric shaping options: 
//              - H - Hindi,
//              - C - Contextual,
//              - N - Nominal.
//          2. No support for following shaping options:
//              - I - Initial shaping,
//              - M - Middle shaping,
//              - F - Final shaping,
//              - B - Isolated shaping.
//          3. No support for source-to-target or/and target-to-source maps.
//          4. No support for LRE/RLE/LRO/RLO/PDF (they are handled like neutrals).
//          5. No support for Windows compatibility.
//          6. No support for  insert/remove marks.
//          7. No support for code pages (currently only UTF-8 is supported. Ideally we should convert from any code page to UTF-8).

Our doc is actually using tabs rather than spaces, so as an experiment, I tried switching to spaces, but that didn't help.

cc @AdrianVasiliu

[wkeese]

From testing at http://daringfireball.net/projects/markdown/dingus, lists and nested lists work, but the (outer) list cannot be indented. I fixed numerous violations in the dojo source related to indentation so I think that if the doc parser just strips the first two tabs, or whatever the indent is on the first line, then things will work.

[wkeese]

Also note that with Neil's parser nested lists were hacked like this (taken from dojo/selector/acme.js):

    //          * attribute queries:
    //          |   * `[foo]` attribute presence selector
    //          |   * `[foo='bar']` attribute value exact match
    //          |   * `[foo~='bar']` attribute value list item match
    //          |   * `[foo^='bar']` attribute start match
    //          |   * `[foo$='bar']` attribute end match
    //          |   * `[foo*='bar']` attribute substring match

That doesn't work anymore either.

[wkeese]

I'll work on a patch for this.

The first problem is here:

function processStandardKey(/**Object*/ metadata, /**string*/ key, /**string*/ line) {
    key = standardKeys[key];

    line = trim(line);

It's indiscriminately trimming all leading whitespace, where it should only be trimming the indent of the value (by our standards, two tabs). That value isn't being recorded at all, so this is more of a case of an unimplemented feature than a bug.

The second issue is that you are expanding tabs to 2 spaces, rather than 4. Not only do dojo developers use 4 spaces for tabs, but markdown requires 4 spaces for indenting lists (and we are using tabs to indent nested lists). Actually, it's more complicated than that, because for something like "//\t" it should expand to 2 spaces.

[wkeese]

https://github.com/wkeese/js-doc-parse/tree/dojodoc

I'll do some more checking tomorrow though, to make sure everything is OK, and maybe work on the return codes problem too.

[wkeese]

OK, that's the patch, seems to be working, @AdrianVasiliu you see any issues?

[csnover]

Same as #59