Add category backlinks - Githubissues

texastoland commented 5 months ago

Looks bigger than it is. I'll break it down!

`make_docs.nu`

Adds backlinks to categories from command pages (goal of this PR)
Embeds the command name and usage info in markdown instead of Vue template strings
Fixes an unescaped quote
(Most significantly) uses a new command-names command to filter what's available in nu --no-config-file instead of based on command properties
- Increased consistency between machines
- Needed for #1210

`commands/README.md`

Wraps URL in $withBase macro for forked preview builds (only needed in 2 places)

`commands/categories/*.md`

Generated $withBase

`commands/docs/*.md`

[!WARNING] Changes below depend on next release ⏸️

🆕 pwd command that was previously hidden (similar to stdlib)
🆕 Commands introduced this release
Removed extra warnings from commands that are now installed by default

fdncred commented 5 months ago

One thing to keep in mind is that you can't just generate command in linux/mac, there are windows only commands.

/cc I'd like @hustcer to approve and land since he does a great job maintaining the docs.

hustcer commented 5 months ago

The script fail to work for me on macOS:

Error: nu::shell::eval_block_with_input

  × Eval block failed with pipeline input
     ╭─[/Users/hustcer/nu-docs/make_docs.nu:382:46]
 381 │     let commands_group = ($commands | group-by name)
 382 │     let unique_commands = ($commands_group | columns)
     ·                                              ───┬───
     ·                                                 ╰── source value
 383 │     let unique_categories = ($commands | get category | uniq)
     ╰────

Error: nu::shell::eval_block_with_input

  × Eval block failed with pipeline input
     ╭─[/Users/hustcer/nu-docs/make_docs.nu:378:11]
 377 │         scope commands
 378 │         | join (command-names) name
     ·           ──┬─
     ·             ╰── source value
 379 │         | sort-by category
     ╰────

Error: nu::shell::eval_block_with_input

  × Eval block failed with pipeline input
     ╭─[/Users/hustcer/nu-docs/make_docs.nu:377:9]
 376 │     let commands = (
 377 │         scope commands
     ·         ───────┬──────
     ·                ╰── source value
 378 │         | join (command-names) name
     ╰────

Error: nu::shell::cant_convert

  × Can't convert to string.
     ╭─[/Users/hustcer/nu-docs/make_docs.nu:214:63]
 213 │             | each { |example|
 214 │                 let result = (do -i { $example.result | try { table --expand } catch { $in } } )
     ·                                                               ──┬──
     ·                                                                 ╰── can't convert binary to string
 215 │ $"($example.description)
     ╰────

My version info:

 version              0.91.1
 branch               nightly
 commit_hash          cc8f2b6419daa7c5c55fb8ca0ef257fc76be6674
 build_os             macos-x86_64
 build_target         x86_64-apple-darwin
 rust_version         rustc 1.75.0 (82e1608df 2023-12-21)
 rust_channel         1.75.0-x86_64-apple-darwin
 cargo_version        cargo 1.75.0 (1d8b05cdd 2023-11-20)
 build_time           2024-03-22 01:31:28 +00:00
 build_rust_channel   release
 allocator            mimalloc
 features             dataframe, default, sqlite, static-link-openssl, trash, which
 installed_plugins    formats, gstat, inc, query

texastoland commented 5 months ago

Hmmm I'm on release build I'll check nightly soon 👍🏼

texastoland commented 5 months ago

Hmmm I'm on release build I'll check nightly soon 👍🏼

I was indeed able to repro with nightly. But it happens on main too:

nushell.github.io on  main [⇕] is 󰏗 v0.0.0 via  v21.6.1 took 4s
❯ nu make_docs.nu
Error: nu::shell::eval_block_with_input

  × Eval block failed with pipeline input
     ╭─[/Users/texas/Developer/nushell.github.io/make_docs.nu:368:46]
 367 │     let commands_group = ($commands | group-by name)
 368 │     let unique_commands = ($commands_group | columns)
     ·                                              ───┬───
     ·                                                 ╰── source value
 369 │     let unique_categories = ($commands | get category | uniq)
     ╰────

Error: nu::shell::eval_block_with_input

  × Eval block failed with pipeline input
     ╭─[/Users/texas/Developer/nushell.github.io/make_docs.nu:363:9]
 362 │     let commands = (
 363 │         scope commands
     ·         ───────┬──────
     ·                ╰── source value
 364 │         | where is_custom == false and is_extern == false
     ╰────

Error: nu::shell::eval_block_with_input

  × Eval block failed with pipeline input
     ╭─[/Users/texas/Developer/nushell.github.io/make_docs.nu:363:9]
 362 │     let commands = (
 363 │         scope commands
     ·         ───────┬──────
     ·                ╰── source value
 364 │         | where is_custom == false and is_extern == false
     ╰────

Error: nu::shell::cant_convert

  × Can't convert to string.
     ╭─[/Users/texas/Developer/nushell.github.io/make_docs.nu:200:63]
 199 │             | each { |example|
 200 │                 let result = (do -i { $example.result | try { table --expand } catch { $in } } )
     ·                                                               ──┬──
     ·                                                                 ╰── can't convert binary to string
 201 │ $"($example.description)
     ╰────

nushell.github.io on  main [!?⇕] is 󰏗 v0.0.0 via  v21.6.1 took 5s
❯ version
╭────────────────────┬──────────────────────────────────────────╮
│ version            │ 0.91.1                                   │
│ branch             │ main                                     │
│ commit_hash        │ b2c5dc204a851c7d2a7cd07b2b3157ba9e1f2173 │
│ build_os           │ macos-x86_64                             │
│ build_target       │ x86_64-apple-darwin                      │
│ rust_version       │ rustc 1.75.0 (82e1608df 2023-12-21)      │
│ rust_channel       │ 1.75.0-x86_64-apple-darwin               │
│ cargo_version      │ cargo 1.75.0 (1d8b05cdd 2023-11-20)      │
│ build_time         │ 2024-03-25 21:20:31 -05:00               │
│ build_rust_channel │ release                                  │
│ allocator          │ mimalloc                                 │
│ features           │ dataframe, default, sqlite, trash, which │
│ installed_plugins  │ explore                                  │
╰────────────────────┴──────────────────────────────────────────╯

nushell.github.io on  main [!?⇕] is 󰏗 v0.0.0 via  v21.6.1
❯ git log -n 1 -- make_docs.nu
commit 9a2a1191d5196ea74020390e38221902d9b2b4b0434^G
Author: Stefan Holderbach <sholderbach@users.noreply.github.com>
Date:   Sun Mar 10 17:29:54 2024 +0100

    Remove feature `extra` detection from make_docs.nu (#1294)

    If we sunset the `--features extra` for building Nushell we don't need
    to warn in upcoming releases anymore.

    Depends on https://github.com/nushell/nushell/pull/12140

    cc @hustcer: Does this simplify the command generation enough so that we
    don't need a parallel clone of the main repo anymore (you run it
    locally, right?)

@fdncred It's either a regression or breaking change. I'm investigating now and will report an issue in Nushell and/or PR a separate workaround ASAP.

AFAIK this and #1310 are ready to 🚢

texastoland commented 5 months ago

Minimal repro:

❯ let repro = scope commands | where name == 'bits and' | get examples.0.result

❯ $repro | each { describe }
╭───┬──────────────╮
│ 0 │ int          │
│ 1 │ binary       │
│ 2 │ list<int>    │
│ 3 │ list<binary> │
│ 4 │ binary       │
│ 5 │ binary       │
╰───┴──────────────╯

❯ $repro | each { table }
Error: nu::shell::cant_convert

  × Can't convert to string.
   ╭─[entry #1:1:17]
 1 │ $repro | each { table }
   ·                 ──┬──
   ·                   ╰── can't convert binary to string
   ╰────

❯ $repro.1 | table
Length: 2 (0x2) bytes | printable whitespace ascii_other non_ascii
00000000:   89 89                                                ××

❯ $repro.1 | to nuon
0x[8989]

❯ 0x[8989] | table
Length: 2 (0x2) bytes | printable whitespace ascii_other non_ascii
00000000:   89 89                                                ××

❯ 0x[8989] | each { table }
Length: 2 (0x2) bytes | printable whitespace ascii_other non_ascii
00000000:   89 89                                                ××

❯ [0x[8989]] | each { table }
Error: nu::shell::cant_convert

  × Can't convert to string.
   ╭─[entry #1:1:21]
 1 │ [0x[8989]] | each { table }
   ·                     ──┬──
   ·                       ╰── can't convert binary to string
   ╰────

❯ /usr/local/bin/nu -n -c '[0x[8989]] | each { table }'
╭───┬─────────────────────────────────────────────────────────────────────╮
│ 0 │ Length: 2 (0x2) bytes | printable whitespace ascii_other non_ascii  │
│   │ 00000000:   89 89                                                ×× │
│   │                                                                     │
╰───┴─────────────────────────────────────────────────────────────────────╯

❯ for $it in [0x[8989]] { $it | table | print }

❯ for $it in [0x[8989]] { $it | print }
Length: 2 (0x2) bytes | printable whitespace ascii_other non_ascii
00000000:   89 89                                                ××

It's either a regression or breaking change. I'm investigating now and will report an issue in Nushell and/or PR a separate workaround ASAP.

Both incoming 🏇🏼

fdncred commented 5 months ago

AFAIK this and https://github.com/nushell/nushell.github.io/pull/1310 are ready to 🚢

I'm not sure why we'd land this if we know that the script doesn't work.

texastoland commented 5 months ago

But it happens on main too:

@fdncred The scripts in the website's main branch are also broken using Nu nightly:

Nu version	before PR	after PR
release (Homebrew)	✔️	✔️
nightly (commit hashes above)	❌	❌

Continuing in nushell/nushell#12287. This PR doesn't introduce changes related to that error.

texastoland commented 5 months ago

@hustcer The script fail to work for me on macOS:

Resolved in nushell/nushell#12294. Should work again with latest Nu build 🚀

@fdncred Pending release then we (I don't currently have Windows set up) should run the script again.

IFL there's a way to automate by scheduling jobs on Nu release in VMs for both OSs 💭 Already explored?

hustcer commented 5 months ago

@texastoland There are some new commands that haven't been release yet:

A       commands/docs/debug_profile.md
A       commands/docs/plugin.md
A       commands/docs/plugin_list.md
A       commands/docs/plugin_stop.md
A       commands/docs/pwd.md
A       commands/docs/uname.md

We should remove the docs for these commands, or just submit make_doc.nu in this PR, or wait until v0.92 been released

texastoland commented 5 months ago

Changes below depend on next release ⏸️

Thanks I added a notice to the description 🙏🏼

hustcer commented 5 months ago

@texastoland Don't forget upgrade to v0.92 after it was released and regenerate the docs again, then we can land this PR.

texastoland commented 5 months ago

Updated :shipit:

nushell / nushell.github.io

Add category backlinks #1312

`make_docs.nu`

`commands/README.md`

`commands/categories/*.md`

`commands/docs/*.md`