Using the Project Wiki as a knowledgebase

[TerryE]

Missing feature – Community FAQs and cheatsheets.

This was raised in #3118, but really merits an issue of its own rather than hijacking an issue about the compatibility of some Lua modules.

Justification

There is a class of content hidden gems or cheat-sheets that where I feel that the user community would benefit from having them accessible through some central project resource but make not so formal as that used to create the docs tree and published to the ReadTheDocs website. My view is aligned with @marcelstoer in the the project Wiki is a good vehicle for this. This being said:

• We really need to get rid of the obsolete crap from this wiki tree
• We need to define some structure and have a loose but still present editorial control of added and updated content.

Control of Content

Each project can elect to make write access to it's wiki either limited to project committers or to all GitHub members. We currently are set to the later, but IMO we should use the former. IMO (but again I would accept other members' views here) we should still use the issues vehicle to discuss changes or additions to the wiki content (e.g. as the individuals to propose content in MD format as a Gist, but use committers to fold this into the wiki.

Take @jmd13391's cheatsheet mentioned in #3118, the content looks good, but the initial format needed a bit work to make it readable. I am note sure that we should be doing this editorial pass to be done make it readible, which I've just done (format not content), but this does highlight an issue. We as a project can elect to make the Wiki updatable either by project members only or by any logged on GitHub member; we have it set to the former. My suggestion would be that only the project committers have write access and that we use the issues list as the vehicle for content submission.

[marcelstoer]

We currently are set to the later, but IMO we should use the former.

I do not share your opinion on this. I feel we shouldn't control more than absolutely necessary. If we have to discuss changes to the wiki through the issues list and restrict write-access to committers then it's pretty much the same as for our docs. I see value in having a "true" open-for-all wiki that the community can contribute to in a more agile way than pages in the official documentation. No bad things will happen if we give that much power to our users.

[TerryE]

I can see the +/- over this committer vs open debate, so I don't have a strong view on this, but whatever the write access rules, the current state of the wiki is crap. It desperately needs a bit of tender loving care, and active moderation.

An example: we have some ~SDK 0.95 vintage partial translates of the documentation, so incomplete and dated to be worse than useless. Google Translate (web page mode) on the live readthedocs site (for example this is the French translation of the node module) is far more usable. Surely the best thing that we can do is to delete these extremely obsolete Russian , Spanish and Chinese pages and have a explanation of how to use Google translate to view the project documentation in your own language.

It would also be good to agree some overall structure and publication guidelines, so we have some minimum level of coherence and ease of navigation / usability. Maybe we can get a volunteer from the non-committer contributors to help here? I am sure that both you and I and possibly some of the other contributors would be willing to support this effort, but at the moment there is just too much for me to do in areas where I am really the sole expert for me to take this on as well.

There are quite a few techniques that can help for example the wiki itself is a straight git repo that be maintained using standard remote functionality, for example this HOWTO explains how to do bulk update using a local copy and the GitHub wiki help tree gives loads of detailed documentation.

[TerryE]

Another point is that unlike MediaWiki (Wikipedia) the GitHub wiki has no concept of talk pages for contributors to discuss content pages; the only analogue is using an issue, so it makes sense to maintain an open "Wiki" issue for each actively maintained Wiki page.

[pnkowl]

Since the actual wiki link has not been provided in this post yet, here it is

https://github.com/nodemcu/nodemcu-firmware/wiki

Please consider the following as brainstorming (no bad thoughts). And not what I recommend we should do (irrespective of how it reads). Only when we get all the thoughts out will the best path hopefully emerge.

Goals

• community supported
• less formal than readthedocs
• must be developed so that it does NOT become just another make more work for @TerryE or anyone else who feels overly burdened
• designed to complement readthedocs, firmware cloud builder, LFS webservice. To try to never duplicate content
• a place to pose topics (ask questions) in an orgainzed and connected manner where we first describe what we think we should know (and what we do) and then allow the content (answers) be developed and fleshed out by the community

An open-as-possbile wiki

• I would vote for an "open" wiki and see how that works. If things go badly in some manner, then investigate the root cause (e.g. poor requirements/design/guidelines/howtos vs no other place to discuss some topic vs bad actors). Only then, increase restrictions in a targetted manner.
• I like to think we are dealing mostly with facts, and as such "editorial control" should not be necessary. To the extent that "bad coding practices" are being shared, then a section of the wiki should be devoted to "best practices". Sometimes "bad examples" are as helpful as good ones allowing one to explain the undesirable results in real verifiable terms. I would rather be working and posting in the wiki than opening an "issue".

Keeping the old

If organized, I would suggest keeping the existing wiki content, but provide a single link from the new main page to the old content. With the caveats, that "we" considered doing this: "getting rid of the obsolete crap from this wiki tree" but in deference to those that came before, you can find it "here" (link). Later on, it will show how far we have come.

What Content (Categories)?

We need to define some structure. Okay, But before that, what kinds of content do we want to host? For me, it would be a place that collects everything that is not covered elsewhere that is important to someone

• stuff in readthedocs that could benefit by a more open environment. e.g. white papers, how tos, hidden gems, cheat-sheets
• real life working examples with comments for every command, module, function,... with no thinking required. There are too many instances, even with my own code, where I have to think/rethink every time. No, this is not going to get built out all at once, and may never be "complete". The low hanging fruit will be picked first. "Empty spots are okay".
• a section for community testing. What do "we" guarantee works? What has been tested? How? What platform? For example: FTP (simply native linux, or filezilla), winscp not supported. How have users configured each for success?
• complete the "exercises left to the reader" in readthedocs
• useful code snippets and explanations of frequently used Lua (e.g. patterns still make be very uncomfortable -- almost magical in nature). All explained in the context of nodemcu.
• building blocks (e.g. telnet, ftp, webide,... I am working on an device IDE having telnet, web, ftp, several levels of panic traps, LFS reloader, boot and application logging, a good bit borrowed from others, that I have high hopes for at least helping me manage and interact with my devices remotely, and without additional intrastructure -- think mqtt).
• just getting started and want to know the benefits and constraints of choosing a particular development path (firmware cloud builder has a 19 module limit. LFS webservice allows 50 Lua files. Service level for each should be outlined. History of issues should be transparent. The filesystem on a 4M device supports well over 5000 files, but myvar=node.filelist() will panic the device well before then. The struggles in the very contrained ESP environment never end. But with proper forewarning and planning, it is absolutely wonderful.
• where appropriate, traceback to "issues" (i.e. if you want to understand the backstory, see here (link). The wiki is a great place to (attempt to) summarize an issue (may span a number of issue #s) and provide the traceback while associating the topic in a logical structure. Issues are like FAQ. See below.
• debug techniques (e.g. using LFS webservice and get "syntax" error with no explanation, then what? How to use Ace (editor) to assist in simple issues, or the device compiler without fear of a syntax panic loop after an accidental restart
• a whole section on rules-of-thumb regarding all things heap related?
• Can the wiki aid in curating/discussing questions about behavior (that may become an "issue") regarding make/firmware/LFS/function/module in the wiki before issues are "formally" raised? How can competence in the easier areas be fostered such that many times, I would image, many folks are silent waiting for @TerryE to respond with the most definitive answer. Is there a "best practices" for investigating "issues"? Perhaps that goes in the wiki (what is the responsibility of the community when an issue is initiated so that folks do not duplicate effort or othewise waste their time)?

Structure

Once we have a skeleton list of types of content (desireable but not hosted anywhere else), then comes structure. But this I believe comes from why folks would be visiting the wiki at all and how they find their way there.

• want more detail or help than was provided in readthedocs? Click this link in readthedocs, which provides the connection(s) to applicable pages in the wiki
• just getting started section on simple cool things to try
• ways to approach classes of problems
• perhaps the categories above suggest a structure

Hosting Application Capabilities

• is github wiki the right place to do this (I have no idea)?
• how easy is it to use?
• what are the types of workflows (how to add code and comments may be entirely different than a whitepaper) that should have a "best practices" template. Are white papers coded in-line or is it more appropriate to use PDF or word or HTML or support an agreed to set?
• Wiki analytics? Does github wiki provide page analysics (e.g. page hits, flow) to see where content is appreciated? How about readthedocs?
• How does collaboration play out? I would suspect that most topics will have 1 or 2 folks who are both interested and can invest time in page creation (of a specific topic). After that, the page will be polished and will evolve at the hands of the community.
• Depending on the method to actually create the pages, I would have a tendancy to collaborate elsewhere (think google docs) and then port the pages, but who knows. The ease of page creation will make or break this effort. Requiring an advanced degree in page creation will just not do.

FAQ

Lastly, I would prefer not to have an FAQ, or if we do, then require every entry to have answered the question "why not somewhere else"? FAQ is a lazy structure. That being said, I think it is perfectly fine if contributors have no idea where to put their content. This is where curation comes in (the "control" part). Moving stuff where is best fits. Making the connections (hyperlinks), between related but not obvious ideas.

Next Steps

• It would be nice if after the framework was discussed and accepted (TBD) that this would be a community effort. I would think that if a structure (helpful containers and flow for content) is provided, the community will come and help where they see the need and the act of contributing is not onerous.
• I have not been around very long, but has there been a skills self assessment for the existing core contributors? Or a curated list of opportunities (problems) facing nodemcu. How is work shared, balanced or allocated? How much human capital is available to draw upon?
• Assess how much content already exists that is looking for a home
• How do we connect with folks that are not completely satisfied with the status quo? With either their understanding or what is provided currently, and want to help.

I am interested in this work, but there is much I do not know.

[TerryE]

This reads like

• You've become familiar with MD syntax
• You are interested in doing such a moderator role
• IMO, your level of engagement has given you enough karma to qualify yourself.
• The wiki is a very simple tool compared to MediaWiki or even DoWiiki say. It is really just a navigation layer around a Git repo and the GitHub MD syntax renderers.

You will find my email address in the git logs for this project. Ping me an email.

[TerryE]

Looking at the git log for the wiki @Pavlines did the Russian translation but he hasn't been active on GitHub for over a year. @66740416 did the Chinese translation, and @yeffrimic did the Spanish page.

I disagree with you, @pnkowl, when it comes to leaving truly obsolete stuff visible on the wiki. You yourself have emphasised the confusion about the small changes between the master and dev. SDK v0.95 stuff is 4 years out of date. Better removing it, IMO. Given that everything is in git then nothing is truly deleted, but it is more just hidden. Why not just document how to read the current documentation using Google translate?

[marcelstoer]

the current state of the wiki is crap

👍 I'm ok deleting everything that's outdated.

It desperately needs a bit of tender loving care, and active moderation.

👍 I'm not sure about "active", though. We have so much other stuff on our plate that looking after the wiki should be the least of our priorities. I don't have the energy to discuss this at great length but if I had to sum my position it would be something like "we look after code & docs and the community is invited to add additional content to the wiki". I could even live with closing it altogether (apart for "Lua module registry"-like pages) to avoid developers having to check two sources (docs & wiki) for information.

[TerryE]

My thinking is that if one or two of the active developers who are not core committers want to take this on then this will be of overall benefit to our user community. I would happy to support them with a little monitoring and troubleshooting advice.

[pnkowl]

avoid developers having to check two sources (docs & wiki) for information.

One of the most important goals of the wiki may be alignment with docs (and vice versa). To the greatest extent possible, the goal should be no duplication of information. Each should simply complement the other.

Providing a method to allow users to seemlessly pass back and forth between docs and the wiki should also be a goal -- hyperlinks seem most natural.

It appears that the wiki is not associated with a branch (e.g. the URL does not contain dev, master) so handling the comingling of multiple "releases" may be one an obstacle discussed early on.

[TerryE]

I have added the project label Wiki, and I suggest that we label any wiki-related issues with this label. This makes it easier to select (or ignore) such issues. Given that we will only have a small number of (actively edited) pages, I don't see the disadvantage in using this issues list as the discussion vehicle.

Wiki Feature Set and Omissions

• No WYSIWYG editor. The wiki uses the standard Write/Preview pain editor, and IMO this really encourages micro-save editing chunk which is a bit of a for audit and review purposes. My suggestion is that any contributors prepare their content off-line and only submit on a reasonably chunked basis. There are various offline and in-browser MD editors available.
• Limited Formatting. The internal format is GitHub markdown syntax. Ever though you can nominally use other markdown formats such as MediaWiki markdown, in really only the feature set of GitHub markdown is supported so all of the extra markdown features supported by the MediaWiki engine don't work. Ditto HTML. For this reason, I suggest that the just stick to GitHub markdown syntax for all of out wiki pages.
• No automatic navigation such as auto-generated contents, categories, etc. No crumb trails. The wiki does support a wiki-wide sidebar and footer, but that is all. Any navigation has to be hard wired into page content.
• No change monitoring and notification. There is now default mechanism to get notified of changes to the Wiki. However, if you do have a local clone then standard git features allow you to run, say a daily cron job to generate this notification using git log HEAD origin/master -1 etc. I will do this for myself to get these email change notifications. If anyone else wants a copy of this notification then email me.

Current Wiki Pages

Page	Main Editor	Notes
Home.md	@marcelstoer	Main wiki landing page
_*.md	TDB	Sidebar and Footer
Lua-modules-directory.md	@marcelstoer	Empty placeholder
Notes-about-writing-docs.md	@marcelstoer	An almost empty template for writing readthedocs externsions to MD docs
ESP8266---ESP32-Compatibility.md	@spiritdude	Last updated over 2 years ago.
[DRAFT]-How-to-write-a-C-module.md	@TerryE	Replacing this is on my TODO list.
Windows * cheatsheet	@jmd13391	Cheet sheet getting started on Windows development
nodemcuapi*.m	N/A	Obsolete Langage pages to be deleted

Next steps

It would be really useful so get some consensus on an overall structure and page naming guidelines. Given the lack of automated links this should be as simple and flat as practical.

I will yield the floor to other commenters.

PS: Here are a couple of Lua FAQs (general and not NodeMCU specific: Lua Unofficial FAQ (uFAQ) and Lua Users FAQ wiki

[TerryE]

Given that nothing has been done to this wiki by the committers or the other posters on this issue, I think that we can tried this a pretty dead. I've just deleted the CH, CN, and RU NodeMCU API pages since these are very fragmentary and over 6 years out of date. We don't even off the version 0.9 firmware that they describe anyway. Any non-English reader would be far better off using Google translate on the current online documentation.

I am minded to close this issue, but leave the wiki moribund. @marcelstoer, what do you think?

[marcelstoer]

Thanks Terry, I don't oppose you closing it. However,

• https://github.com/nodemcu/nodemcu-firmware/wiki/%5BDRAFT%5D-How-to-write-a-C-module has been in draft mode ever since it was created. Is it still relevant and accurate? Also, it doesn't say but it assumes ESP8266 (rather than ESP32). It deserves a dedicated discussion but to me anything ESP8266 is "maintenance only".
• I appear to have committed https://github.com/nodemcu/nodemcu-firmware/wiki/Windows-Docker-LFS-esptool-ESPlorer-Workflow-Cheat-Sheet but the content certainly isn't mine; no idea about the origin anymore. Delete it?
• https://github.com/nodemcu/nodemcu-firmware/wiki/ESP8266---ESP32-Compatibility may have been incomplete but accurate at some distant point in the past. I propose deleting it.
• https://github.com/nodemcu/nodemcu-firmware/wiki/Lua-modules-directory -> looks like it was mainly myself who thought this be a good idea. I propose deleting it.