We need a channel for #WorkInProgress or something

[fj128]

I want a Slack channel (or whatever we are going to use the next time) where I can see a post saying "I've implemented data_files.py that has a lightning_problem(n) function that gives you the data, you all should use it". And other similar posts.

Even with our disastrous failures to implement the necessary stuff, there was more failure on top of that, I remember @manpages being uncharacteristically rude about someone's code storing problems in several different ways while being unaware himself that we actually had had the right way to access problems for the past 12 hours or something.

It is impossible to restrict the main channel for any sort of on-topic discussion and offload off-topic discussions to specific channels (like "lightning", "solvers" or whatnot). The main channel is going to be chaotic, deal with it.

But I think that we could do with a channel dedicated specifically to announcements of new functionality in code that improves everyone's lives, as far as writing code specifically goes. Like, "Diff now has an operator for diff[axis] for axis in [0, 1, 2]". That kind of stuff.

[Vlad-Shcherbina]

I like this a lot.

I was unsure how much I should spam https://github.com/Vlad-Shcherbina/icfpc2018-tbd/issues/10#issuecomment-406808394 to get everybody who writes solvers to read it. It would be nice to have a place where I could post it once and be certain that everybody who needs it will see it.

It appears to me that it has better chance at working than earthdok's OKR document idea (#24). First, slack channel is more visible, you are reminded of its existence every time new announcement is made. Second, it's easier to remember to post functionality announcement because of the desire to get your stuff used by others. No need for "I'm off to bed" announcements.

It will be important to limit noise in this channel, it will only work if everybody can afford to read it.

[fj128]

I think that we need a simple single-page wiki for tracking these things in addition to the channel. Github issues are wholly inadequate to the task IMO, they are too big and coarse and get everywhere don't provide a big picture. Actually not a wiki but README.md or another single file.

edit: so pretty much exactly @earthdok's OKR document idea (#24) lol

Something like

• Commands: implement classes for commands
  • classes
  • parse and compose
  • In C++, for hypothetical C++ solvers
• Files
  • load models
  • load/write traces
• Emulators:
  • interface for getting filled/volatile cells, try to apply command
  • implementation in Python
  • C++
• Basic bot utilities
  • move from A to B
  • fission and fusion
  • compose multiple bot command streams
  • bfs/A* for movement
• ...

So basically a post in this channel more often than not corresponds to something getting an emoji checkmark (and possibly a new line, if you couldn't be bothered to add it before starting working on it) in that wiki page, with a link to the file where it was implemented and a description in a comment there.

[Vlad-Shcherbina]

To contrast with earthdok's OKRs, the emphasis here is on what's done, not on who is working on the stuff that's not done, is that correct?

---

My concern is the possible redundancy between this document and the source itself, assuming it's well structured and documented. Redundancy will result in this document being out of sync with reality. To avoid that, it should be limited to high-level overview that does not fit at the lower levels of the hierarchy.

How about this:

• The documentation for a bot utility function goes to this function's docstring. Or this function is so obvious that it does not require a docstring.
• The overview of the bot utility functions (for example, how they can be used together to achieve stuff, or what core concepts the rely upon) goes to the docstring of the bot_utils.py module. But there is no need to mechanically list all bot utilities in the doc because they are already listed in the code. And no need to give any details about the individual function because it's in its docstring already. Perhaps it would make sense to put the main() function with the usage examples at the beginning of the module, right after the docstring, to make it more prominent.
• The overview of the important modules goes to the docstring of the package's __init__.py for continuity. Or maybe into package/README.md for discoverability.

Advantages: no redundancy, and it encourages people to read the source (which is a useful habit because you can't document everything).

[earthdok]

My concern is the possible redundancy between this document and the source itself, assuming it's well structured and documented.

I'm somewhat confused by your comment. I think you're misunderstanding fj's idea as if e.g. "classes" should be the actual list of classes documented in the wiki page. Whereas the idea as I understand it is that is should literally say "classes". As in, "command classes have been implemented". You can't go any higher-level than that.

My concern was precisely that figuring out the status of a particular feature involved some combination of the following:

• reading the chat history
• checking the issue tracker
• checking git log
• looking at the repo dir trying to make sense of all the new files
• reading someone else's code.

These are ordered roughly in the order of increasing annoyance. It should definitely be possible to get the big picture without having to read any code.

[Vlad-Shcherbina]

Ok, yes, if it does not repeat the layout of the source tree too much and does not list too many things that's fine.

[Vlad-Shcherbina]

Perhaps instead of scratch/production distinction we should think in three categories:

1. Anything whatsoever. No constraints. That's what the scratch is meant for.
2. Stuff easily runnable by anybody. This means no bash, no PowerShell, no Haskell, no operations on local files using relative paths, no exotic dependencies, etc. It could be readily run on any development machine, in the cluster, or packaged into the final submission if necessary. Examples include individual solvers, solver worker, the dashboard, submission script, test_all. Requirements: should work out of the box, optionally (where appropriate) should be covered by smoke tests so it's not broken by refactorings. Does not have to be extensively documented or tested.
3. Stuff easily reusable by anybody. Examples include Pos, Diff, Command, the mysterious emulator state, the pathfinder, orchestrate2.py, data_files, pyjs_emulator, some of the solver_utils. Requirements: nice agreed upon interface, should be announced in the channel, should be mentioned in the overview doc, should have docstrings if not obvious, functionality should be reasonably tested.

Producton makes no distinction between 2 and 3, and probably no point trying to expess it in the source layout, but this could be useful categories in our heads.

[fj128]

Yes, I meant that as literally what could be in that file at some point in this contest, word for word.

I think that coordinating work (who works on what) should be done in some transient channel, the purpose of the file should be more to give an overview of what's already done and what should be done in the near future.