Address reviewer's comments

[kostko]

Reviewer 1

Overall the "work" is valuable and it is extremely relevant with the call for paper. However the "paper" is a little bit boring. The authors have difficulty in simply explaining at higher level the system architecture and get the discussion in too much details, in order to motivate any choice. It would be better to introduce an overview section with highlights all the user/administrator steps from the user decision to participate to the community network up to the time when her node is operational. By sequentially of describing each piece of the (good) system, the reader have a complete view only at the end. And this discourage the reading. In would be better to provide a bird-eye-view at the start of the paper.

• [x] Add a high-level overview section containing a story of how a user deploys nodes in a community network.
• [x] Add a schematic overview that goes together with the story.

  Reviewer 2

Please discuss: 1) how nodewatcher with hardware diversity and novelty. Which are the constrains (e.g. OpenWRT like firmwares required? Implications of requirement of having to install a daemon in each node). Recommendations for new adopters and specially for (massive) migrations.

• [x] Highlight the case for platform extensibility and show an example of how we can support other platforms like Mikrotik.
• [x] Highlight the fact that there is no inherent requirement to install a daemon in each node as for example SNMP may be used instead through a module.
• [ ] Discuss importing from other systems.

Please clarify (and discuss the consequences if appropriated): 1) if all nodes need to see each other. I.e. how isolated clouds are handled, does each need a nodewatcher? if so, what if two nodes merge. Federation. if it splits?

• [x] Highlight that there is no need for the nodes to see each other. It is just that either nodewatcher needs to be able to reach the nodes (in case polling is used) or the nodes need to be able to reach nodewatcher (in case push is used).

  2) if how to develop "monitor pipelines" presented Fig.6 is well documented, e.g. to develop one for BGP

• [x] Describe how we can currently support multiple protocols. We support both OLSR and Babel with ease, show that additional protocols are easily added.

  Reviewer 3

Particularly in the former aspect, the review given by the paper could be significantly enhanced. The paper covers in too great detail technical engineering decisions to achieve the somehow common objectives of modularity and extensibility, which are of course noble goals. But it reads a lot like a white paper or technical report that praises a bit too much its own achievements without providing sufficient quantitative or qualitative comparison or overview of existing solutions, weaknesses, unaddressed aspects or yet achieved disseminations and impacts.

A table summarizing characteristics of existing management solutions would be desirable, (e.g. showing features, addressed issues, concerns, openness, usage, age, developer community size, ...).

• [x] Prepare a table that compares all the different existing management solutions (through time, features, programming language, license, ...).

For example the guifi.net management system is able to generate firmware configurations (building of the entire firmware is due to the closed-source policy of Mikrotik, a different issue than for openWRT based systems) from a web-user interface for many different devices. But the paper claims this as a somehow unique feature of the nodewatcher platform.

• [x] Highlight why having a complete firmware in addition to just configuration is beneficial (testing the configuration together with the exact software versions and only upgrading everything at once, ensuring that it works correctly).

Another interesting comparison would be on the implications of different designs approaches. One examples is the issue of centrality, single-point of failure, and security which seems pretty neglected in the whole discussion but, to the best of my knowledge, plays a very important role in community networks.

• [x] Highlight that network operation does not depend on there being a working nodewatcher instance. Even if one fails, the network will still work.
• [x] Mention security aspects, we are in the process of adding node authentication as part of the contract with OTI.

Another critical implication seems that the proposed system mandates exclusive configuration rights as any manual parameter change by a node-admin would violate the foreseen approach. This could also be considered a real burden when trying for example to test different parameters (eg when looking for link channels with least interference and highest TP in a given environment).

• [x] Highlight that nodewatcher will only display warnings, it will not rollback any custom configuration. If the node admin knows what he is doing, he may simply mark the warnings as resolved. And then when the admin finds the best settings, he may just save them into nodewatcher in order to persist them across upgrades.
• [x] In the future, nodewatcher may also help with link planning by having wireless survey information available from nearby nodes and can therefore estimate spectrum usage.

It remains unclear to what extend the proposed system can indeed achieve its promises. An overview of successful adoptions by other communities or completely different (unforeseen by the original developers) scenarios would be interesting.

The authors could describe in more detail since how long and with which acceptance and experience the new system (nodewatcher v3) has been used in their own community.

• [x] Explain (or highlight as I think that we already say so) that we have been using the system in parallel since January 2015. I think we also currently list experience with adoption from the Croatian network.

[kostko]

Ok, so my plan for the weekend is to address some comments raised by reviewers 2 and 3. Can you two look into that high-level overview and schematic?

[lukacu]

I started thinking about the illustrations yesterday, but I have some questions. I will probably prepare some drafts and then we can discuss them.

[lukacu]

So I have a question about high-level overview: is it supposed to be a step-by-step illustration of how a user joins a community network by registering a node, flashing it and installing it somewhere? Will this be the story? We can add some detailed steps like firmware builder and tie them to the specific sections in the paper.

[kostko]

Yes, I think that something like this should be in the high-level overview? @mitar? I also like the idea with referencing specific sections, so that it is immediately apparent what happens where.

[mitar]

Yea, I agree. I would also maybe then have a diagram of a node life-cycle. Probably the same. So things do not end with installation. But then you start monitoring, observing warnings, and if a node fails, you can reflash it with new device (potentially different hardware completely), but same configuration.

On the other hand, it should be clear that this is only one possible way to do it and nodewatcher can support others as well. Maybe we should have this diagram in a section explaining wireless community network building and then an example how we do it in wlan slovenija, and referencing the sections. But it should be clear that this is just our motivation, and not required.

[lukacu]

Node life-cycle is already illustrated in Figure 1, we should not make redundant plots. Besides, going into too many details about possible, but less likely scenarios will just make the overview complex and dissolve its meaning.

Another question is where should we place this overview in the text? If it is too early then we will be describing the general idea of a generic community network. If it is too late it will not be an introduction any more.

[mitar]

Yea, maybe we should expand Figure 1 then. Like have the social part which leads to technical part, which is then current Figure 1.

One other thing. I think also we need steps before registering a node. So a person learns about the network, wants to participate, come to the site, sees other nodes, wants to add their node. Then they register, and so on.

Not sure how to make this in a diagram. Probably it can be just in text. But I think that the text will have to discuss the whole social and bottom-up aspect as well.

[lukacu]

I guess that social aspects are important, but I would not mix them into the overview. The overview has to be short and if you want to discuss the social aspects it can point to a section that discusses them further. But we should focus on comments and those say: the paper does not have an overview that would motivate the technical details - provide one. I do not see a request for detailed social analysis, so that can be another paper :)

[mitar]

How I read is that it also do not have an explanation how a community setup network is done, and what is different from traditional networks. This is what I have in mind when I talk about "social" aspects.

But yea, diagram probably cannot have that anyway. I can write a bit about that then in the text around it.

[kostko]

Attaching the proposed design for Figure 1 by @lukacu for comments, so it will not get lost in the e-mails.

I like it as it clearly shows where the community is involved. I think that this was also one of @mitar's early comments (while we were preparing the initial paper), that Figure 1 should somehow include people. And now it does :-)

So I assume that a, b, c, d, e and f would then be explained in the figure caption?

[lukacu]

Yes, the idea is to tie in the letters to the description in caption or in text. I think that this figure still captures the cycle in the current figure, but is less abstract. If the community looks too uniform I can make more silhouettes but it will take time. If you think that something is missing I can add it, but the figure is pretty full already. On 20 May 2015 10:10, "Jernej Kos" notifications@github.com wrote:

Attaching the proposed design for Figure 1 by @lukacu https://github.com/lukacu for comments, so it will not get lost in the e-mails.

[image: deployment-3] https://cloud.githubusercontent.com/assets/543739/7721457/1a3750c8-fed8-11e4-84bb-096349fca95f.png

I like it as it clearly shows where the community is involved. I think that this was also one of @mitar https://github.com/mitar's early comments (while we were preparing the initial paper), that Figure 1 should somehow include people. And now it does :-)

So I assume that a, b, c, d, e and f would then be explained in the figure caption?

— Reply to this email directly or view it on GitHub https://github.com/mitar/paper-nodewatcher/issues/1#issuecomment-103803899 .

[kostko]

I see that you have included the new figure, it looks great! There is just one small typo, it says "mantainer" instead of "maintainer".

[kostko]

BTW, there has really been a lot of feedback from other communities, so we now really have a huge feature table:

• https://goo.gl/uHPjir

[kostko]

I have created a snapshot of the above table and marked the items, which should be checked:

• https://docs.google.com/spreadsheets/d/1UcuZTDeNV2kzWhHroJeaiK2kTidB3P70vxBzLdrPmyI/edit#gid=0

[mitar]

Can you add notes to the table what you think is wrong with each?

[kostko]

Updated.

[kostko]

@lukacu great work on the table!

Btw, Figure 1 previously had a feedback arrow between the generation and configuration stages. This arrow is very important and is also mentioned elsewhere in the paper. Can it be added back somehow (it doesn't need to be so fat)?

[lukacu]

Sure, I will update it, I can also make these arrows bidirectional, but I guess that would be too confusing. In any case we should update the text so that this is more clear.

[kostko]

I have now done some updates to the table and described it in text.

[kostko]

BTW, the illustration showing the differences between proprietary and community networks is really nice.

[kostko]

I have now updated the revision letter, adding some responses. You can check if these are ok.

The missing responses are now to Reviewer 1 and the last question of Reviewer 3 about the use of the platform in our community and elsewhere. For this last comment, I think that we should highlight that the Croatian community is using it and we have been able to easily support some new devices, which were previously unknown to us (the lantiq platform). This is already mentioned in the paper so it just needs to be highlighted in the revision letter.

Additionally, we can now say that OTI has decided to use nodewatcher as a basis of their Commotion Wireless dashboard:

• https://commotionwireless.net/blog/2015/04/22/announcing-commotion-collaboration-with-wlanslovenija/

[lukacu]

I have added the text for the response to Reviewer 1, but someone has to write the details about the changes (I do not know which sections were moved from where). I have set up a skeleton for the summary of changes, so just fill in which sections were moved where.

[kostko]

I have now updated the response to Reviewer 1.

[mitar]

In the table 1, why for manman we have in TP column a check, and not D or S? For webframeworks, what is a difference between - and custom?

[mitar]

We are claiming that "Both node database solutions are built on custom web frameworks, further limiting the ease of adoption by a new community network.", but it seems that guifi.net is using Drupal? That is pretty standard web framework.

[kostko]

Yeah, this was from before, because their code really doesn't look to use any framework even though it might be a Drupal module :-) I will fix it.

[kostko]

In the table 1, why for manman we have in TP column a check, and not D or S?

Well, there is no source code for manman, so we can't know which is it. And whoever entered manman into the table just placed a "yes" there.

For webframeworks, what is a difference between - and custom?

For example, Ondataservice is not a web application, so this probably doesn't apply. For the others, I will replace them with "custom".

[kostko]

Hm, but it says that Ondataservice uses PHP. Ok perhaps there is some frontend for that distributed OLSR database. I will just change it to "custom" everywhere.

[mitar]

In revision letter, for reviewer #3, comment 2, should we mention that guifi.net does not support modular firmware configuration generation, nor image generation? I think the reviewer's comment somehow misrepresents what we are claiming to be unique? Should we point to the section where we are clearly explaining this?