Closed i2p-lbt closed 8 years ago
WardCunningham
commented
8 years ago A good strategy is to only load js from a site you own. Federated wiki works great that way fetching only png and json from remote sites.
We render only a few item types on the server for the benefit of search engine robots. This mode does work ok for those who only write basic stuff. No interactive visualizations though.
i2p-lbt
commented
8 years ago Thx 4 reply. And on that occasion a big thank you for your work - meaning not only this current iteration!
Indeed a sober strategy, but you probably have noticed how unusable or at least ugly most the Internet generally becomes after turning JS off. And have you tried to apply it to FedWikis? I had a look at http://fed.wiki.org/ with and without JS. With JS I land on /view/welcome-visitors and see 4 paragraphs. Without I land on /welcome-visitors.html and only see the first and last paragraph with grey boxes in between. So more than half of the content of the page is missing if not using JS. That's what made me ask in the first place. I btw also noticed that I in addition need to allow a cookie even if just browsing the site, otherwise I get SecurityError in the script.
If I understand your answer right, the JS-dependency currently much depends on how one edits a page. On the example of the fed.wiki.org: doing the headings as type html makes them disappear without JS and doing the links + paragraph as type reference also makes them not getting displayed. I will try to understand how these work and if/why they have to be rendered in client-side JS. If I don't get far with that, I could still opt for a version without or with a limited version of plug-ins, I guess. But it would of course be a big benefit if compatibility with the federated wikis in the ClearNet could somehow be established/maintained.
As a side note on that: I got an installation running and had it reachable over an I2P tunnel already. A FedWiki behind a prettylongaddresskey.b32.i2p URL. The internal links seemed to work fine. But that's how far I checked it for now. I will need a second installation to check the federation things later. One problem that seems foreseeable is that ClearNet nodes today will by default be unable to contact I2P nodes (unless they also run an I2P router, which of course would be great), while the other way round can work (depending on the user that might use Tor/ClearNet in addition to I2P thus enabling both). That would mean that at least in some cases only a one-sided transfer will be possible. Unsure about the implications of that.
All in all it sounds like it should be possible to get better readable version for non-JS-display, so I will at least take further steps to investigate ... The list of core plugins is based on core-plugins-page. The page on core-plugins btw does not render two html-type-fields. I looked into the page source and found the content is actually contained, but only as meta-data and not displayed:
<div class="item html" data-static-item="{"type":"html","id":"aa5dcc7a9cc9db93","text":"<h3> Basic Content"}" id="aa5dcc7a9cc9db93">
<p class="error">html</p>
</div>So the page displays "html" in a grey box instead of displaying the <h3> Basic Content, although it's there. Guess that's because of more complex cases of html? But well, no additional plugins, just headings. So I got to this list of core-plugins:
However, the html-plugin is not listed on the page of core-plugins. It is used often for headings on pages which then do not get displayed without JS, though. Is html a core-plugin? How about pagefold, I noticed that is being used on the default pages as well.
Once I'm clear about what is default I would like to add a page somewhere that displays an example for each. To use as a test-page, kind of. I will try to look into the plugins to understand if / to what degree they can be modified to be readable without JS. At least that seems like a viable approach. I'd be thankful for any hints/help of course.
i2p-lbt
commented
8 years ago Note to self: Authentification seems to use
exclusively (?) now. This seems to be js-dependent. And it also seems otherwise questionable if/how this can be used within I2P. Maybe check if it can be avoided to use claim functionality or check old OpenID code (mentioned on claims) and OAuth as well as what is used on I2P already (there should be some of these in use).
P.S. and rather off-topic: the following links on hosting on admin.asia resulted in a "page not found" for me:
On names-of-things a PDF is linked that sounds promising. I only get a Cannot GET /things.pdf though, if I'm trying to access it. Any chance to retrieve that in a different way? I am assuming this might again all be due to de-activated js. Would appreciate access to that without.
paul90
commented
8 years ago @i2p-lbt Yes, currently Mozilla Persona is used, OpenID fell out of favour sometime ago (a long story, mainly involving identity providers that either ceased to exist, or no longer wanted to support it).
There are paul90/modular-security branches in this, wiki-server and wiki-client repositories, together with wiki-security-persona, that are looking making it easier to replace Persona with something different. But, this is currently a work in progress. There is a very simple scheme in wiki-server that does not provide any login, making a claimed site read-only (though unclaimed sites can be edited by all) – content could be edited with a local/private server, and published by copying the page json to the public facing server.
i2p-lbt
commented
8 years ago Thx for those hints. I read more into Persona yesterday and came to wonder how easy it would be to implement an own provider on this. But from my current viewpoint this is "meta-work" which only indirectly helps me on towards the goal of getting a setup to work on I2P "properly".
Read-only and pushing from local sounds like real good to me. I read Ward is using a setup like that but haven't found any detailed instructions on how to set that up. And I'm still trying to find out what kind of connections the server attempts to build for federation purposes. FedWiki seems to rely on DNS entries on a low level, at least for farms. There is no DNS in I2P so this type of things rather scare me. But will see ...
paul90
commented
8 years ago Read-only and pushing from local sounds like real good to me. I read Ward is using a setup like that but haven't found any detailed instructions on how to set that up.
There was a page that mentioned that, think he used rsync. However, the mechanics are not important - the files you would want to move the the json files in the site's pages directory. You will also want to delete the sitemap.json in the site's status directory, which will trigger it being recreated to reflect the updated content.
The federation is built in the client, creating a neighbourhood as you navigate, rather than in the server. While loading each page the client scans it looking for references to other sites - these might in the page's history, where page has been forked from, or using the reference plugin - the sitemap.json for each of these sites is loaded to create the local neighbourhood for the current session. It is this local neighbourhood that forms the area the client will look when resolving links for a page.
The only server to server connection is used for forking a page, but this would be better performed by the client.
Farms use the HTTP Host request field to determine which server within the farm the request should be sent to, just like any other virtual host. How the request is routed to the server is not really important.
WardCunningham
commented
8 years ago Here is where I have written on offline authoring. I made it work. I consider it not worth the trouble unless legal constraints make immediate publication impractical.
We're happy to work with crypto-networking folks and have engaged with four other efforts before you. We benefit when their problems make our work better. I haven't seen too much come of it for them. Perhaps it is because they all disappear one way or another while we remain on the internet.
i2p-lbt
commented
8 years ago Very helpful guys, thanks a lot!
@paul90 Very informative, should I have found that information somewhere else? The federation being client only is quite interesting, and a bless but also a problem. A bless as it could mean the server needs no changes to not reveal the real ip address while being accessed over the I2P tunnel. A problem/pain because I assume a non-js-client, which would mean the federation won't work at all without changes. Also I understood the client cannot be loaded and used from "my site" only (which would allow me to use the js after checking it once per new version). But instead it dynamically loads e.g. required/missing plugins from all involved sites, right? While this sounds wonderful in terms of potential on functionality (and maybe useability), it's an unbearable security risk from my current viewpoint.
If you don't mind brainstorming on this: I thought it would/could/should be like this: As the site-owner I add a link to another site to clone things from there, edit them on my server and am able to push the changes back then. That would allow to have a central site (think Wikipedia) making the "collection" available while having backups in terms of the local forks at all contributors. My site would "watch" those external links I used for changes and inform me about changed content that I can check/pull then, if I'm still interested.
@WardCunningham Do you mind sharing how you tamper the status/open_id.identity? This seems key to using this. Is there an easy way to define a local, non-networked identity for oneself? That's basically what I'm searching for currently. Next question would be how to claim the site in the code. So like: "how to define a new wiki, claim it with a locally defined identity and deny logins". I assume the identity should/needs to be the same on dev and production then?
Then you don't seem to touch the sitemap as mentioned by @paul90 before. Did you just leave out that code on this page? Or is that not necessary after all? Actually being able to work off-line is a strong feature I think. You do not edit files though, but write in the wiki software, or? Why would the browsers local-cache make a better world then? If you think big wikis (Wikipedia) and many contributions, is that really an appropriate scenario then? Thinking memory usage of browser etc.?
My scenario is like having a plugin in the router console of I2P (that's the core piece of software every i2p user has). If you activate personal email there (called I2P Bote - a decentralised, always encrypted and always anonymously transferred email system), you will get your own little server. Although it's just activating a "plugin" in that console for the user. Now I think like you could as well activate (one or more) wiki's there. And ... done, you got your decentralised wiki(s) up'n'runnin (as domains + certs are handled by the underlying systems). If you go to the central I2P wiki site, then, and want to edit a page, you go to your own wiki and say "clone this and offer me change+pull-request". Such an official site is still needed to represent a current state on a large collection of pages like a central Wikipedia or I2P-wiki (I guess/assume? You got other concepts for this scale of distribution and the federation to handle it?).
WardCunningham
commented
8 years ago But instead it dynamically loads e.g. required/missing plugins from all involved sites, right?
No. If you want to add plugins to your server you need to do so with your own server credentials and retrieve source from some reputable open-source community with some mechanism like git or npm.
You can fetch inert json text from my site and post it to your site. But to see my custom plugins rendering in their dynamic beauty you will need to either visit my site with js turned on or install my plugins in your site after carefully inspecting them.
Do you mind sharing how you tamper the status/open_id.identity?
I logged into the server with ssh and wrote some random non-identity text like "no login allowed" into the file with a command like echo no login allowed > open_id.identity.
An official site is still needed to represent a current state on a large collection of pages.
No. You could hack our server to talk to your system using cryptographically generated ids stored in your own private host file pointing to a private interface like localhost. This system would be as private and secure as any other text editor editing files that your network transports. Your hacked server could still participate in our federation wherever it was visible, which would be your network friends but not me or mine. This works because your participation is centered on your browser running on your own computer talking to your private resources and (if you choose) any of mine on the public internet.
I understand that this is unusual. Facebook doesn't work this way. Neither does GitHub or Wikipedia. We've worked hard to make it feel like we are all in one place, one wiki, even though we are not.
i2p-lbt
commented
8 years ago Hm, on your first point: when I visit localhost:3000/view/how-to-wiki on my own wiki and activate js, then that js generates something that attempts to load more js from wiki.org (there was no attempt to load a script from wiki.org before activating the js for the localhost). Why is this? I didn't find the code doing it, but I'm unfamiliar with Coffee and Browserify. From the page view some of the history link to there, but if there is no plugin-loading as you say?
About the identity. I did such a change and the "Claim ..." thingy on the page disappeared. Also I'm able to modify the pages and can see that the changes are written to the server. How do you preserve the identity when transferring the files to the production server where you don't want to have that specific file?
@paul90 wrote:
The federation is built in the client, creating a neighbourhood as you navigate, rather than in the server. While loading each page the client scans it looking for references to other sites - these might in the page's history, where page has been forked from, or using the reference plugin - the sitemap.json for each of these sites is loaded to create the local neighbourhood for the current session. It is this local neighbourhood that forms the area the client will look when resolving links for a page.
So the client gets loaded into my browser from the first wiki I visit and might thus not match in versions and available plugins in comparison to other FedWiki installations. If I allow js from other sites I visit while having any such client loaded, then those can install own (versions of) plugins into my client. If I would start my session from another site the next time, I might be using a different version of the client then, that from that site. Correct so far? And is this session-based only, then?
I'm starting to grasp some fundamental things. Changes and content are all centered on the OpenID thing, it doesn't actually matter that much where a specific wiki is hosted, only what ID claimed the page, right? Hosters just provide resources where identities keep pages, kind of? And clients can use the history and links provided to access other servers. That's actually interesting, too, although not at all what I thought what federated wikis would be. Need to think about and play around with this some more ...
WardCunningham
commented
8 years ago I think you are assuming a great deal of mechanism that is simply not present in our work.
Here is a site that explains for developers how this all works. http://dev.asia.wiki.org/view/welcome-visitors/view/hacking-federated-wiki
If you are concerned that this site requires javascript, you are free to download all of the pages in json and make of it what you will. We've had middle level programmers render this successfully. http://dev.asia.wiki.org/system/export.json
i2p-lbt
commented
8 years ago Good links, thank you. And concerning your first statement, I'm not sure, the functionality seems there, I'm just confused because I didn't expect that client/javascript architecture.
I actually did some surfing with js activated on what I assume to be your sites. The "process" of submitting changes seems rather complicated. And didn't work out when I tried - the page that was supposed to display the changes I could submit was just empty. Am I missing something here? Is this "still in the making" or actually how it is supposed to be?
i2p-lbt
commented
8 years ago I found information at http://ward.fed.wiki.org/view/welcome-visitors/view/diversity-in-service that says:
A usefully rendered .html version of pages should be available to clients without javascript.
Last changed 10 monts ago. Is this still accurate? If so, count me in and tell me what to do ;) As you obviously have been thinking about this, I assume here, that you have an idea on implement this best. I wasn't sure on your point of view on this really. Hints concerning what I need for a development environment (besides editor and git) appreciated. I know programming in general, just not the frameworks used here ...
I envision to add a very simple html-only ("disableable") "quick anonymous contribution" mechanism in addition later, which would provide a basic wiki-functionality without Javascript then. Once we get there, I will try to promote using FedWikis within I2P. If you advise me how/where to do this best, I would be happy to contribute to the documentation on the way.
WardCunningham
commented
8 years ago I check that wiki works well with lynx every now and then. I don't think we see lynx as a growth sector of the internet. We are interested in how wiki could work better with screen readers and pay attention to ARIA standards.
https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA
i2p-lbt
commented
8 years ago Richard Stallman uses Lynx as his default, far as I know - after he retrieved the page using wget and getting it emailed to himself (https://www.stallman.org/stallman-computing.html - see section "I am careful in how I use the Internet."). Now this way of accessing the Internet indeed might not be seen as a "growth sector" as you say, but getting people like him behind a project could actually bring all the growth needed. And I do indeed know more people using similar mechanics ... Also getting those people to instead say they cannot use the system might not be helpful. From my point of view these seem rather to be experts and getting them "in" as contributors might turn out a big step forward.
For the Javascript-version you might want to consider conformity to https://www.gnu.org/software/librejs/ then btw., otherwise that would be the next blocker in case of Richard I guess. Want me to report that as an issue on its own? That addon will selectively block "non-free" scripts, so it's probably a "problem" of it's own (JQuery might be recognised and allowed but other scripts would probably get blocked until modified accordingly).
On the accessibility: It's good to support this. I didn't even know about ARIA and btw. would have thought that providing a version without Javascript would be a sound approach to solve this, too. I remember a CSS optimised for screen-readers should basically do the job when sticking to HTML and paying attention to some details or something. But for the blind people that use Javascript supporting that sounds good, too, of course, as I assume HTML will be view-only at first and might never get all the functionality of the client.
Although the node-framework seems js-based which could basically allow to execute the client-code on the server as well? Still seeking your help in identifying a sound approach in terms of the architecture.
WardCunningham
commented
8 years ago I don't see that we have come up with anything specific to be done here so I will close this issue.
i2p-lbt
commented
8 years ago Well, I disagree.
A usefully rendered .html version of pages should be available to clients without javascript.
There is a list of plug-ins with non-proper output here which I assumed to be a task-list for fixing. Please elaborate what specifics you are missing.
A good strategy is to only load js from a site you own. Federated wiki works great that way fetching only png and json from remote sites.
I fail to reproduce that. I tested the reference-plugin in my local wiki. A reference like:
{
"type": "reference",
"id": "a6421c8d56a21bf9",
"site": "ward.fed.wiki.org",
"slug": "ward-cunningham",
"title": "Ward Cunningham",
"text": "I wrote [[Federated Wiki]] while I served as Nike's open-data fellow. I currently work for [[NewRelic]] in Portland, Oregon."
},will link to /ward-cunningham.html locally albeit the tooltip showing "View => ward.fed.wiki.org". So I will get the empty "Create Page" unless I allow Javascript to be loaded from the domain wiki.org. As said, this is when surfing my local installation and having cookies + local storage + javascript allowed for that.
The conformance to LibreJS also seems "specific" to me. I am actually really confused again. I seeked advise here on how to approach improving the identified deficits. Not answering to how this should be furthered in a coherent way according to your current and planned architecture etc. and then stating this and closing the issue is kind of ... Not sure what you want to say here. "Go away" maybe? Please clearly state if the things things cited here are still accurate and if you want some help in getting there. I can live well with all other scenarios, so no problem anyways.
I am basically able to navigate pages of a fedwiki with JavaScript (js) disabled. Some fields, however, get displayed as a grey box only, naming the type of content though. I found out "html" most often marks headings. Why is js needed to display a heading? Others I stumbled upon until now are "code" and "method". Allowing arbitrary sites to execute js at least can be seen as a security risk (I do think it should be). I hoped the federation would allow a system where browsing is possible without js while editing can be done on ones local server with js enabled then.
How realistic is that idea? I ask this, because I had the idea of making fedwikis run within I2P, the Invisible Internet Project. This would probably require some changes for the federation (due to different addresses) and I'm trying to determine if I should attempt that. Let me shortly explain why that idea appeals to me. I2p makes certification authorities obsolete and combines key-exchange with address-acquisition. That means, once you got the address, you also have the key and thus are done with verification. It also solves the problem of dynamically changing ip addresses and removes the requirement to register a domain. As such it sounds like fedwiki and i2p would be rather perfect fits. It provides (adjustable) anonymity in addition, which people require in varying degrees. Being able to provide it definitely should be seen as plus nowadays. All things that are at least nice to have for a federated application. Actually the discussion today is more if it isn't "must have". But that aside ...
However, having to enable js to browse a site is a requirement that seems not acceptable to me. I assume you go easier on security, but for me it's a no-go on all but a few selected sites. So allowing js for all sites I come along while jumping from wiki to wiki really is no-no. But I understand if you don't care and only ask you for your estimation how easy/hard getting javascript-free-browseability and -cloneability to ones own site might be. Just in case it doesn't make much sense at all with the current implementation. Don't get me wrong, I don't expect you to do what I would like to have. Just trying to find out if it is sensible to approach this with FedWikis current state at all. Concerning the networking stuff I'm pretty sure it's doable although I haven't looked into any glory details yet.
Thx 4 your time!