Data is wiped out when upgrading via npm update -g

[wukkuan]

The default directory for wiki data is in the wiki install directory. The README for wiki suggests using the -g option with npm when installing. However, when updating global packages using npm update -g (I haven't tested any of this with a local install), npm wipes out the entire package directory. That means all data is lost when updating to the newest version.

A few solutions if the user launches wiki without a --data flag:

• Prompt the user for a data directory.
• Make --data a required parameter.
• Warn the user what the consequences are of updating their package.

[almereyda]

As --data can also be provided via a config file, point one and two would need additional refining.

[nrn]

Doh. This was a slip when moving from clone to install. Having a default data directory is helpful, not just so you don't need to think about where to put it before starting, but so you don't lose track of it by default. Anyone have an idea what a reasonable default would be, or do we have to bite the bullet and require users to decide where to save their data?

[WardCunningham]

Can we assume that no path within the install is safe from destruction?

[paul90]

Eeek!

Yes, the safe assumption is that if the wiki is installed from npm, rather than cloned from github, that the default data location is not safe; and will get wiped when the wiki is updated. This holds for both global and local installs - with the global install being more risky as running npm update -g will wipe all the globally installed packages and re-install them, even if they are at the latest version :(

A quick and simple check would be to see if node_modules is in the data path, and, if it is, to issue a warning.

This warning also needs to be in the README file.

A related problem is that the default locations to search for config.json are also places that will get wiped. Probably an idea to save a copy in the data directory, and maybe have a minimal config file saved in package directory, to point to the location of the real config.

[WardCunningham]

So, what advice should we provide for those who might have written something they want to keep in their npm wiki? How do we get this advice to them before they upgrade? I'd like to rev npm with this information today.

[WardCunningham]

Um, now that I think about this, maybe offering an upgrade is the worst thing to do. One wouldn't want the notice that they could loose data as they are loosing data. There is sequencing here that is important to get right.

[wukkuan]

I don't believe there's any way to save the user's data if they blindly run npm update -g. We would probably be best off adding a big bold brightly colored warning message to the top of the npm package page and the Github page.

We could experiment with adding a preupdate script in our package.json, but I doubt that's executed at the right time for us to take advantage of in this instance. I'm willing to try experimenting with it, but I don't think I'll have any bandwidth for it until at least the weekend.

[paul90]

Having a default data directory is helpful, not just so you don't need to think about where to put it before starting, but so you don't lose track of it by default. Anyone have an idea what a reasonable default would be, or do we have to bite the bullet and require users to decide where to save their data?

The only thing I can think of would be to use a sub-directory of the user's home directory, as that is fairly simple to locate.

[almereyda]

mabe depending on the installation type?

-g goes to /var/wiki/

and without to ~/.wiki/ ? Am 12.12.2013 21:59 schrieb "Paul Rodwell" notifications@github.com:

Having a default data directory is helpful, not just so you don't need to think about where to put it before starting, but so you don't lose track of it by default. Anyone have an idea what a reasonable default would be, or do we have to bite the bullet and require users to decide where to save their data?

The only thing I can think of would be to use a sub-directory of the user's home directory, as that is fairly simple to locate.

— Reply to this email directly or view it on GitHubhttps://github.com/WardCunningham/wiki/issues/41#issuecomment-30461153 .

[nrn]

I'm leaning towards ~/.wiki/ as the default global or not, the data is typically user centric. Two users of the same machine starting a wiki would expect different data directories. If we were installed through cloning the repo instead of through npm it would be nice to default to doing the same thing we do now, but that might be more confusing behavior than it is worth.

On Thu, Dec 12, 2013 at 1:01 PM, almereyda notifications@github.com wrote:

mabe depending on the installation type?

-g goes to /var/wiki/

and without to ~/.wiki/ ? Am 12.12.2013 21:59 schrieb "Paul Rodwell" notifications@github.com:

Having a default data directory is helpful, not just so you don't need to think about where to put it before starting, but so you don't lose track of it by default. Anyone have an idea what a reasonable default would be, or do we have to bite the bullet and require users to decide where to save their data?

The only thing I can think of would be to use a sub-directory of the user's home directory, as that is fairly simple to locate.

— Reply to this email directly or view it on GitHub< https://github.com/WardCunningham/wiki/issues/41#issuecomment-30461153> .

— Reply to this email directly or view it on GitHubhttps://github.com/WardCunningham/wiki/issues/41#issuecomment-30461361 .

[almereyda]

Why not?

If I'm cloning it I know why I don't install it via npm. It's git'ified and most likely for development purposes. So I would not want it to interfere with my existing data in ~/.wiki/ by default.

I like that idea.

---

A solution could be the node_modules scan mentionned by @paul. That'd appear as a strange hack that I wouldn't want to have in 'production' code. Scanning whether or not it is a 'dev' environment.

Slightly OT:

I understand wiki is early alpha, but since we have the npm releases and even different servers, are refactoring plugins and clients, it seems to mature a little.

There could even be tests or specs for the Federation Protocol somewhere in the future, so the server diversity could expand.

Somehow I'm still waiting for the GitHub Organization, @WardCunningham, just for the sake of it. But this can just be a silly idea from a too aesthetic point of view.

Therefore I'm proposing a seperate branch **npm* for the npm releases carrying the ~/.wiki/ default whilst keeping master developer friendly and especially safe. They'd only differ by this little patch in the end, when for each release the commits are to be merged.

I don't know, really. This would complicate things in the long run a lot, too. But as the wiki builds on top of npm, it is also the official release channel for it, so intimately intertwined.

---

This is where I confused myself: master should also represent this dependency on the other hand.

Merde. Any hints where my thoughts got stuck?

[WardCunningham]

We shouldn't be too worried about an accidental collision so long as it is not destructive. I run both the ruby and the node versions off of the same flat-files. This works great and isn't even too confusing when I have both in my neighborhood.

If we were to default to /var/wiki in some case we would have to think through how permissions work on all platforms. For that matter, do we know how to find the user's home directory on all platforms? Does ~/.wiki work everywhere?

[wukkuan]

It's possible to have users with no home directory on some platforms.

On Windows I doubt ~ will work, but you can use the USERPROFILE environment variable to find it. Perhaps there's a more standard Node.js way of finding the home directory in a platform independent way.

On Fri, Dec 13, 2013 at 11:26 AM, Ward Cunningham notifications@github.comwrote:

We shouldn't be too worried about an accidental collision so long as it is not destructive. I run both the ruby and the node versions off of the same flat-files. This works great and isn't even too confusing when I have both in my neighborhood.

If we were to default to /var/wiki in some case we would have to think through how permissions work on all platforms. For that matter, do we know how to find the user's home directory on all platforms? Does ~/.wiki work everywhere?

— Reply to this email directly or view it on GitHubhttps://github.com/WardCunningham/wiki/issues/41#issuecomment-30522052 .

[paul90]

Thought I posted this earlier...

Not sure ~ works on all platforms - while it works in the Windows 7 PowerShell, the normal command prompt does not understand it. Also as far as I can see Node fs does not understand ~ either. Something like

getUserHome = ->
  process.env.HOME or process.env.HOMEPATH or process.env.USERPROFILE

will, I think, get the user's home directory on all platforms.

/var/wiki not only has potential permissions issues, but does not exist on all platforms.

[almereyda]

Am 13.12.2013 17:41 schrieb "William Blasko" notifications@github.com:

It's possible to have users with no home directory on some platforms.

I'd be interested in knowing about platforms that don't endorse a home directory, but are capable of running Node.

[wukkuan]

Linux allows it at the very least, but that doesn't mean it's "endorsed". I was just bringing up a potential error condition that should be checked for.

[almereyda]

Yeah, this was me choosing a wrong word as I'm not a native speaker. Could have been "come with" i.e.

[nrn]

I think it is fair to make people without a home directory specify a place to put their data, as long as we can detect the home dir in a cross platform way for users with one. On Dec 15, 2013 7:55 AM, "almereyda" notifications@github.com wrote:

Yeah, this was me choosing a wrong word as I'm not a native speaker. Could have been "come with" i.e.

— Reply to this email directly or view it on GitHubhttps://github.com/WardCunningham/wiki/issues/41#issuecomment-30610031 .

[WardCunningham]

I've published version 0.0.7 of wiki which has @paul90's warnings and home page logic. I set the default to ~/.wiki as @almereyda suggests. There is no special handling for global installs. Thanks also to @wukkuan for the report and @nrn (who took my phone calls) for helping straighten this out.

ANYONE or frinds of ANYONE who could loose their work should be notified as best we can. The simple message is: backup before you update && find your new home in .wiki.

Those who update will find a neighborhood page count next to the search box as a small thank you.