Change to using Bandana for storage

[GoogleCodeExporter]

I've attached a patch that changes the storage for the templates from using 
the bizzare extra plugin thing to using Bandana.    This fixes several issues:

1) Fixes some issues where uninstalling the template plugin isn't working, 
especially after an upgrade from 2.x to 3.2.

2) Should fix defect #27 

3) Fixes a bunch of race conditions where if you have a couple people 
simultaneously editing templates or exporting things and such, the template 
plugin may not be installed or changes could be lost or such. 

The patch is a starting point for discussion.   At load time, it will map the 
old 
template plugin data into Bandana for compatibility.   I think it doesn't cache 
templates anymore which may be a slight performance issue, but that would 
need to be investigated to see if it really is a problem.

Original issue reported on code.google.com by dan.k...@gmail.com on 27 Apr 2010 at 8:51

Attachments:

• patch.txt

[GoogleCodeExporter]

Well, by not using Confluence's velocity configuration we do lose out on its 
template
cache and configuration, which I was trying to avoid.

I'm not totally against switching to Bandana, but I'd like to understand the 
current
problems better.  What issues are you having when upgrading from 2.x to 3.2? 
Couldn't we fix the race conditions by adding a lock to ensure two templates 
cannot
be changed at the same time?  How could you have a race condition during 
exports? 
Can you reproduce #27 as I'm not able to with a clean install on 3.2.  

To help reproduce these issues, I've bumped up the build to use 3.2, so we can
hopefully replicate the problems in the integration tests.

Original comment by donald.b...@gmail.com on 28 Apr 2010 at 7:19

[GoogleCodeExporter]

I should be able to get the template cache back in place shortly.  I'm working 
on my 
"fork" at github:
http://github.com/dkulp/couldit-autoexport/

The plugin seems to work fine with a clean 3.2 as is which is how the tests run 
and 
such.   However, it DOESN'T seem to work fine on an "upgraded" 3.2.   
Basically, a 
confluence 2.x install that had a bunch of custom templates that has been 
upgraded to 
3.2.    An example of this is the Apache confluence install.    Right now, we 
cannot edit 
any templates as the uninstall of the template plugin fails.   It seems as part 
of the 
upgrade confluence as changed the internal id or key or something of the plugin 
and 
now we cannot uninstall it.

One option was to manually got through all the templates and copy them  (keep 
in mind, 
with 3.2, the plugins are kept in the db someplace, not jars on disk), then 
uninstall 
everything, install fresh version of autoexport, then manually set the 
templates one at a 
time again.   Kind of crappy.   I'd rather use the Bandana storage for what 
it's designed 
to do, store required state.  IMO, osgi jars aren't exactly the best 
persistence engine.  :-
)

There would still be a race on export if a set is occuring.   There is a period 
of time 
where the template plugin is not registered.   If an template is needed during 
that time, it 
would fail to find it.   Some additional locking could get around that, but 
again, I rather 
use the proper storage mechanism and avoid the potential "upgrade" issues again.

In anycase, I should have Template caching back in place shortly.

Original comment by dan.k...@gmail.com on 28 Apr 2010 at 1:19

[GoogleCodeExporter]

And putting it as a plugin has another disadvantage......

We just had a sys-admin trying to diagnose things and hit the "remove" on the 
template 
plugin.    All custom templates are now gone.   :-(

Original comment by dan.k...@gmail.com on 28 Apr 2010 at 4:24

[GoogleCodeExporter]

The problem with the upgrade is probably just because the plugin system changed 
how
it references OSGi bundles, specifically, the plugin key is now name-version 
and not
just name.  I'd like to isolate exactly why bandana is better.  What I see so 
far:
# Admins can't delete the templates separately
# Bigger window of unavailability during edits
On the other hand, a bundle is better because:
# Ability to save/backup templates
# Re-use of the template configuration of Confluence
# Less memory usage as a velocity engine is something like 2M 

The fact that admins can't delete templates is pretty useful if no one is using 
the
save/backup capability.  However, before we'd switch, I'd like to see:
# A thorough analysis of what we'd be missing if we didn't use Confluence's 
velocity
config...only the shared cache or more?
# Func tests showing problems with bundle storage that are fixed by bandana,
particularly around concurrent edits.

Original comment by donald.b...@gmail.com on 28 Apr 2010 at 8:15

[GoogleCodeExporter]

# Ability to save/backup templates 
This no longer applies with confluence 3 as the "jar" is not saved on the 
filesystem at all 
and there isn't a way to get the jar.     It could easily be solved by adding a 
"backup" 
button to the Auto Export admin page that would get all the custom templates 
and return 
a jar if needed.

# Re-use of the template configuration of Confluence
Not sure what you mean by this.  What configuration?   

In anycase, the Apache confluence instance is now running with a new version 
that 
incorporates the changes.   

Original comment by dan.k...@gmail.com on 28 Apr 2010 at 8:47

[GoogleCodeExporter]

Yes, you can get a copy of plugin jars.  All plugin jars are saved on the 
filesystem
and actually loaded from there, even though Confluence primarily stores them in 
the
database for clustering support.

As for the Velocity configuration, we found that when we created new velocity 
engines
for dashboards, we added a significant resource increase that went up with 
load. 
This would be an issue for cwiki, since it historically has been on a
resource-constrained box.

I see in your patch you didn't add any integration tests and apparently didn't 
run
the existing ones against 3.2 as they weren't working until yesterday.  Could 
you
please add a few, since we should probably resolve the fork to maintain one 
codebase?

I should probably add in my four or so years at Atlassian working with and on 
the
plugin system and Confluence specifically, this is the first time I've ever 
heard
bandana called "a proper storage mechanism"... :)

Original comment by donald.b...@gmail.com on 28 Apr 2010 at 11:01

[GoogleCodeExporter]

I've given you commit privs on this project, since you deployed your fork on 
cwiki,
making it difficult to not apply this commit.  Please still run and add tests.

Original comment by donald.b...@gmail.com on 29 Apr 2010 at 12:59