Next steps for the 0.11.x branch

[martin-bts]

In the migration repo we have a couple of open issues left, but none of them seem blocking. So far, I can run off-the-shelf Askbot 0.11.x and four "advanced" use cases:

• fenced-code-blocks as described here
• a private OAuth2 provider
• skins
• multiple concurrent processes and/or containers, using REDIS as cache

All use cases use the code as provided in the 0.11.x branch. Everything is done through settings.py. How should we proceed?

[martin-bts]

Specifically I am looking for three things:

1. There has been some activity in the other branches while we built the 0.11.x branch. I would like a todo list of changes that we need to make to 0.11.x in order to mimic the progress of the other branches.

2. Askbot in the 0.11.x branch still has 0.10.2 as its version. "Somebody" should do the honors of officially assigning a bigger version to Askbot in the 0.11.x branch

3. When we initially started our Python 3 journey, @evgenyfadeev laid out a rough plan which included moving the installation on askbot.com to our new Python 3 version. I would like a timeline and/or list of prerequisites to make this migration.

[evgenyfadeev]

Thank you, this will be my job for the coming weekend:).

[evgenyfadeev]

@martin-bts askbot-setup needs work. I am looking into this currently.

[martin-bts]

I am too. Tomorroe I'll create a branch to show you. Then we can discuss issues and priorities.

I came up with some code which is supposed to handle user input and validation differently. For now, the basic concept seems to work, but I haven't even tried integrating it with the installer, yet.

[martin-bts]

Here is my current work as a draft PR https://github.com/ASKBOT/askbot-devel/pull/827

The installer is the first contact people will have, when they seek to use Askbot and therefore I feel it just has to work. There is two sides to this problem. First, the installer takes many parameters provided by the user, which makes "just working" a difficult task, to say the least. Secondly, the installer must guarantee a certain basic quality of the deployed Askbot. I am not feeling too well about the installer's capabilities in either regard.

As a first goal, the parameters module introduced in the PR shall enable input validation before the installer does any other work. So far, this is code that can only look at dictionaries and complete them ... in some cases. I haven't spent a single thought on integrating this into the installer.

[martin-bts]

@evgenyfadeev Ok, I spent a couple of hours on this. I do not think we should give askbot-setup such a high priority. In general, the installer does the things it does well. Its problem is the things it does not do at the moment. IMO, improving the installer is not as important as a working Python3 version of Askbot.

What blockers do you see in the installer?

[evgenyfadeev]

@martin-bts Besides the installer - passing tests is a an issue. Do they work for you? I see multiple failures, but they go away if failed tests are ran separately, it must be some issue with the teardowns between the runs (at least I'm running into that).

Please have a look at https://github.com/ASKBOT/askbot-devel/pull/827#issuecomment-515815133 I've tried to summarize the issues that I've found with the installer. I agree that the "must just work" requirement is a bit vague:).

[martin-bts]

@evgenyfadeev Yes, all tests work for me... If I use postgres. With sqlite I get >100 errors

[martin-bts]

We have huge issues with the semantics of "app" and "project". The installer never deploys Askbot as an app. It creates a django project which loads Askbot as an app. Consequently the entire structure of the installer's work is a bit ... arbitrary. Eventually this needs a lot of restructuring and rewording.

At the moment, there is a parameter --app-name which replaces the hardcoded name askbot_app. When copying files, the installer should already show warnings/errors when overwriting files.

I will look deeper into your comment-branch, I think it would improve the installer if we use your new method of gathering collisions/warnings in the default workflow. With this in place I think its sensible to change my checks (which occur later, I think) to raise exceptions as a last safe guard against unwanted overwriting attempts. For a short term solution I think its best to use your approach of rendering manage.py. Ultimately, I am leaning towards calling django-admin startproject as part of the installer (see beginning of this post).

Currently, there are 13 tests for the ConfigManagers I introduced with the askbot.deployment.parameters module. The managers themselves seem to work very well. Additionally I did some manual testing to compare the ConfigManagers with the current code. IMO they perform at least as well as the current code. I will go ahead and replace the current code with the ConfigManagers and then write unit tests for AskbotSetup.

[martin-bts]

I'll remove the --force switch because its functionallity is not what its name suggests.

The only thing it does is replace urls.py with the static file from setup_templates. If this is a use case we want/need, I think this should be a management function.

Overwriting an existing project does not really make sense. I suppose that is why the current installer never did anything else but overwrite urls.py in that particular use case.

[martin-bts]

I just force pushed the first version of the installer that relies on ConfigManagers. Effectively this

• moves logic and sanitychecks pertaining to databases to askbot.deployment.parameters.database
• moves logic and sanitychecks pertaining to caches to askbot.deployment.parameters.caches
• moves logic and sanitychecks pertaining to files and paths to askbot.deployment.parameters.filesystem

The API is supposed to be cleaner now. For each installer parameter that is verified there is an instance of ConfigField. The two main methods of ConfigField are

• acceptable(), which is a boolean verdict on whether it can be okay to deploy Askbot with a particular setting, and
• ask_user(), which interacts with the user to obtain a new value for the ConfigField. ConfigManagers organize ConfigFields to combine them into a useful input-validation. Basically, ConfigManagers will call ask_user() while values are not acceptable(). There are some bells and whistles attached, though. There is documentation in parameters/base.py.

There are 17 test cases that should give some confidence that the new installer does at least perform as well as the old installer.

Do we really want/need an installer that can overwrite existing Django projects? We haven't had one until now. If we don't need one right now, I'd like to move on to things like the failing tests.

[evgenyfadeev]

@martin-bts let's do the ability to overwrite later. It might be nice to have if you want to install askbot on top of the existing install. If you think what you've done is ready to go, I'll test it out and merge.

[martin-bts]

As far as validating input, I think this code is ready for review. I added my recent tox.ini, which now has separate testenvs for pgsql and sqlite based deployment tests.

I wrote a page or two about possible and sensible deployment/overwriting scenarios. I'll create a dedicated issue for that and start working on the open issues we have identified here.

[martin-bts]

I came up with some thoughts on Askbot deployments and a first class hierarchy in https://github.com/ASKBOT/askbot-devel/pull/833.

There are individual classes for the basic tasks we want to do, i.e.

• create directories (Directory)
• copy files (CopiedFile)
• render files using Jinja (RenderedFile)

Additionally, there are classes as models of what should be deployed (DeployableComponent). The overarching idea is that one can now implement logic that creates/modifies DeployableComponents for specific use cases and ultimately one calls deploy() on one or many DeployableComponents.

As with the input validating, I created the PR as a draft for discussions.

[evgenyfadeev]

Thanks, commented here https://github.com/ASKBOT/askbot-devel/pull/833#issuecomment-520250586

[martin-bts]

bump

The most recent proceedings are IMHO

1. New installer
2. Bulk acceptance of old pull requests into Python 2 branches

The new installer is ongoing work. How do we ensure everything that was accepted into Python 2 branches is also ported to the master branch? Is this even an open issue, or already taken care of?

[martin-bts]

The new installer is in place and there has been a lot of merging. Is the master branch now a complete Python3 version of Askbot, or are there remaining changes, i.e. merged pull requests, in the Python2 branches that still need to merged into master?