Implement User Accounts

[avacariu]

Just creating this for discussion for now, but I think it's basically done.

The notes thing isn't implemented fully implemented (db stuff is, but frontend isn't), but I'm not sure if I want to implement the frontend in this branch or in a new branch.

This fixes #173, and fixes #175.

[anoopsarkar]

Let me know when this should be merged with master.

[avacariu]

I've switched to using a data-only container which contains the database and index. These are persisted on the host (somewhere in /var/lib/docker), and are exposed to the other two containers. If you remove this container, then the data is gone, too (although it'll probably still be somewhere in /var/lib/docker).

I've also changed how configuration works a bit, so that the only things you'll need to change are in config.env and keys.env. The latter will contain the Google OAuth2 API keys and is in .gitignore.

[anoopsarkar]

It isn't clear what syntax keys.env should have. I can reconstruct it from web/config.py but there should be something to copy and modify.

[avacariu]

I added a key.env.sample file that explains what the contents should be.

[anoopsarkar]

Hmm. Problem. I set config.env to my local index as so:

LENSING_DOMAIN=wikipediahistory
INDEX_PATH=/Users/anoop/lensing.demo/data/wikipedia/latest/

Here is what is in that directory:

$ pwd
/Users/anoop/lensing.demo/data/wikipedia/latest
$ ls
fullData.index fullData.json

On docker-compose up after building I get this error:

bash-3.2$ docker-compose up
Recreating 20150902_data_1...
Recreating 20150902_web_1...
Recreating 20150902_query_1...
Attaching to 20150902_data_1, 20150902_web_1, 20150902_query_1
web_1   | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.65. Set the 'ServerName' directive globally to suppress this message
query_1 | DEBUG:root:serving
query_1 | DEBUG:root:considering settings
query_1 | DEBUG:root:no settings file
query_1 | DEBUG:root:considering index
query_1 | DEBUG:root:changing to new index file
query_1 | DEBUG:root:opening index from /data/index/fullData.index
query_1 | DEBUG:root:using new index
query_1 | Exception in thread Thread-1:
query_1 | Traceback (most recent call last):
query_1 |   File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
query_1 |     self.run()
query_1 |   File "/usr/lib/python2.7/threading.py", line 763, in run
query_1 |     self.__target(*self.__args, **self.__kwargs)
query_1 |   File "/opt/lensing/build/backend", line 145, in state_watcher
query_1 |     index_path, new_index = consider_index(index_path, index, server_settings)
query_1 |   File "/opt/lensing/build/backend", line 91, in consider_index
query_1 |     new_index = whoosh.index.open_dir(index_path)
query_1 |   File "/usr/local/lib/python2.7/dist-packages/whoosh/index.py", line 123, in open_dir
query_1 |     return FileIndex(storage, schema=schema, indexname=indexname)
query_1 |   File "/usr/local/lib/python2.7/dist-packages/whoosh/index.py", line 421, in __init__
query_1 |     TOC.read(self.storage, self.indexname, schema=self._schema)
query_1 |   File "/usr/local/lib/python2.7/dist-packages/whoosh/index.py", line 616, in read
query_1 |     gen = cls._latest_generation(storage, indexname)
query_1 |   File "/usr/local/lib/python2.7/dist-packages/whoosh/index.py", line 593, in _latest_generation
query_1 |     for filename in storage:
query_1 |   File "/usr/local/lib/python2.7/dist-packages/whoosh/filedb/filestore.py", line 81, in __iter__
query_1 |     return iter(self.list())
query_1 |   File "/usr/local/lib/python2.7/dist-packages/whoosh/filedb/filestore.py", line 518, in list
query_1 |     files = os.listdir(self.folder)
query_1 | OSError: [Errno 2] No such file or directory: '/data/index/fullData.index'

[avacariu]

I see the issue. The Dockerfile documentation specifies that you can use environment variables (declared by ENV) in your Dockerfile, and I figured that the environment variables in docker-compose.yml would also be available at build-time, but I guess not.

Unfortunately, it also looks like you also can't do source config.env since those variables aren't going to be available during build, either.

I'll keep looking for a good solution.

[avacariu]

I think I've figured something out. Add the stuff in config.env just like before, and then run ./configure.sh. This'll copy the index to build/index, and then when you run docker-compose build data, it'll add the index from build/index.

I tried to do it without copying (by using a template Dockerfile and filling in the INDEX_PATH variable), but that doesn't work because ADD requires the source directory/file to be in the 'context' (specified by build in docker-compose.yml), because it sends that context, recursively, to the docker daemon. So you can't just add files from any directory, and you can't make your build path be /, either, because it'll send everything to the daemon, and there's also no Dockerfile at /.

[anoopsarkar]

With the latest push to web/user-accounts the site builds and works. However, the Google signin still does not work for me.

The following warning message appears. Unsure if it is problematic or not:

web_1   | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, 
using 172.17.0.77. Set the 'ServerName' directive globally to suppress this message

[anoopsarkar]

Also with the new banner on top, the Timeline view is cut off at the bottom and it is hard to see or select the time intervals using the lower part of the Timeline.

[avacariu]

Unsure if it is problematic or not

Not an issue.

Timeline view is cut off at the bottom

I increased the size of the bottom margin. It works for me now.

[avacariu]

What issues are you having with the Google signing?