Run Kinto demo server outside Mozilla infra

[evberrypi]

The link in the docs: https://kinto.dev.mozaws.net/v1/ is no longer working, brings up a blank screen and has been this way since at least 10/10/19.

[Natim]

Hello @evberrypi sorry about that, I cannot fix it since I don't work at mozilla anymore. However I encourage you to deploy your own demo server using this button:

[Natim]

It looks like this: https://kinto-dev.herokuapp.com/v1/

[glasserc]

Somebody else mentioned this to me today and it seems like the server is totally down (can't ssh in or anything). I asked ops but they didn't seem to know anything about it. I'm not sure what to do about it..

[leplatrem]

The domain should be associated to some AWS instance, there must be a way to find out :) It's possible that it was deleted though. @Natim do you remember how you set it up?

[Natim]

It's possible that it was deleted though.

Yes exactly, I remember I configured the mandatory variable to make sure the ripper doesn't remove it but someone probably did manually.

[leplatrem]

Alright, ckolos restored the instance. But it'll be better to update our docs and run it somewhere that we control (Heroku for instance).

I realize it's mentioned in many places https://github.com/search?q=kinto.dev.mozaws.net&type=Code We can ask if we can keep the domain :)

[leplatrem]

@Natim or @dstaley, would you be interested in hosting an instance?

[dstaley]

I'm happy to run an instance! Could you share more info about the specs (CPU, memory, disk space, DB, etc.) for Mozilla's instance? I bet I could get away with something pretty small. Also if you could share the Kinto config for the server that would be helpful as well.

[leplatrem]

Thanks Dylan!

The machine is pretty small, 8GB of hard disk, only 500MB of RAM is used, and CPU is quadcore Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz. And it runs a PostgreSQL locally.

The configuration is:

# kinto.ini
[app:main]
use = egg:kinto

kinto.project_name = Kinto DEV
kinto.cache_backend = kinto.core.cache.postgresql
kinto.cache_url = postgres://dbuser:dbpass@localhost/dbname
kinto.storage_backend = kinto.core.storage.postgresql
kinto.storage_url = postgres://dbuser:dbpass@localhost/dbname
kinto.permission_backend = kinto.core.permission.postgresql
kinto.permission_url = postgres://dbuser:dbpass@localhost/dbname
kinto.http_scheme = https
kinto.http_host = demo.kinto-storage.org
kinto.eos =
kinto.experimental_collection_schema_validation = true
kinto.experimental_permissions_endpoint = true

pyramid.debug_notfound = false
# kinto.backoff = 10
kinto.userid_hmac_secret = <<redacted>>

kinto.includes =
    kinto.plugins.flush
    kinto.plugins.default_bucket
    kinto.plugins.admin
    kinto.plugins.openid
    kinto.plugins.accounts
    kinto.plugins.history
    kinto_attachment
    kinto_portier

multiauth.policies = account auth0 basicauth portier

# Accounts
multiauth.policy.account.use = kinto.plugins.accounts.authentication.AccountsAuthenticationPolicy
kinto.account_create_principals = system.Everyone
kinto.account_write_principals = account:admin

# Kinto portier
multiauth.policy.portier.use = kinto_portier.authentication.PortierOAuthAuthenticationPolicy
kinto.portier.broker_url = https://broker.portier.io
kinto.portier.cache_ttl_seconds = 300
kinto.portier.session_ttl_seconds = 86400  # Flushed everyday anyway
kinto.portier.webapp.authorized_domains = *.github.io localhost localhost:* *.mozaws.net

# Kinto OpenID
multiauth.policy.auth0.use = kinto.plugins.openid.OpenIDConnectPolicy
multiauth.policy.auth0.issuer = https://auth.mozilla.auth0.com
multiauth.policy.auth0.client_id = XNmXEZhGfNaYltbCKustGunTbH0r8Gkp
multiauth.policy.auth0.client_secret = <<redacted>> 
multiauth.policy.auth0.userid_field = email

# Kinto attachment
kinto.attachment.base_url = 
kinto.attachment.extra.base_url = https://demo.kinto-storage.org/attachments/
kinto.attachment.base_path = /home/ubuntu/demo/v1/attachments/
kinto.attachment.extensions = images+audio+video+archives bin json gz geojson

[uwsgi]
wsgi-file = app.wsgi
enable-threads = true
socket = /home/ubuntu/demo/v1/kinto-latest.sock
chmod-socket = 666
cheaper-algo = busyness
cheaper = 5
cheaper-initial = 9
workers = 14
cheaper-step = 1
cheaper-overload = 30
cheaper-busyness-verbose = true
master = true
module = kinto
harakiri = 120
uid = ubuntu
gid = ubuntu
virtualenv = /home/ubuntu/demo/v1/venv
lazy = true
lazy-apps = true
single-interpreter = true
buffer-size = 65535
post-buffering = 65535
#plugins=python36

[server:main]
use = egg:waitress#main
host = 0.0.0.0
port = 8888

# Begin logging configuration

[loggers]
keys = root, kinto

[handlers]
keys = console

[formatters]
keys = color

[logger_root]
level = INFO
handlers = console

[logger_kinto]
level = DEBUG
handlers = console
qualname = kinto
propagate = 0

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = color

[formatter_color]

~~I can give you the secrets via a secure channel. ~~ Maybe it would make sense to use an Auth0 account outside Mozilla as well. And the hmac for user ids doesn't really matter since we flush the DB every day

But I'm not sure we should keep Portier, since we don't really maintain kinto-portier (it works but we don't use it at Mozilla, and it would need a maintainer in the community).

We also have a script in a cronjob that flushes everything everyday:

0 7 * * * /bin/bash /home/ubuntu/demo/reset.sh

# reset.sh

/usr/local/bin/http POST 127.0.0.1/v1/__flush__
/bin/rm -fr /home/ubuntu/demo/v1/attachments/*
/bin/echo '{"data": {"password": "s3cr3t"}}' | /usr/local/bin/http PUT 127.0.0.1/v1/accounts/admin
/bin/echo '{"permissions": {"read": ["system.Everyone"], "group:create": ["system.Authenticated"], "collection:create": ["system.Authenticated"]}}' | /usr/local/bin/http PUT 127.0.0.1/v1/buckets/main -a admin:s3cr3t