When using REDIS_URL and the redis password has a / in it the connection fails

[marrobi]

As discussed with @martinpeck on Slack.

Workaround was to generate new key without a / .

[martinpeck]

@colinschoen feel free to assign this to me if you like...it'll probably end up as a doc bug (with an update in SETUP.md)

[marrobi]

@martinpeck regenerating isn't a usable solution. The Azure deployment templates dynamically retrieve the password and pass it to the Web app.

[colinschoen]

For some odd reason github won’t allow for assigning outside the org, but feel free to take it.

[martinpeck]

redis-py supports the ability to have URLEncoded passwords, but this is not on by default. You have to opt-in to this because you might have a password with % character in it. To opt-in, you have to pass an additional parameter to the from_url method when you create your Redis client. For example,

client = redis.Redis.from_url(redis_url , decode_components=True)

... where decode_components tells redis-py "perform a URLDecode on this url before you process it"

Both Flask-Cache and RQ make use of the redis-py package. Neither use this additional parameter, and so it's not possible to use a URLEncoded password and, therefore, it's not possible to avoid this issue without both of these libraries accepting a change of some sort.

[martinpeck]

@marrobi is there some way to influence the key generation? Can we tell it to avoid certain characters?

[marrobi]

Not that I'm aware of.

[c-w]

Would it be possible to have an ACI that checks and re-generates the password until it doesn't contain a slash?

[marrobi]

We could do that, and discussed with @martinpeck . Just feels like bit of a faff!

[TimLovellSmith]

@martinpeck So you guys are using redis-py? Does redis-py require you to configure the password with an url or could you be using another way?

[c-w]

@TimLovellSmith It's possible to configure the password directly via a config variable, however, if I remember correctly, @martinpeck told me that some of the libraries currently don't allow configuring SSL via any way other than the connection URI.

So either we'd have to make a change to a bunch of libraries to enable URL-encoded connection URIs, or we'd have to make a change to a bunch of libraries to enable configuration of SSL via a non-URI mechanism. All of the above are possible, but it's a non-trivial amount of work/forking/pushing pull requests, etc.

[martinpeck]

So, the issue is that we don't use redis-py directly. resis-py has support for taking a URL and decoding it first, but you need to explicitly do this as it's a breaking change to some users (where they have a password with a % character in it).

The two libraries that are used here are Flask-Cache and RQ. Both of these libraries would need to expose a setting/config to say "I'm using a URL encoded url" so that they could then pass it to redis-py with the right config. To that end I've opened the following two bugs to start that conversation...

• https://github.com/rq/rq/issues/960
• https://github.com/thadeusb/flask-cache/issues/190

FYI The reason we use the URL rather than setting host/password/port/db separately is for a similar reason...these two libraries don't support any option to say "use SSL". By using the URL we get around this issue because the URL expresses this desire for SSL via the rediss:// scheme.

So, right now the options are:

1. Find some way to generate keys for Azure Redis Cache that are "URL-friendly" (unlikely to be an easy fix)
2. Convince both of the dependencies we have to update the way they deal with URLs and encoding
3. Convince both of the dependencies we have to update the way they deal with SSL
4. Something I've not thought of that's awesome and easier than 1, 2 or 3.

I suggest we push for 2, hope for 4, and in the mean time deal with the fact that we might have to re-deploy

[TimLovellSmith]

Update: Azure redis cache has stopped generating slashes in new access keys, so you might not need to fix this any more - if you regenerate all your keys.

[martinpeck]

As @TimLovellSmith mentions, we no longer need to URL encode anything because the Redis Cache keys no longer container the / character that was causing issues. I shall close this issue.

[martinpeck]

Actually, @marrobi needs to close it.

[colinschoen]

I can close. Thanks for reporting back.