Not all aliases reaching hubot-stackstorm

[mbrannigan]

We have approximately 135 aliases:

# st2 action-alias list -j -a all | jq '.[].formats[]' | egrep "^\""  | wc -l
135

Not all of them are being made available to hubot.

When dumping Raw actions from ST2 API during loadCommands in /opt/stackstorm/chatops/node_modules/hubot-stackstorm/scripts/stackstorm.js, it was determined that about 100 aliases were making it.

[mbrannigan]

@blag thanks for helping out with this! If you need any attachments / additional information, please let me know.

[LindsayHill]

about 100 aliases were making it

I have a strong suspicion that this is the same fundamental issue as https://github.com/StackStorm/st2/issues/4034

Changing https://github.com/StackStorm/st2/blob/master/conf/st2.conf.sample#L34 might work.

[mbrannigan]

Updated st2.conf to set max_page_size to 300 in the api section and restarted st2 with st2ctl restart.

https://gist.github.com/mbrannigan/1a0919e8d488402be6048092ccdb4d9a

unpause still isn't listed.

[mbrannigan]

After cleaning up the gist a bit:

cat bar2 | jq -r '.[].formats' | egrep " \"" | grep ".*pause sidekiq"
  "pause sidekiq queues (on|in|for|with|using) environment {{ environment }}"

Validates that unpause sidekiq is still not listed.

cat bar2 | jq -r '.[].formats' | egrep " \"" | wc -l
148

148 aliases now listed during loadCommands

[LindsayHill]

Validated, this is related to: https://github.com/StackStorm/st2/issues/4034

The "148" number is a red herring. There are 100 aliases returned, but those aliases have an average of > 1 format per aliases. So counting formats gives us 148 unique formats, but there are really only 100 aliases.

Proper fix requires proper pagination handling with st2client.js

[mbrannigan]

The following shows the number of aliases returned by the API

curl --silent -X GET -H  'Connection: keep-alive' -H  'Accept-Encoding: gzip, deflate' -H  'Accept: */*' -H  'User-Agent: python-requests/2.14.2' -H  'X-Auth-Token: sdfsdf' http://127.0.0.1:9101/v1/actionalias  | grep -c '\"ref\"'
100

Providing a different limit allows the rest of the aliases to be listed.

curl --silent -X GET -H  'Connection: keep-alive' -H  'Accept-Encoding: gzip, deflate' -H  'Accept: */*' -H  'User-Agent: python-requests/2.14.2' -H  'X-Auth-Token: sdfsdf' http://127.0.0.1:9101/v1/actionalias?limit=200  | grep -c '\"ref\"'
111

The missing aliases are present in the json returned.

It looks like the resource module defaults this list to 100:

https://github.com/StackStorm/st2/blob/master/st2api/st2api/controllers/resource.py#L96

[mbrannigan]

I updated actionalias.py to default the limit to 500 in the meantime. The missing aliases are now showing up.

    def get_all(self, sort=None, offset=0, limit=500, requester_user=None, **raw_filters):
        return super(ActionAliasController, self)._get_all(sort=sort,
                                                           offset=offset,
                                                           limit=500,
                                                           raw_filters=raw_filters,
                                                           requester_user=requester_user)

[mbrannigan]

Any update on adding pagination?

[blag]

Sorry for the errant status update. Actually, this isn't yet fixed. We have a few more things to do before this fix is published in a release:

• [x] ~make a few more changes to st2client.js~
• [x] publish st2client.js version 1.1.2 to NPM and GitHub
• [x] update hubot-stackstorm changelog, which should re-kick Travis tests and ensure #176 passes tests
• [x] ~merge in a few more changes to hubot-stackstorm~
• [x] bump hubot-stackstorm version
• [x] publish hubot-stackstorm version 0.9.4 to NPM and GitHub
• [x] update hubot-stackstorm dependency version in st2chatops
• [x] bump version in st2chatops
• [x] merge in changes into st2tests, which will ensure that we don't break alias pagination in the future
• [x] wait for st2chatops packages to be built and promoted into unstable package repo

As you can see, there's a bit of yak shaving we still need to do, but we are actively working on it.

[mbrannigan]

Thank you so much for the work that was put into this. I really do appreciate it!