Build your own hack night check in tool

ondrae commented 9 years ago

Description

We've built the platform for tracking attendance, yet its up to you to build the right tool for your Brigade.

Attendance Platform

Attendance data can be pulled from the Code for America API. The data is updated once an hour.

Checkins can be POSTed to the Brigade site.

The schema expected for checkins is:

fieldname - type - default - optional - example
name - string - None - Yes - `Margaret Hamilton`
email - string - None - Yes - `apollo@nasa.gov`
event - string - None - Yes - `Moon Landing`
date - datetime - datetime.now() - Yes - `1969-07-20`
cfapi_url - string - None - No - `https://www.codeforamerica.org/api/organizations/Code-for-Houston`
question - string - None - Yes - "What did you do today?"
answer - string - None - Yes - "I invented software engineering and sent the first people to the moon."

Scenarios

Talking to the Brigades, there are a few different ways they track attendance.

At the beginning, they ask everyone to go to a short url to check in.
They have a security guard or laptop at the entrance, where they collect attendance.
They pass around an attendance sheet during the meeting.
A volunteer manually counts people
Testing

We made a simple testing route to POST to at https://www.codeforamerica.org/brigade/test-checkin/. It won't save anything, but it will return the correct error codes if you get the cfapi_url wrong. It will return your POSTed data back to you if you get succeed with a 200.

migurski commented 9 years ago

Note that the schema refers to an application/x-www-form-urlencoded POST body, as might be sent by a basic web browser form or curl.

cfapi_url values can be found in http://www.codeforamerica.org/api/organizations.

wbushey commented 9 years ago

I've started hacking together a Sync-Meetup-Attendees-to-CfAPI script over at OpenTwinCities/cfapi_attendance_sync. PRs welcome
Is there a dummy org we can POST to for dev/test?

wbushey commented 9 years ago

Also, how do timezones work? Would you prefer UTC timestamps?

ondrae commented 9 years ago

Hi @wbushey this is great. I'll definitely send in a PR with the part that actually POSTS the checkin.

I don't have a test endpoint for you to post too, let me work on that. https://github.com/codeforamerica/brigade/issues/356

We'll probably add timezone data. I'm not convinced we need it, I think just the date is good enough. Have any good needs for knowing when in the day a person checked in?

I've got a mention of Meet up over at https://github.com/codeforamerica/brigade/issues/347. They are upgrading their API and we just switched to a Pro account, so a more robust way to do all this may appear in the future.

I'm starting to think about to make this easy for all other Brigades too. We've already got most everyones Meetup url in the CfAPI. During our hourly run_update.py we could start pulling attendance data as from Meetup as well.

patcon commented 8 years ago

Think I'm going to make a hubot script that uses a slack API token to submit this for whoever says

hubot checkin [<event>]

Event optional. perhaps something like hubot checkin @someoneelse Weekly Civic Hack Night could be used to check in friends persons besides yourself? Yay chatops!

ondrae commented 8 years ago

Great idea. When you are ready, let me know and I'll give you access to your raw check in data.

patcon commented 8 years ago

hm. seems that a successful POST returns a 302. Is that expected? I lost a bit of time as I was expecting a 200

patcon commented 8 years ago

I would say that the above is particularly difficult because the attendance api endpoint isn't updated for a day, so it's difficult to know whether the POST was successful when it's sending back odd responses and HTML payloads.

patcon commented 8 years ago

Also, this would be helpful info: Will the POST allow someone to add infinite checkins? I have the script working, but am unsure whether to rate-limit internally, in case someone checks themselves in and then a friend checks them in again.

Does it limit on the server side based on email and event name or time (ie. only one checkin per email per hour)?

If not, it would be great to get some simple limits in place on CfA's end :)

patcon commented 8 years ago

Anyhow, got the hubot script working and deployed in Toronto: https://github.com/patcon/hubot-toby/blob/master/scripts/codeforamerica-brigade-checkin.coffee

If others are interested, I can put it into a script package (ie on NPM) sooner rather than later :)

schlos commented 8 years ago

Nice work Patrick! Could you share step by step guide how to deploy it on Heroku?

On Fri, 12 Feb 2016 04:23 Patrick Connolly notifications@github.com wrote:

Anyhow, got the hubot script working and deployed in Toronto:

https://github.com/patcon/hubot-toby/blob/master/scripts/codeforamerica-brigade-checkin.coffee

If others are interested, I can put it into a script package (ie on NPM) sooner rather than later :)

— Reply to this email directly or view it on GitHub https://github.com/codeforamerica/brigade/issues/352#issuecomment-183162922 .

patcon commented 8 years ago

Ok, so I split the script into it's own npm package, so it's more maintainable: https://github.com/CivicTechTO/hubot-cfa-brigade-checkin

Also, I forked an opinionated "hubot for slack" repo, and added some CfA-specific config and instructions, along with that script :) https://github.com/CivicTechTO/hubot-cfa-heroku-slack

Let me know if it's helpful for you!

patcon commented 8 years ago

Hm. Just realized I'm still not entirely positive that the POSTs are working, as the response codes are a little unclear, and the API update is delayed. Could someone with access to the backend data try this out and confirm that it's working? The links above should get you going

schlos commented 8 years ago

@patcon I'm having problems with Hubot for CfA Checkins on Heroku (https://github.com/CivicTechTO/hubot-cfa-heroku-slack), links I tried are:

Tried auto deploy:

-----> Node.js app detected
-----> Creating runtime environment

       NPM_CONFIG_LOGLEVEL=error
       NPM_CONFIG_PRODUCTION=true
       NODE_ENV=production
       NODE_MODULES_CACHE=true
-----> Installing binaries
       engines.node (package.json):  unspecified
       engines.npm (package.json):   unspecified (use default)

       Resolving node version (latest stable) via semver.io...
       Downloading and installing node 5.6.0...
       Using default npm version: 3.6.0
-----> Restoring cache
       Skipping cache restore (new runtime signature)
-----> Building dependencies
       Pruning any extraneous modules
       Installing node modules (package.json)

       > ws@0.4.31 install /tmp/build_f2bde2b1bfecf6ec7cfcc47a7aede863/CivicTechTO-hubot-cfa-heroku-slack-530d412/node_modules/ws
       > (node-gyp rebuild 2> builderror.log) || (exit 0)

       make: Entering directory `/tmp/build_f2bde2b1bfecf6ec7cfcc47a7aede863/CivicTechTO-hubot-cfa-heroku-slack-530d412/node_modules/ws/build'
       CXX(target) Release/obj.target/bufferutil/src/bufferutil.o
       make: Leaving directory `/tmp/build_f2bde2b1bfecf6ec7cfcc47a7aede863/CivicTechTO-hubot-cfa-heroku-slack-530d412/node_modules/ws/build'
       hosted-hubot@1.0.0 /tmp/build_f2bde2b1bfecf6ec7cfcc47a7aede863/CivicTechTO-hubot-cfa-heroku-slack-530d412
       ├─┬ hubot@2.18.0
       │ ├── async@0.9.2
       │ ├─┬ chalk@1.1.1
       │ │ ├── ansi-styles@2.1.0
       │ │ ├── escape-string-regexp@1.0.4
       │ │ ├─┬ has-ansi@2.0.0
       │ │ │ └── ansi-regex@2.0.0
       │ │ ├── strip-ansi@3.0.0
       │ │ └── supports-color@2.0.0
       │ ├── cline@0.8.2
       │ ├── coffee-script@1.6.3
       │ ├─┬ connect-multiparty@1.2.5
       │ │ ├─┬ multiparty@3.3.2
       │ │ │ ├─┬ readable-stream@1.1.13
       │ │ │ │ ├── core-util-is@1.0.2
       │ │ │ │ ├── inherits@2.0.1
       │ │ │ │ ├── isarray@0.0.1
       │ │ │ │ └── string_decoder@0.10.31
       │ │ │ └── stream-counter@0.2.0
       │ │ ├─┬ on-finished@2.1.1
       │ │ │ └── ee-first@1.1.0
       │ │ ├── qs@2.2.5
       │ │ └─┬ type-is@1.5.7
       │ │   ├── media-typer@0.3.0
       │ │   └─┬ mime-types@2.0.14
       │ │     └── mime-db@1.12.0
       │ ├─┬ express@3.21.2
       │ │ ├── basic-auth@1.0.3
       │ │ ├── commander@2.6.0
       │ │ ├─┬ connect@2.30.2
       │ │ │ ├── basic-auth-connect@1.0.0
       │ │ │ ├─┬ body-parser@1.13.3
       │ │ │ │ ├── iconv-lite@0.4.11
       │ │ │ │ ├─┬ on-finished@2.3.0
       │ │ │ │ │ └── ee-first@1.1.1
       │ │ │ │ ├── qs@4.0.0
       │ │ │ │ ├─┬ raw-body@2.1.5
       │ │ │ │ │ ├── bytes@2.2.0
       │ │ │ │ │ └── iconv-lite@0.4.13
       │ │ │ │ └─┬ type-is@1.6.11
       │ │ │ │   └─┬ mime-types@2.1.9
       │ │ │ │     └── mime-db@1.21.0
       │ │ │ ├── bytes@2.1.0
       │ │ │ ├─┬ compression@1.5.2
       │ │ │ │ ├─┬ accepts@1.2.13
       │ │ │ │ │ ├─┬ mime-types@2.1.9
       │ │ │ │ │ │ └── mime-db@1.21.0
       │ │ │ │ │ └── negotiator@0.5.3
       │ │ │ │ └─┬ compressible@2.0.7
       │ │ │ │   └── mime-db@1.21.0
       │ │ │ ├── connect-timeout@1.6.2
       │ │ │ ├── cookie-parser@1.3.5
       │ │ │ ├─┬ csurf@1.8.3
       │ │ │ │ └─┬ csrf@3.0.1
       │ │ │ │   ├── base64-url@1.2.1
       │ │ │ │   ├── rndm@1.2.0
       │ │ │ │   ├── scmp@1.0.0
       │ │ │ │   └─┬ uid-safe@2.1.0
       │ │ │ │     └── random-bytes@1.0.0
       │ │ │ ├─┬ errorhandler@1.4.3
       │ │ │ │ ├─┬ accepts@1.3.1
       │ │ │ │ │ ├─┬ mime-types@2.1.9
       │ │ │ │ │ │ └── mime-db@1.21.0
       │ │ │ │ │ └── negotiator@0.6.0
       │ │ │ │ └── escape-html@1.0.3
       │ │ │ ├─┬ express-session@1.11.3
       │ │ │ │ ├── crc@3.3.0
       │ │ │ │ └── uid-safe@2.0.0
       │ │ │ ├─┬ finalhandler@0.4.0
       │ │ │ │ ├─┬ on-finished@2.3.0
       │ │ │ │ │ └── ee-first@1.1.1
       │ │ │ │ └── unpipe@1.0.0
       │ │ │ ├── http-errors@1.3.1
       │ │ │ ├── method-override@2.3.5
       │ │ │ ├─┬ morgan@1.6.1
       │ │ │ │ └─┬ on-finished@2.3.0
       │ │ │ │   └── ee-first@1.1.1
       │ │ │ ├── on-headers@1.0.1
       │ │ │ ├── pause@0.1.0
       │ │ │ ├── qs@4.0.0
       │ │ │ ├── response-time@2.3.1
       │ │ │ ├── serve-favicon@2.3.0
       │ │ │ ├─┬ serve-index@1.7.3
       │ │ │ │ ├── batch@0.5.3
       │ │ │ │ ├── escape-html@1.0.3
       │ │ │ │ └─┬ mime-types@2.1.9
       │ │ │ │   └── mime-db@1.21.0
       │ │ │ ├─┬ serve-static@1.10.2
       │ │ │ │ ├── escape-html@1.0.3
       │ │ │ │ └─┬ send@0.13.1
       │ │ │ │   ├── depd@1.1.0
       │ │ │ │   ├── destroy@1.0.4
       │ │ │ │   └─┬ on-finished@2.3.0
       │ │ │ │     └── ee-first@1.1.1
       │ │ │ ├─┬ type-is@1.6.11
       │ │ │ │ └─┬ mime-types@2.1.9
       │ │ │ │   └── mime-db@1.21.0
       │ │ │ └── vhost@3.0.2
       │ │ ├── content-disposition@0.5.0
       │ │ ├── content-type@1.0.1
       │ │ ├── cookie@0.1.3
       │ │ ├── cookie-signature@1.0.6
       │ │ ├─┬ debug@2.2.0
       │ │ │ └── ms@0.7.1
       │ │ ├── depd@1.0.1
       │ │ ├── escape-html@1.0.2
       │ │ ├── etag@1.7.0
       │ │ ├── fresh@0.3.0
       │ │ ├── merge-descriptors@1.0.0
       │ │ ├── methods@1.1.2
       │ │ ├─┬ mkdirp@0.5.1
       │ │ │ └── minimist@0.0.8
       │ │ ├── parseurl@1.3.1
       │ │ ├─┬ proxy-addr@1.0.10
       │ │ │ ├── forwarded@0.1.0
       │ │ │ └── ipaddr.js@1.0.5
       │ │ ├── range-parser@1.0.3
       │ │ ├─┬ send@0.13.0
       │ │ │ ├── destroy@1.0.3
       │ │ │ ├── mime@1.3.4
       │ │ │ ├─┬ on-finished@2.3.0
       │ │ │ │ └── ee-first@1.1.1
       │ │ │ └── statuses@1.2.1
       │ │ ├── utils-merge@1.0.0
       │ │ └── vary@1.0.1
       │ ├── log@1.4.0
       │ ├── optparse@1.0.4
       │ └── scoped-http-client@0.11.0
       ├── hubot-cfa-brigade-checkin@1.0.1
       ├── hubot-help@0.1.3
       └─┬ hubot-slack@3.4.2
       └─┬ slack-client@1.4.1
       ├── coffee-script@1.9.3
       └─┬ ws@0.4.31
       ├── commander@0.6.1
       ├── nan@0.3.2
       ├── options@0.0.6
       └── tinycolor@0.0.1

-----> Caching build
       Clearing previous node cache
       Saving 2 cacheDirectories (default):
       - node_modules
       - bower_components (nothing to cache)
-----> Build succeeded!
       ├── hubot@2.18.0
       ├── hubot-cfa-brigade-checkin@1.0.1
       ├── hubot-help@0.1.3
       └── hubot-slack@3.4.2

-----> Discovering process types
       Procfile declares types -> web
-----> Compressing...
       Done: 13.5M
-----> Launching...
       Released v5
       https://oz-checkin.herokuapp.com/ deployed to Heroku

and also manual deploy:

-----> Using set buildpack heroku/nodejs
-----> Node.js app detected
-----> Creating runtime environment

       NPM_CONFIG_LOGLEVEL=error
       NPM_CONFIG_PRODUCTION=true
       NODE_ENV=production
       NODE_MODULES_CACHE=true
-----> Installing binaries
       engines.node (package.json):  unspecified
       engines.npm (package.json):   unspecified (use default)

       Resolving node version (latest stable) via semver.io...
       Downloading and installing node 5.6.0...
       Using default npm version: 3.6.0
-----> Restoring cache
       Loading 2 from cacheDirectories (default):
       - node_modules
       - bower_components (not cached - skipping)
-----> Building dependencies
       Pruning any extraneous modules
       Installing node modules (package.json)
-----> Caching build
       Clearing previous node cache
       Saving 2 cacheDirectories (default):
       - node_modules
       - bower_components (nothing to cache)
-----> Build succeeded!
       ├── hubot@2.18.0
       ├── hubot-cfa-brigade-checkin@1.0.1
       ├── hubot-help@0.1.3
       └── hubot-slack@3.4.2

-----> Discovering process types
       Procfile declares types -> web
-----> Compressing...
       Done: 13.5M
-----> Launching...
       Released v7
       https://oz-checkin.herokuapp.com/ deployed to Heroku

I'm keep getting error page when accessing help page /help.

patcon commented 8 years ago

@schlos what username did you choose for your hubot when creating the bot account on slack?

bot creation form

I nearly certain hubot takes it from there, as I intentionally didn't hardcode anything, so that hubot config doesn't disagree with what slack thinks the name is (ie. for username autocomplete)

schlos commented 8 years ago

first time I've used "hubot" second time "toby" (in Slack hubot integration configuration)

patcon commented 8 years ago

Did you see the screenshot in my last message? Which name did you use in the form that I screencapped? :)

EDIT: ie. which username is it using right now? If it's not working, you could give me contributor access and I could debug real quick?

schlos commented 8 years ago

yep, it's set to "toby" currently, see screenshot: https://qsnapnet.com/snaps/oo2t29vkhcfecdi

I'll grant you contributor access, check again in 3 minutes.

patcon commented 8 years ago

ok should be good to go: https://oz-checkin.herokuapp.com/toby/help

sorry, fixing issues soon. wasn't your fault

schlos commented 8 years ago

thanks!

patcon commented 8 years ago

@schlos I also removed all the files in "scripts/" in the template repo, as they're kinda just junky. Feel free to remove them in yours so that the hubot help output is a little more clear

patcon commented 8 years ago

@schlos important update. Just discovered that the free heroku apps have some constraints that need to be mitigated. This commit makes the changes for anyone spinning up a new bot, but you'll have to read it and make the appropriate changes to your app (special note of the heroku environment variables that must be set)

https://github.com/CivicTechTO/hubot-cfa-heroku-slack/commit/70e7a258f1ddb197895c7a425d2c08fde3bf3ad4

ondrae commented 8 years ago

@patcon I emailed you access to your attendance data.

This is great! A Brigade hubot would be awesome. I had started making different slash commands for the Brigades to use, like /hackyourcity. The problem with that is the free Slack accounts only let you have ten integrations. So bundling up all the useful stuff for Brigades into one app makes for sense. You should add /hackyourcity next.

patcon commented 8 years ago

Thanks @ondrae! With the backend access, I was able to fix some bugs that were rendering the script useless :)

fyi @schlos, I'll always update the hubot template repo when I make any changes, even if just to bump minimum versions on a separate package repo (like so), so follow that if you wish to stay on top of bugfixes

codeforamerica / brigade