search

openoakland / OakCrime-Decommissioned

Code supporting citizen analysis of crime in Oakland, CA
22 stars 16 forks source link

Update harvestPatrolLog variables to match EMR variables #99

Closed tdooner closed 4 years ago

tdooner commented 4 years ago

They are all specified in all-caps by convention. Additionally, it appeared that BoxHarvestBotEmail, BoxHarvestBotUserID, BoxDevpToken were unused within the codebase, so I have removed them.

Also, it appears that the boxsdk library is able to receive the RSA key as a string, which would streamline the process (in comparison to having to write it out to a file). Let's try this and see if it works.

rbelew commented 4 years ago

not sure where you're not looking? cf. harvestPatrolLog.connectJWTAuth() (line 111) https://github.com/openoakland/OakCrime/blob/master/showCrime/dailyIncid/management/commands/harvestPatrolLog.py

Also, it appears that the boxsdk library is able to receive the RSA key as a string, which would streamline the process (in comparison to having to write it out to a file). Let's try this and see if it works.

you haven't tested this change?

rbelew commented 4 years ago

what are "EMR variables"?

tdooner commented 4 years ago

not sure where you're not looking?

I searched the whole codebase for BoxHarvestBotEmail, BoxHarvestBotUserID, and BoxDevpToken and didn't see them being used anywhere.

you haven't tested this change?

I was hoping you could test it, or we could test it on AWS. I think it will be too much for me to try to get the whole thing set up.

what are "EMR variables"?

Sorry, sloppy language. I mean the environment variables that are defined in the Elastic MapReduce configuration by terraform.

rbelew commented 4 years ago

hey @tdooner , digging into this further... i can see your reasons to pass BOX_RSA_KEY vs needing another file, but looking into the boxsdk doc the boxsdk.JWTAuth() method needs to be passed a byte string, not a unicode string. do you know how to produce this in a way that is impervious to the teraform, env(), etc layers?

tdooner commented 4 years ago

Do you get an error with the unicode string? Perhaps it can be coerced into a byte string via the encode method? https://www.programiz.com/python-programming/methods/string/encode

rbelew commented 4 years ago

These changes incorporated into https://github.com/openoakland/OakCrime/commit/0967a66bf9a48b1a6def44e3e14cb0fcf62b15b2#diff-d9fa9f2a5ea51032e13a7278a1dc9e7c