WallStreetAnalytics / wallstreetanalytics

An endeavor to create an analytics tool to democratize the information hedge funds are creating teams to collect.
812 stars 30 forks source link

Security and Architecture #11

Open jbwaclawski opened 3 years ago

jbwaclawski commented 3 years ago

Considering Discord is down and I need to get to bed soon (10:25p here and I'm up at 5:00am tomorrow) I want to gather some thoughts here.

We're building a brand new product here so we have a rare opportunity to do things RIGHT. By this I mean we have the opportunity to bake security directly into every step of the process from the ground up. I don't know about some of you, but in my experiences this is too rare to pass up; too many times I'm brought onto a project to mold security around an already built product only to be told to rip pieces out because of problems with efficiency (that would not have existed if security was a baseline requirement).

I believe that step 1 here is to cobble together an "Architecture and Security" team that consists of one or two members from each discipline involved that ultimately decide how the project ends up. I don't want to remove the democracy piece of this by any means, but I don't think there exists a positive outcome for this project if every architectural and security decision are chosen based on majority preference. If done right it should work in unison with the voice of the many by taking all of the ideas and options presented then choosing what fits together best and what's best for the product.

Once this team is brought together we need to start talking about processes we can put in place to help ensure that the core principals are followed every step of the way, that security is an integral part of the conversation in every decision, and that each line of code submitted is safe and positively contributes to the mission as a whole.

Please feel free to contribute your points of view as well or poke holes in my thoughts. Just want to get the engine churning.

FYI - I'm beeficecream in Discord, may be a different name soon if I can sign into my actual account any time soon.

Bradfordly commented 3 years ago

Proactive security design sounds like I am dreaming - thank you for calling that out! On the architecture note is it safe to assume a cloud native design is desired? In my experience, cloud native presents a unique (and fun) challenge of building an app that is both cloud native and cloud agnostic. What this means is that the distributed services of the app should be capable of running in any public cloud using those public cloud's managed cloud services. This is accomplished in large part by containerization of the services but also is dependent on the design of the actual service. Abstracting up/downstream dependencies through extensible interfaces and keep configuration of all services orthogonal through environment variables are some fundamentals for doing this.

DrewMcArthur commented 3 years ago

I don't want to remove the democracy piece of this by any means, but I don't think there exists a positive outcome for this project if every architectural and security decision are chosen based on majority preference.

i’m picturing something similar to how the r/wsb moderation team functions. yes somewhat undemocratic, but also maybe necessary for some structure it seems.

i’m super into the organizational structure of this kinda thing (tech & politics are my intersection) so my focus will definitely be around that and how we keep this project transparent and people-run. i’ve gotta crash too, but i’m excited to push this forward!!

jbwaclawski commented 3 years ago

We can chat more in Discord once the powers that be are able to wrangle in all of the requests and filter out the noise. For the time being I'm going to start brainstorming some ideas so we can hit the ground running as soon as we can get together. I suggest anyone interested in the security/architecture of this project do the same and we'll see what we can shake out of the idea tree when the time comes.

I like the idea of cloud agnostic - it'll afford us the opportunity to uproot and move around the moment any negative forces try to shut this down for some arbitrary reason.

Thankfully I'll be trapped in a hotel most of Saturday while waiting to tour some homes; that should give me plenty of time to get something resembling organization together.