search

DiversityCorp / companies-on-blm

We are collecting corporate statements in response to racial injustice, along with Diversity/Inclusion reports and ICE contract information. We have several issues for requested companies, please feel free to add more!
19 stars 28 forks source link

Collect data for the Fortune top 100 companies #9

Open ravenclaw1999 opened 3 years ago

ravenclaw1999 commented 3 years ago

We should start the collection of data for the top 100 companies. For now, we would be collecting the companies' statements in response to the Black Lives Matter movement as well as a picture of their executive team. Also, link their public Diversity/Inclusion report if they have one as well as if they have any contracts with ICE.

LaChapeliere commented 3 years ago

Hey, do you have a data structure set up for this that we can start feeding?

ravenclaw1999 commented 3 years ago

Hi, we do not currently have a database set up. One of the other issues I listed was that we needed to set up a SQL database using Heroku. If you want to work on that, feel free to get that started. But if you prefer to collect the data using an excel sheet, that would also be very helpful! You can export the excel sheet as a CSV and add that file to a pull request. Thanks and let me know if you have any other questions.

LaChapeliere commented 3 years ago

I'm terrible with Ops stuff so I wouldn't know how to set up anything with Heroku, but I can design the database scheme it that's useful? Either a very simple one that can be exported to a CSV, or a more flexible one that you can upgrade later if you want to collect more info than what you've described here.

ravenclaw1999 commented 3 years ago

That sounds good! Either option works đź‘Ť

LaChapeliere commented 3 years ago

Alrighty! If you have stuff you already know you want to include, I'm all ears :ear:

LaChapeliere commented 3 years ago

How about that: https://dbdiagram.io/d/5f7f8afe3a78976d7b76ef2c

ravenclaw1999 commented 3 years ago

That looks great!

LaChapeliere commented 3 years ago

Cool! A couple of questions then.

ravenclaw1999 commented 3 years ago

Hi, these are some great questions! Here are some answers that I hope you find helpful:

LaChapeliere commented 3 years ago 
* Employers have a federal EIN which is granted through the IRS.  I don’t think we need to store this yet - giving them a unique ID in our system should be fine!

:+1:

  • Not sure about the national database - the IRS may be a good source but for now the main resources will probably be the companies’ own websites and social media pages :+1:

  • I think we could use a field for a link to their public diversity report. Not all companies have them since they are not required by law. Generally they release 1 per year (I know Zillow is a good example which has theirs publicly available) It is different from the public diversity policy, right? I can add a field for that.

  • How do we plan for people to contribute to the database? This is a really good question.. a Google form which could be reviewed is one option? Or submit a pull request.

  • Let’s start simple to begin - I think a CSV sounds great. Okay, will loook into writing contribution guidelines for that then :+1:

ravenclaw1999 commented 3 years ago

You can add a new field for the public diversity policy

LaChapeliere commented 3 years ago

@ravenclaw1999 Mind if I try promoting the database creation issue? It'll be easier to test queries and update processes with an actual database setup ;)

ravenclaw1999 commented 3 years ago

Go ahead! That would be great if you set it up :)

LaChapeliere commented 3 years ago

We can use this to get the list of companies in Fortune 500 and part of the financial data. Diversity data will need to be added manually like @Anjali001 has done, of course, but it'll be easier to coordinate once we have the companies in the database because they'll have been assigned a unique id. From there we'll also be able to set up a process to add other companies not in the list.

kelseyhuse30 commented 3 years ago

@LaChapeliere omg, that repo is a goldmine. that is perfect! while we don't have the database set up, we can add individual markdown files that have data about the companies that we want to display

Here's an example of Walmart I just added some random fields for testing but we can make this more structured based on your design.

LaChapeliere commented 3 years ago

I'm working on a data contribution pipeline with the database queries so people can start adding .csv (but we could do a script to transform markdown into csv) that will be uplodable to the database when it exists, avoid having to redo work on data that has already been collected :)

Do you want me to add a logo field to the db design?

ravenclaw1999 commented 3 years ago

@LaChapeliere adding a logo field to the db design sounds great!

Anjali001 commented 3 years ago

@ravenclaw1999 Hi ! I was thinking to add more data. I guess you are now doing it in .md files as shown in examples. If there are no issues , then I can make .md files for Fortune 21 to 30 companies.

ravenclaw1999 commented 3 years ago

@Anjali001 That sounds great! @kelseyhuse30 also added a couple of different companies you can collect data for under issues.

LaChapeliere commented 3 years ago

@Anjali001 @kelseyhuse30 .csv would be much easier to process and feed into the database later, if that works for you too. @Anjali001 If you can use the column names from this diagram https://dbdiagram.io/d/5f7f8afe3a78976d7b76ef2c that'd be great but if not I'll figure a way to reformat it later. Don't worry about filling all the columns, most of them can be empty.

Anjali001 commented 3 years ago

@LaChapeliere I don't have any problem. I'll then add a new CSV file with data for fortune 21-30. I hope that works. Also, for column headings, I can change them but I'm not able to understand the diagram completely. Can you upload a CSV with necessary headings, I'll directly make changes to it.

LaChapeliere commented 3 years ago

@LaChapeliere I don't have any problem. I'll then add a new CSV file with data for fortune 21-30. I hope that works. Also, for column headings, I can change them but I'm not able to understand the diagram completely. Can you upload a CSV with necessary headings, I'll directly make changes to it.

Great coordination :D I just pushed a draft PR ( #41 ) where you can find the explanations and headers for the different tables. Tell me if something is unclear and I'll do my best to improve the guide.

Anjali001 commented 3 years ago

@LaChapeliere Thanks. Your guide was very helpful. I've collected data according to it this time. Let me know if any changes are required.

LaChapeliere commented 3 years ago

I'll look at it and see if I need to change some stuff in the guide, I've already noticed a couple of mistakes and unclear things