choldgraf commented 9 years ago

Hey folks - just a reminder that we'll be meeting tomorrow at 2:00pm at BIDS. Let me know if you will not be able to be there in person.

Here's what we'll cover:

Project updates and recap. I've got the presentation I gave at BIDS, but any new results are encouraged.
Come up with goals for each sub-project. Think about results that would be useful for Andrew.
Plan the documentation / wrapping up phase of this project.

Let me know if I'm missing anything, or if you want to make suggestions to the plan.

@berkeley-dsc/purchasing

juanshishido commented 9 years ago

I'll need a call in number, @choldgraf.

choldgraf commented 9 years ago

Call-in number is: https://bluejeans.com/8816518516

choldgraf commented 9 years ago

Hey all - here's a recap of the meeting from last Friday, as well as a "final deliverable" for each one. Let me know if you guys have any questions.

All projects

Comment all code well
Modularize functions
Use ipython notebooks to explain analyses run and how the code works

Network analysis

Structural equivalence measure - how similar are two departments in time
Pick similar suppliers and see what connections they have to departments. Find examples of lots of departments who are using similar, but separate suppliers and see if we can clump them together.
Look into niche suppliers (on the edges of graphs)
Come up with a tool that makes it easy to filter graphs, plot them by month etc, and calculate some simple statistics. Do the filtering on the front end, and then have the network stuff just work on whatever is fed into it. Maybe give some flexibility for deciding "distance" between things, and look into building weighted graphs.
come up with how spread out a given product type is across lots of departments (as a measure of how complex a bid would be)
Deliverable: Set of functions to quickly generate network graphs and some simple statistics, so that Andrew can use this quickly. Also include a notebook with results so far, and explanations for how the code works.

Text Clustering

See if there are "needs" that the clusters represent.
Is a PO "software"? What about "printer supplies"?
Use data that's already categorized to try and create a "test set"
Classify PO lines, not just POs
See how we can make the clusters intuitive
See if POs in a cluster are split between multiple suppliers
Deliverable - Finish clustering to see if we can identify software. Either build a classifier, or just use the text clustering techniques to make predictions about new POs. Include a set of functions for performing LDA, as well as a notebook that explains how it works.

Time to completion

Spend vs. quantity x unit price - spend is the amount paid so far, quantity * unit price is the total committed amount
Need some way to scale by size of department, # of personnel, etc.
Use this to find "Blanket POs" - get blanket PO details from Andrew
Come up with a list of blanket POs and their % complete
Keep brainstorming efficiency metrics, and then cluster departments together based off of this
Deliverable - Use the "time-to-completion" as a means of finding Blanket POs. Include a notebook that explains the analysis run so far, as well as the reasoning behind the particular analysis.

Time classification

Break up the yearly purchasing behavior by product type (or custer, or manufacturer) rather than by department
Identify people at the time that they have a need for something, and use this to determine when "need" on campus is highest for a given thing.
Also try filtering out all suppliers of a particular type first, and then see what the behavior over time is like.
Create a way to quickly create these graphs and clusters after filtering out different kinds of groups.
Think about extending this to multiple campuses.
Deliverable - Write a set of functions to create these time-analyses relatively quickly. Include a notebook explaining how this works and the results so far

choldgraf commented 9 years ago

Hey all - meeting updates and final project goals have been updated (see above). Give them a look and let me know if there are any questions. We can chat about this in the next group meetings, and I'll start thinking about putting together documentation and such for finishing up this project. See you guys soon!

choldgraf commented 9 years ago

Hey everybody - thanks again for making the final push to create some visualization etc. for the final project presentation, and thanks to @dariusmehri for presenting the network analysis slides. I appreciate your guys' hard work on this project!

If you are curious, I've put both our project's final poster and our final presentation on the group's google drive.

As the semester finishes up, I want to remind everybody of final deliverables for Andrew. We need our code to be readable, intuitive, re-usable (to people other than ourselves), and semi-complete. This way he can make the most use of what we've done, and continue these projects into the future.

I know it's finals week, so if you guys can get this done by mid-late next week, that would be fantastic. Let me know if this sounds reasonable. Check out the comments above on this issue for inspiration.

dariusmehri commented 9 years ago

hi chris, will do, i have lots of code i can put up on git, can you please send a hard date so i can put in schedule, thks, darius

On Tue, May 12, 2015 at 11:00 AM, Chris Holdgraf notifications@github.com wrote:

Hey everybody - thanks again for making the final push to create some visualization etc. for the final project presentation, and thanks to @dariusmehri https://github.com/dariusmehri for presenting the network analysis slides. I appreciate your guys' hard work on this project!

If you are curious, I've put both our project's final poster and our final presentation on the group's google drive.

As the semester finishes up, I want to remind everybody of final deliverables for Andrew. We need our code to be readable, intuitive, re-usable (to people other than ourselves), and semi-complete. This way he can make the most use of what we've done, and continue these projects into the future.

I know it's finals week, so if you guys can get this done by mid-late next week, that would be fantastic. Let me know if this sounds reasonable. Check out the comments above on this issue for inspiration.

— Reply to this email directly or view it on GitHub https://github.com/berkeley-dsc/purchasing/issues/15#issuecomment-101366594 .

Darius Mehri Ph.D. Candidate, Sociology University of California, Berkeley

choldgraf commented 9 years ago

Sounds good, let's say Friday, May 22nd

On Tue, May 12, 2015 at 11:30 AM, dariusmehri notifications@github.com wrote:

hi chris, will do, i have lots of code i can put up on git, can you please send a hard date so i can put in schedule, thks, darius

On Tue, May 12, 2015 at 11:00 AM, Chris Holdgraf <notifications@github.com

wrote:

Hey everybody - thanks again for making the final push to create some visualization etc. for the final project presentation, and thanks to @dariusmehri https://github.com/dariusmehri for presenting the network analysis slides. I appreciate your guys' hard work on this project!

If you are curious, I've put both our project's final poster and our final presentation on the group's google drive.

As the semester finishes up, I want to remind everybody of final deliverables for Andrew. We need our code to be readable, intuitive, re-usable (to people other than ourselves), and semi-complete. This way he can make the most use of what we've done, and continue these projects into the future.

I know it's finals week, so if you guys can get this done by mid-late next week, that would be fantastic. Let me know if this sounds reasonable. Check out the comments above on this issue for inspiration.

— Reply to this email directly or view it on GitHub < https://github.com/berkeley-dsc/purchasing/issues/15#issuecomment-101366594

.

Darius Mehri Ph.D. Candidate, Sociology University of California, Berkeley

— Reply to this email directly or view it on GitHub https://github.com/berkeley-dsc/purchasing/issues/15#issuecomment-101376853 .

dariusmehri commented 9 years ago

ok, sounds like a plan, d.

On Tue, May 12, 2015 at 12:37 PM, Chris Holdgraf notifications@github.com wrote:

Sounds good, let's say Friday, May 22nd

On Tue, May 12, 2015 at 11:30 AM, dariusmehri notifications@github.com wrote:

hi chris, will do, i have lots of code i can put up on git, can you please send a hard date so i can put in schedule, thks, darius

On Tue, May 12, 2015 at 11:00 AM, Chris Holdgraf < notifications@github.com

wrote:

Hey everybody - thanks again for making the final push to create some visualization etc. for the final project presentation, and thanks to @dariusmehri https://github.com/dariusmehri for presenting the network analysis slides. I appreciate your guys' hard work on this project!

If you are curious, I've put both our project's final poster and our final presentation on the group's google drive.

As the semester finishes up, I want to remind everybody of final deliverables for Andrew. We need our code to be readable, intuitive, re-usable (to people other than ourselves), and semi-complete. This way he can make the most use of what we've done, and continue these projects into the future.

I know it's finals week, so if you guys can get this done by mid-late next week, that would be fantastic. Let me know if this sounds reasonable. Check out the comments above on this issue for inspiration.

— Reply to this email directly or view it on GitHub <

https://github.com/berkeley-dsc/purchasing/issues/15#issuecomment-101366594

.

Darius Mehri Ph.D. Candidate, Sociology University of California, Berkeley

— Reply to this email directly or view it on GitHub < https://github.com/berkeley-dsc/purchasing/issues/15#issuecomment-101376853

.

— Reply to this email directly or view it on GitHub https://github.com/berkeley-dsc/purchasing/issues/15#issuecomment-101395680 .

Darius Mehri Ph.D. Candidate, Sociology University of California, Berkeley

choldgraf commented 9 years ago

Hi all - checking back in about the status of the final code for this project. I'd like to get something to Andrew asap so that he can start thinking about the next steps of analysis.

@dariusmehri @nlin3330 and @juanshishido, do you guys have anything ready yet?

juanshishido commented 9 years ago

Hi @choldgraf,

Thanks for following up. The code in the scripts folder is the latest version. I'd been creating and updating the script in parallel with the notebook. You can find it here: https://github.com/berkeley-dsc/purchasing/blob/master/script/purchasing_lda.py.

Thanks!

dariusmehri commented 9 years ago

hey chris, will get to it later today, sorry for the delay, darius

On Fri, May 29, 2015 at 6:28 PM, Chris Holdgraf notifications@github.com wrote:

Hi all - checking back in about the status of the final code for this project. I'd like to get something to Andrew asap so that he can start thinking about the next steps of analysis.

@dariusmehri https://github.com/dariusmehri @nlin3330 https://github.com/nlin3330 and @juanshishido https://github.com/juanshishido, do you guys have anything ready yet?

— Reply to this email directly or view it on GitHub https://github.com/berkeley-dsc/purchasing/issues/15#issuecomment-106970853 .

Darius Mehri Visiting Scholar, Sociology Department University of California, Berkeley

dariusmehri commented 9 years ago

hey chris, ok, i just made my first commit to git, data cleaning for network analysis, will get the rest of the code up there tonight, darius

On Mon, Jun 1, 2015 at 1:10 PM, Darius Mehri darius_mehri@berkeley.edu wrote:

hey chris, will get to it later today, sorry for the delay, darius

On Fri, May 29, 2015 at 6:28 PM, Chris Holdgraf notifications@github.com wrote:

Hi all - checking back in about the status of the final code for this project. I'd like to get something to Andrew asap so that he can start thinking about the next steps of analysis.

@dariusmehri https://github.com/dariusmehri @nlin3330 https://github.com/nlin3330 and @juanshishido https://github.com/juanshishido, do you guys have anything ready yet?

— Reply to this email directly or view it on GitHub https://github.com/berkeley-dsc/purchasing/issues/15#issuecomment-106970853 .

Darius Mehri Visiting Scholar, Sociology Department University of California, Berkeley

Darius Mehri Visiting Scholar, Sociology Department University of California, Berkeley

BIDS-collaborative / purchasing

Project finish goals #15

All projects

Network analysis

Text Clustering

Time to completion

Time classification