Final Project - Githubissues

steven-king commented 6 years ago

Due April 25th at 10:59 a.m.

Your job is to build a visually appealing information portal that is a usable tool for users. It should provide both overviews and trends as well as glandular detail views of multiple data points. The web application must be visually appealing and the style of the site should reflect your audience and the topic. and provide the ability to interact with data, not just view it.

Be creative in how you present the information and what usable tools and insights you can provide. The goal is to provide helpful information that makes people want to come back to your site as a tool.

Audience: 18-40 year-olds living in Durham, Chatham and Orange counties. Mostly affluent with access to multiple devices including mobile.

Requirements: Python Django Site with multiple Visualizations (Dashboard) Overview and detail views of the data (ex. profile page) Primary Data must be collected using web scraping (secondary sources can be via API) Build an API to your data Must use D3.js Must implement a map. It can be d3.js, Google Maps or other API but must use API with info from the database.

Options: NC School Report Card Using publically available information (http://www.ncpublicschools.org/src/), scrape all of the schools within three counties, for example, Durham, Chatham, and Orange because they are close proximity and provide for interesting comparisons of the data.

You should include a map of all the schools in the three counties. Each county and each school should have its own detail page. The homepage and each country page should include multiple visualizations (dashboard)

NC Top 50 Companies using Yahoo Finance Data Using Yahoo Finance, scrape the data for top 50 Companies in North Carolina and build an interactive dashboard that ranks and sorts the companies. Each company should have its own detail page. https://finance.yahoo.com/quote/%5EDJI/components?p=%5EDJI).

Fandango Movie Data Using Fangango.com, scrape all movie theater locations in Durham, Chatham, and Orange counties and store the theater information as well as the show times and details for all listed movies.

You should include a map of each theater and each movie should have its own detail view page that includes a listing or map of the theaters in the area where that movie is playing. There should also be a dashboard that includes visualizations of the available data making it easy to see which movie a user should see.

Student Choice You can pitch me on your unique idea but remember is must use web scraping and have multiple views of the data. Pitches are due no later than 7 days after the assignment is given (next week’s class).

elisabeth-parker commented 6 years ago

Hey all,

I'm doing the school report cards project and am working on my data scraper. The problem I'm running into is that the links from the tiles to the schools' detail pages aren't explicitly stated in href tags. I'm assuming they're generated via Javascript, but I can't find where that's happening. Has anyone else come up with a solution to this, or a different way to get from tile to detail page?

Thanks! Elisabeth

steven-king commented 6 years ago

I haven’t tested it in this case but one trick I have used in this situation in the past is using the browser, download and save the html page locally. Scrape the local html file and that will have the absolute links.

Sent from my iPhone

On Apr 5, 2018, at 11:35 AM, elisabeth-parker notifications@github.com<mailto:notifications@github.com> wrote:

Hey all,

I'm doing the school report cards project and am working on my data scraper. The problem I'm running into is that the links from the tiles to the schools' detail pages aren't explicitly stated in href tags. I'm assuming they're generated via Javascript, but I can't find where that's happening. Has anyone else come up with a solution to this, or a different way to get from tile to detail page?

Thanks! Elisabeth

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/steven-king/mj583/issues/13#issuecomment-378978479, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABo_GX87mvZ4T-DZ0BbN5XPVlKE9DWPUks5tljm9gaJpZM4TDxXa.

cboliek commented 6 years ago

Does anyone know how to merge two or more json files into one? I have a separate json file for each company in the yahoo finance because they weren't part of one chart, so i'm just trying to figure out how to format that correctly in python

jtizon001 commented 6 years ago

I think this is helpful. The content of a json file is a string so convert them to a dictionary. You should be able to merge them from there. https://stackoverflow.com/questions/22698244/how-to-merge-two-json-string-in-python?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa

cboliek commented 6 years ago

hey y'all. I"m trying to get my individual company pages to work on my site, and right now every time they load, the bootstrap.min.css is not loading. I have put the {% extends "companies/base.html" %} and {% load static %} tag in at the top of the html page, but every time the page loads, it gives me this error in the consule: 1:9 GET http://localhost:8000/business/static/companies/css/bootstrap.min.css net::ERR_ABORTED

business is the url so when i type in localhost:8000/business/1 it will come up with the data from the first company. I don't know why the computer is looking through my url to get the static.

If anyone has any suggestions on how to fix this it would be very much appreciated!

steven-king commented 6 years ago

You should be using the static template tag.

{% static %} https://docs.djangoproject.com/en/2.0/howto/static-files/

Sent from my iPhone

On Apr 22, 2018, at 5:54 PM, cboliek notifications@github.com<mailto:notifications@github.com> wrote:

hey y'all. I"m trying to get my individual company pages to work on my site, and right now every time they load, the bootstrap.min.css is not loading. I have put the {% extends "companies/base.html" %} and {% load static %} tag in at the top of the html page, but every time the page loads, it gives me this error in the consule: 1:9 GET http://localhost:8000/business/static/companies/css/bootstrap.min.css net::ERR_ABORTED

business is the url so when i type in localhost:8000/business/1 it will come up with the data from the first company. I don't know why the computer is looking through my url to get the static.

If anyone has any suggestions on how to fix this it would be very much appreciated!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/steven-king/mj583/issues/13#issuecomment-383415381, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABo_Gdn-1-fTfSv-3VYe1Q2fUEJUMPlEks5trPwogaJpZM4TDxXa.

jtizon001 commented 6 years ago

http://192.241.142.13/ twofish.cs.unc.edu:5000 (school vpn required) https://github.com/jtizon001/visa-info-website

elisabeth-parker commented 6 years ago

http://138.197.103.205/movies/ https://github.com/elisabeth-parker/mj583-final

aryaswanie commented 6 years ago

Hi I know we learned this in class, but I could not find out how to put my website online (I can only access it locally), where should I look at?

steven-king commented 6 years ago

@aryaswanie https://github.com/steven-king/mj583/tree/master/class-22-deployment

cboliek commented 6 years ago

http://165.227.77.225/ https://github.com/cboliek/yahoo_finance_working

steven-king commented 6 years ago

I will be in the lab to help with deployment problems after I get my buys in the bed tonight. I will probably be there around 8pm if you need help.

Sent from my iPhone

On Apr 29, 2018, at 12:50 AM, cboliek notifications@github.com<mailto:notifications@github.com> wrote:

http://165.227.77.225/ https://github.com/cboliek/yahoo_finance_working

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/steven-king/mj583/issues/13#issuecomment-385225768, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABo_GdG8AFpQF0knuJk7wtmgjN4cyRb-ks5ttUargaJpZM4TDxXa.

aryaswanie commented 6 years ago

I created my project locally because I didn't know how to work on a docker image. Is there any way I can push my folder to a docker image?

steven-king commented 6 years ago

It does not matter to me how you get it to the server but it must be on the server with a public URL. I recommend using Docker images and containers with Digital Ocean.

Sent from my iPhone

On Apr 29, 2018, at 6:30 AM, aryaswanie notifications@github.com<mailto:notifications@github.com> wrote:

I created my project locally because I didn't know how to work on a docker image. Is there any way I can push my folder to a docker image?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/steven-king/mj583/issues/13#issuecomment-385241458, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABo_GRwmQ6jKpkZVSm_EV87I2TR6tLqaks5ttZZGgaJpZM4TDxXa.

pandringa commented 6 years ago

https://peterandringa.com/facebook-news https://github.com/pandringa/facebook-news

gabriellepalacio commented 6 years ago

Hey @steven-king! There's a few of us in the Reese News Lab working on the project, any chance you'll have time tonight to help us out with some of our issues?