Closed jwirfs-brock closed 11 years ago
This needs some sketches/wireframes for the UI to connect data with the assets.
Does this also mean getting rid of the "Add Data" workflow step?
What should happen to data for stories that already have data added to them that's associated with the story (as opposed to the asset)? I don't think it would be terrible to just lose the UI to add/remove that data and ask the few users who have this data how they want to handle it.
@ghing Yes, this would be go along with removing the "Add Data" workflow step, which is addressed in #579.
On Mon, Feb 25, 2013 at 4:43 PM, Geoffrey Hing notifications@github.comwrote:
This needs some sketches/wireframes for the UI to connect data with the assets.
Does this also mean getting rid of the "Add Data" workflow step?
What should happen to data for stories that already have data added to them that's associated with the story (as opposed to the asset)? I don't think it would be terrible to just lose the UI to add/remove that data and ask the few users who have this data how they want to handle it.
— Reply to this email directly or view it on GitHubhttps://github.com/PitonFoundation/atlas/issues/583#issuecomment-14083819 .
@ghing I think there are few enough stories that have data associated with them that we can deal with this on a case by case basis.
There's a distinction here between data visualizations, which you would want to embed within your story, and source data, which someone may want to download or view when they are done reading the story. Ideally, we would want this to work the way that data assets were originally set up in the Django admin. So, you could create a data asset (that was a chart, graph, map, etc.) and then add a source data set to it, so that when someone sees the visualization, they would also see a link that says, "Get the data" or something like that.
On Mon, Feb 25, 2013 at 4:49 PM, Jordan Wirfs-Brock jordanwb@gmail.comwrote:
@ghing Yes, this would be go along with removing the "Add Data" workflow step, which is addressed in #579.
On Mon, Feb 25, 2013 at 4:43 PM, Geoffrey Hing notifications@github.comwrote:
This needs some sketches/wireframes for the UI to connect data with the assets.
Does this also mean getting rid of the "Add Data" workflow step?
What should happen to data for stories that already have data added to them that's associated with the story (as opposed to the asset)? I don't think it would be terrible to just lose the UI to add/remove that data and ask the few users who have this data how they want to handle it.
— Reply to this email directly or view it on GitHubhttps://github.com/PitonFoundation/atlas/issues/583#issuecomment-14083819 .
To clarify a bit further, I suggest that for data-related assets (chart, table, map) we offer the option of linking to (or uploading) source data. This would replace the "Add Data" step.
What we'd be losing is when, say, someone just wants to link to a data set without really referencing it in their story view a table/chart/map. I don't think that's a huge problem, because it encourages people to put data into a user-friendly format before sharing it.
I still think this needs a bit more analysis, though. I can try to connect with some of the groups that have data they want to use in their stories (Hinkley high group, DPS teams we just met with last week) and try to learn more about how they might use data. I have a pretty good idea of how Piton would use this feature.
Updated issue title from "Move elements from "Add Data" step to data asset" to "Associate uploaded/linked data with assets instead of in the separate "Add Data" step"
These are design artifacts and answers I feel like I need in order to move forward with development.
[ ] sketches of UI for adding data associated with asset in builder [ ] sketches of UI for removing data associated with asset in builder [ ] sketches of UI for updating properties of data associated with an asset (e.g. Title, Source, or URL) in builder [ ] sketches and microcopy for highlighting data on story detail page [ ] sketches and microcopy for highlighting data in lists of stories (if desired) [ ] sketches and microcopy for highlighting data in story viewer [ ] Question: When does a user associate data with an asset? When they're creating the asset, or do they create the asset first and then associate data? [ ] Question: What happens when a data set is disassociated with an asset? Does it get deleted? Stored in some secret bucket? Stored in a bucket available in the UI somewhere?
Assigning to @jwirfs-brock to collect artifacts, answer questions. Assign back to me when some of these have been collected.
@ghing Here are some preliminary responses to your questions above. @bpawlak Could you review and make suggestions? (You'll probably be able to improve on these...)
[ ] sketches of UI for adding data associated with asset in builder
Note: I did these mock-ups assuming that a user would add source data to an asset as they are creating it. I still need to talk to the Piton folks to get their feedback about this. If it ends up that we want to go with creating an asset first, then saving it, then adding source data later (the way users currently add a caption after adding an asset), then we could move things around.
Users will be able to add source data to chart, table and map assets.
When I looked at how users create these assets, I saw some places that we could clean them up. Because attribution and labeling can be included in the caption feature, I suggest removing those fields. Also, because table is the only asset with a "Title", I suggest removing that as well.
Thus, chart, table and map will have common entry fields:
Here are some sketches that step you from adding source data to a chart asset: There are two main pieces to adding a chart asset, (1) adding/editing the chart itself, (2) adding/editing source data. I pictured them as two collapsible parts. The adding/editing source data part would be hidden in the initial view. This is what it looks like when a user clicks to expand the adding/editing source data section. The workflow here is almost identical to what is currently in the "Add Data" step of the story builder. When a user clicks "Add Data Set," these are the fields he/she can enter, identical to "Add Data" set of the story builder. (The only thing I don't like here is how we have to have Save/Cancel twice -- once for the data set being added, once for the entire asset. Thoughts on how to fix this?) Once a user adds a data set, it shows up thusly. There are links to remove or edit it. The user can also add another data set.
@ghing and @bpawlak Here's my first stab at...
[ ] sketches and microcopy for highlighting data in story viewer
...it's not very fancy. Just straightforward.
This sketch shows a chart asset with source data sets. I was thinking that if there are multiple data sets for an asset, we could list them with a semi-colon in between.
It might be cool to have some sort of icon instead of just text. Thoughts?
@jwirfs-brock, for the most part this makes sense. The mockup of the add data set form adds inline help text as opposed to displaying it in the help drawer. I'd suggest moving it to the help drawer for a) consistency and b) to reduce clutter. With all the help text, it might require quite a bit of scrolling when using the 2-up layout on narrower displays.
@jwirfs-brock I also think that the presentation of data could benefit from using icons instead of text. I'm thinking an icon both to say "hey this is data" and another to replace the "get the data" text ("get the data" could still be used as the title attribute of the link).
We use this "download" icon for providing access to the data sets on the story detail page:
For a "hey this is data icon", do any of the icons listed at http://fortawesome.github.io/Font-Awesome/icons/ stand out as appropriate to you?
Or perhaps some of these are better: http://thenounproject.com/en-us/search/?q=data
@ghing I agree about the clutter and keeping things clean. My concern is that people aren't seeing/using the help drawer, so they may just miss it and end up being confused by the source data feature.
This is probably something that we can fix on our end by making the help text more relevant and useful. See issue #641.
Is there any way for us to see, via Google Analytics or some other way, whether people are actually clicking on the help drawer?
Whether they are or not, #641 is something I can start working on because it's a mostly non-technical issue. It might make sense to start out with the help text in the help drawer. If we then observe that people are confused by the adding source data feature and aren't seeing the help text, we can move it back.
@ghing I do like the download icon. But it's always kind of bugged me because in some cases, it's a link to an external site and not a download. I'm wondering if people will not click on it because they don't want to start a file download.
Perhaps we could, if it's feasible, use the download icon for downloadable data sets and the external link (http://fortawesome.github.io/Font-Awesome/icon/external-link/) for links.
To show that there is data, how about the table: http://fortawesome.github.io/Font-Awesome/icon/table/ The other option would be the data base cylinder thingy. I didn't see one on Font Awesome but here's one from Noun Project: http://thenounproject.com/noun/database/#icon-No6001
@ghing Another question about the help drawer. Do we have the ability to change the help drawer text when someone is adding a particular asset type? Right now, it seems that when you add an asset, you still just see the generic help drawer text for creating a new section.
It's been a while, so my brain is a little fuzzy. I thought we created specific help text that is tailored to asset types. Maybe I'm not seeing it because I'm in the sandbox template, and it's tied to the template type?
I always thought of adding a data source to a chart or table to be very similar to a caption for an image. It's not required, of course, but part of the process of adding data/charts/etc. would be to enable users to set a download link for data sets above/below where a user fills out the attribute information in the current UI (screenshot below).
Also, I'd ditch the collapsible containers and go with a text link instead of an icon (a clearly worded text link is almost always better than an icon). Why not just make it so that if there is a link provided, we make the title and attribution a link (which, when clicked, downloads the data)
A question: do the data links actually download the data or simply take the reader to the source page/site where the data is stored?
@bpawlak
A question: do the data links actually download the data or simply take the reader to the source page/site where the >data is stored?
They do both. Users can either enter a link to an external data set, or upload a .csv or Excel file. That's why I suggested either a generic link phrase ("Get the data") or the different icons that reflect whether it's a download or a link.
@bpawlak A little clarification on how data sets are implemented which might clarify a few things. Data sets can be:
a. A file uploaded by the user and stored in Floodlight b. A URL to an external file c. A URL to some kind of web-hosted resource (e.g. a Google Spreadsheet or Fusion table, or an old school web page with an HTML table in it )
So, to answer your question "do the data links actually download the data ...", the answer is, in the case of a and b, yes, in the case of c, no, it just takes the user to that resource in their browser.
The way that data sources can be either an uploaded file or a URL certainly adds complexity to the "get the data" link behavior as well as the UI for adding a data set.
The other thing that adds complexity is that an asset can have multiple connected data sets. I find this pretty common with data things that I make (for instance, a coropleth map uses both the shapefiles from the census and the demographic data; a data table may include data from a survey conducted by a nonprofit merged with demographic data from the census), but it's really hard to tell what the typical user will want/need. This is a place where getting feedback from Matt/Jennifer is helpful.
@bpawlak
I always thought of adding a data source to a chart or table to be very similar to a caption for an image.
This is one of the questions we still have to answer. I sent this to Matt, Jennifer and Kait at Piton, who will be main data-story authors, to see if they have a preference. I'm still waiting on their feedback.
Personally, I view it more as meta-data about the asset, so to me it made more sense having it as part of asset creation. But I can see how it's logical either way, and I'm pretty ambivalent.
@ghing I also asked Matt/Jennifer/Kait how frequently they think they'd be linking to multiple data sets. I'll post that feedback here when I get it.
The other question I'd ask them, is what's the point of storing information about data sets. Is it just "show your work", or is it to one day use Floodlight as a way of helping people discover and see the impact of civic data sets? Or something else entirely? If it's data set discovery, I think it's important to continue to store information about datasets linked to assets, but not stored as part of the asset model.
@jwirfs-brock If it's an excel/csv file, then yes, I'd suggest including a file type icon that's (for better or worse) is going to be recognizable to most of the users. I'd also include the file size info in case it's a really large file. If it's a link to another site, then we should use an icon that indicates a link to an external resource. Something like: http://thenounproject.com/noun/external-link/#icon-No3878 or http://thenounproject.com/noun/link/#icon-No1690
Or, on Font-Awesome: http://fortawesome.github.io/Font-Awesome/icon/external-link-sign/ http://fortawesome.github.io/Font-Awesome/icon/external-link/ http://fortawesome.github.io/Font-Awesome/icon/link/
@bpawlak Kind of like what I said a few comments up? :)
Just teasing. I agree!
@jwirfs-brock yeah, my brain's not working at 100% just yet today. And gitHub doesn't allow you to respond to a particular message (does it?) so everything is in chronological order according to when you click the "Comment" button.
@ghing
The other question I'd ask them, is what's the point of storing information about data sets. Is it just "show your work", or is it to one day use Floodlight as a way of helping people discover and see the impact of civic data sets? Or something else entirely? If it's data set discovery, I think it's important to continue to store information about datasets linked to assets, but not stored as part of the asset model.
Cool, I will pass that along. That's an important distinction.
@bpawlak Yes! I totally wish GitHub had threaded commenting so that we could reply to specific comments. Someone tell the powers that be...
Note to self: There's the mimetypes python package that we can use to select the appropriate icon for the download link for a file/URL.
Just got some feedback from the Piton crew:
Question: When does a user associate data with an asset? When they're creating the asset, or do they create the asset first and then associate data?
When you are creating it.
Question: What happens when a data set is disassociated with an asset? Does it get deleted? Stored in some secret bucket? Stored in a bucket available in the UI somewhere?
They'd like us to built this feature to be extendable, so it makes sense to store it. Matt says: "We should build it to be as robust as possible." One use case cited was if you delete an asset, but are just tweaking your story -- you may want to drag the associated data set back onto a new asset (the way you can drag in a deleted asset).
Question: Where should source data sets appear?
On the story detail page AND in the story viewer (as an icon or text link next to the asset). We didn't discuss other places yet (like on the Explore page), but they'd like to see it in as many places as possible.
Question: How common is it that you would be associating multiple data sets with a single asset?
With maps, very common. With charts, less common.
Question: Type of files you'd like to be able to upload; uploading v. links?
They'll probably both link to the Data Engine and upload .csv or Excel files. If there are other files, like shape files, they would put them on Box.net or some other file hosting service and link to them.
Some other observations:
A common use case for Piton will be fielding data requests. Kait is working on a few (as Floodlight stories) right now that we can look at.
@jwirfs-brock Ok. So it sounds like we can't compress the data set information into the asset fields as suggested by @bpawlak above.
Note to self: Think about having one form for uploading the initial datasets when you create the asset and then showing the chrome to remove/edit/add additional datasets after the created asset is displayed. It seems like a fail to have to edit your asset in order to get to a list of the datasets.
Note to self (mostly). These are the cases, as far as I can tell, for dealing with datasets (as connected to assets) in the builder. Items with a '?' before them are ones I'm unclear about:
My big concerns with @jwirfs-brock proposal is the nesting of dialogs or the dual-purpose of dialogs (adding source data and adding the asset), the density of elements, the overhead of managing complex UI state in order to make this work. I'm going to lay out what I think the problem is, and also what feels cleaner to navigate/implement. These mockups are less proposals than a description because "a picture is worth a thousand words"
We can improve things somewhat by kicking off the addition of data with a button rather than the toggling link.
But then we have double the form elements. Lots of scrolling and potential confusion of which form we're filling out or submitting:
We could completely replace the add asset dialog with the add data set dialog, but is that what the user expects? It also makes the code to manage state more complex.
Then, we have to support adding multiple data sets and showing this somehow:
Do we let users remove datasets at this point? What if they realized they misspelled the name, can they edit it here? Remember, we still haven't submitted the asset!
Piton folks responded to the question:
When does a user associate data with an asset? When they're creating the asset, or do they create the asset first and then associate data?
With the answer:
When you are creating it.
But I wonder if they mean, "We're ready to upload (or link to) the data at the time we upload (or link to) our asset" but not "We want to do this in the same form"
What if instead of trying to merge these two tasks (which I think is made particularly difficult because of the multiple data sets per asset requirement), we make an easily accessible place to add/edit/remove datasets for an asset that's available immediately after uploading the asset.
Then users can list and remove ...
add ...
and edit
@ghing Thanks for these mock-ups. They are really clear. Initially, it made sense to me to have the "add data source" feature in the asset creation step, but moving it to the edit step seems much more logical now.
@bpawlak Do you think you could make some mock-ups that have the same styling as our current Story Builder design that implement Geoff's solution? Then, we can show the mock-ups of the workflow to the Piton folks and see if this fits with what they had in mind.
Some details Geoff and I discussed that I want to call attention to (that may not be immediately evident from the drawings above):
@bpawlak, @jwirfs-brock also, a totally different solution could be possible or preferable. What I proposed was mostly a way to visually clarify the things I found problematic about Jordan's original mockups and to describe one solution that would support all the different cases.
I kind of feel like putting the "Sources" up with the icons for edit and remove is confusing, esp. considering that for images, after you upload an image, you get a prompt to "click to edit caption" below the image.
Is it not possible to put the data source initiate button/action below the chart/map/etc. like we do for captions? It's "outside" of the add asset flow (i.e., we've "added" the asset to the story). Alternatively, we should handle adding captions and other meta-data to go with assets differently than we do now and put all of the controls in the upper right of the asset container for all asset types.
BTW, I'm getting errors on dev trying to add a chart (it's telling me {"error": "You must specify an image, body, or url"}) even though I've tried uploading an image, putting in a URL, etc.
Sorry... clicked "close and comment" thinking "close" was close the commenting container...
@bpawlak That's a good point that it's confusing to edit some asset metadata (captions) below the asset, and other metadata (source data sets) at the top.
@ghing and I had a conversation this morning about changing the caption editing process (because it's a bit awkward as it is now -- it's easy to miss that the caption is there if your image pushes the box below the current view on your screen), which would fix with your second suggestion. I'll open a new issue for that momentarily...
If the edit caption feature were also incorporated into that top area, would the workflow otherwise make sense to you?
Just opened up the issue on caption behavior: #765
@bpawlak The caption is edited below the asset because that's where it shows up in the viewer and we used to use in-place editing throughout the interface. Now that we don't use in-place editing, I think editing the caption feels odd, and I can understand why it feels like it's somehow different information from the other asset fields. I feel like it should just be moved into the asset creation/editing form where everything else lives.
Data sets, at least from a data model perspective, are fundamentally different from captions. While a caption (and attribution in the case of quotations) is stored in the same database table as the asset content, data sets are a completely separate entity, allowing multiple data sets to be associated with a single asset (or data sets being associated with multiple assets or stories). While we don't have to make users think of it in this way, this shapes how I think of data sets a lot, and I wanted to clarify this so everyone can see where I'm coming from.
I don't know if that changes whether the area used for the edit/remove icons is the right place to put a button/icon/link to launch working with data sets for an asset.
As for your errors, you might have to do a hard-refresh in your browser (I turned off automatic js compression and versioning to make it easier to debug code in the development instance). I was able to create a chart asset using both an uploaded image and an embed code. If you still have problems after a hard-refresh, let me know.
Here's my synthesis of the conversations above about how we present data sets in the viewer:
The first source is downloadable (using the font awesome download icon: http://fortawesome.github.io/Font-Awesome/icon/download-alt/) The second source is an external link (using the font awesome external link icon: http://fortawesome.github.io/Font-Awesome/icon/external-link-sign/)
I also put some comments on related issue #733 about how we might indicate in places other than the story viewer that a story has associated data.
@jwirfs-brock Cool. This makes sense, though we might have to re-think the details given the length of the dataset titles. Below is a roughly generated list of dataset titles and attribution in the system. We could truncate titles after a certain length with elipses, or lay them out in a table style so that the titles/attribution wrap in a way that is fixed-width. Alternately, we could use a marquee effect on mouseover. Or maybe it's not really a problem. Just something to think about
Title | Attribution |
---|---|
2009 Births in the Children's Corridor by Mother's Education Level | The Piton Foundation |
Corridor Births 1990 to 2009 by Mothers Education Level | The Piton Foundation |
Health Outcomes in Colorado by Race/Ethnicity | The Piton Foundation |
Children's Corridor Population by Race/Ethnicity, 2010 | The Piton Foundation |
Youth Age Distribution by Corridor Hub, 2010 | Analyzed by The Piton Foundation |
Youth Age Distribution by Corridor Hub, 2010 (Google Spreadsheet) | The Piton Foundation |
Crime rates in Denver | |
Original Aurora SnapShot | |
Transportation Statistics for Soccer Access at the Holly | |
Colorado School District Funding, 2011-2012 and projection for 2012-2013 | |
2012 TCAP results for Colorado, by school | |
2012 academic growth scores for Colorado schools | |
2012 ACT scores for Colorado, by school | |
Metro Denver Census Tracts: Population by Age Group (Under 5) | |
Colorado Tracts: Under 5 Population | |
Under 5 Population by Corridor Hub, 2010 | |
Corridor Births by Mother's Education Level, 1990 to 2009 | |
2009 Births by Mother's Education Level for Corridor Hubs, 2009 | |
Census 2010 Information for Original Aurora: Population and Household Characteristics | |
This is a data set | |
My data set | |
The History of the Holly Poles and Their Future | |
Hefsa in DAVA computer art lab | |
The Confirmation | |
Pole2 | |
Tu Tu Pole | |
Shooting | |
Tutu Pan Down | |
DAVA Computer | |
DAVA computer arts | |
Hefsa in Computer Arts | |
dava computer art | |
2010 Population by Race/Ethnicity for Denver Neighborhoods | |
Births in the Children's Corridor by Annual Household Income, 2007 to 2010 | |
after school programs | |
sex by age | |
Bhutanese Refugees in Colorado | |
Original Aurora Map | |
Original Aurora and East Colfax Map | |
Children's Corridor APS and DPS free and reduced lunch students by geographic hub, 2006 to 2011 | |
ISTE's NETS*S Skills Seal of Alignment | |
Video About Sims-Fayola International Academy | |
OpenWorld Learning | |
Light Rail | |
The Front of Heart and Hand Foundation | |
Side Fence off of 28th and Walton | |
Light Rail passing through | |
Children Walking across the tracks. | |
Tree Stump | |
The Fencing not properly placed in the ground. | |
Fencing | |
Kids playing | |
Light rail | |
Graph | |
Bullying statistics | |
Bryce Merrill Interview - raw notes | |
project christ | |
My test data set | |
Ages 16 to 24 Education, Language, and Employment Tables |
@ghing That's a good point...some of the data set titles can get quite long, especially with attribution. I like the first two options (truncation, table). Would you mind doing a mock-up of what the table might look like?
I agree with either wrapping the text or truncating with an ellipsis (and a tooltip on hover that gives the whole title).
Damn I hate that "Close & Comment" button...
Some things I noticed while testing on dev:
Placement of download icon
Data set title acting as a link
We don't display the data source/attribution anywhere
When I removed a data set...
Story: http://dev.floodlightproject.org/en/build/f08ab17d31af429bad4290c4526eda99/
@ghing Here's a video of some of the strange behavior I observed (still processing on YouTube, but should be viewable in a few minutes...): http://youtu.be/NP7UIwyfmIc
Other things I noticed.
Some other thoughts:
Also, our strategy for placing the "Data Sources" button next to "Edit" and "Remove" may be confusing. If I successfully add a Data Source and then go back to the normal builder view, it's not entirely clear that the "Edit" button is only for the (chart) asset and not for editing the data source. Perhaps we can change the text to be "Edit Chart" and "Edit Data Source" to accompany the icon?
@bpawlak, I agree about the confusion of showing all 3 of the mutually exclusive fields for asset types that allow different kinds of input.
I've opened this as a separate issue, #767, since it's implemented separately from all the data set stuff.
@jwirfs-brock, let me know if you want me to address this along with the data set tasks in this iteration, or if I should deploy the data set stuff and then deal with #767.
Or maybe a generic "Edit Asset" (if it's too tricky to have the text update based on the asset type)?
When you are uploading a map, chart or table asset, you should be able to upload and/or link to the raw data.
It would also be cool if on the Story Detail page there was some indication that a story contains data elements (like there is if you've uploaded a dataset on the "Add Data" step).
Update: Outstanding issues: