openaustralia / morph

Take the hassle out of web scraping
https://morph.io
GNU Affero General Public License v3.0
462 stars 74 forks source link

Data table in some cases should be displayed wider #635

Open mlandauer opened 9 years ago

mlandauer commented 9 years ago

See for example https://morph.io/keithcollins/uspto2 morph io keithcollins uspto2

The data in the right hand column is being truncated. More if it could be shown because there's plenty of screen space left (at least on desktop)

equivalentideas commented 9 years ago

Good call. This can be done by adding some more smarts to the .panel-scrollable class and the script that adds it.

equivalentideas commented 9 years ago

Ok, this is a bit complex. No matter what's in the table we want to display the column widths at their optimal width so that the citizen can see all the columns. Quite often the scraped content is quite long–left to auto, columns would often stretch wider that 500px. It quickly becomes annoying to scroll to see all the column names, so we truncate them to max width 200px. But then you get the situation raise in this issue where a column is truncated more than it needs to be.

We don't want to have really wide columns with a tiny bit of content. We don't want to have truncated columns where they don't need to be (example above).

I'll try rubber duck this by noting down the different scenarios we want to achieve.

There are 1 - 5 columns. None of the table cell content is very long, each cell fits it's content in full without extra whitespace: screen shot 2015-04-28 at 3 57 57 pm

A 5+ columns and a few of them have lots of content. The columns with lots of content are cut off, with the overflow hidden. The citizens scrolls along to see all the columns. They can inspect the content of the column by tapping or hovering over the cell.: screen shot 2015-04-28 at 3 43 40 pm

(not sure about this one) There are 1 - 4 columns. One of the columns has a lot of content and is quite wide, but the others are suitably thin to match their content. The content in the wide column wraps and is truncated if it goes for more than 3 lines screen shot 2015-04-28 at 3 31 21 pm

equivalentideas commented 9 years ago

A couple of solutions come to mind after going through that.

  1. As a next iteration we could just extend the max-width of wide columns to 400-500px (currently 200px) which would save a lot of scraper tables from being cut off. This wouldn't completely solve the situation at the top of this issue, but would improve the situation and be quick. We just need to pick a width that doesn't become annoying if there are lots of columns.
  2. We could set a much wider max-width of something like 600px and wrap the content in the columns. This would mean you have uneven row heights (I don't think that's a big problem). Then we could truncate the HTML in the cell at long character width with an affordance to hover/tap for the full content (suggested by @henare so you don't get epic long tables). I'm not sure about how well this would work if there were 10+ columns.

Option 1 seems like a small/simple step in the right direction, as an iteration of this system.

More thoughts very welcome—I feel like I could be missing some really simple solution here.

equivalentideas commented 9 years ago

I'm going to give this a go as a next iteration:

As a next iteration we could just extend the max-width of wide columns to 400-500px (currently 200px) which would save a lot of scraper tables from being cut off. This wouldn't completely solve the situation at the top of this issue, but would improve the situation and be quick. We just need to pick a width that doesn't become annoying if there are lots of columns.

equivalentideas commented 9 years ago

I've tried that and I think it's a small improvement generally (feedback very welcome). I actually found 400px was too wide and went with 300px. It doesn't do much to fix the example scraper though.

Before

screen shot 2015-05-01 at 11 19 01 am

screen shot 2015-05-01 at 11 31 57 am

After

screen shot 2015-05-01 at 11 31 12 am

screen shot 2015-05-01 at 11 29 56 am

Looking at the other option above:

We could set a much wider max-width of something like 600px and wrap the content in the columns. This would mean you have uneven row heights (I don't think that's a big problem). Then we could truncate the HTML in the cell at long character width with an affordance to hover/tap for the full content (suggested by @henare so you don't get epic long tables). I'm not sure about how well this would work if there were 10+ columns.

I now don't think this would work for all tables because column widths of 400px making the tables really wide, let alone wider. Then you'd get really tall cells all the time, and I think the current way of showing extra text through the tool tip would be better than that situation.

In the process I've come up with another idea. If setting the max-width of cells to 300px still makes it too small (as in the example at the top), then set it's width of the wide cells to be the maximum available width to make the table 100% width of the container. That way you see as much as you can without having to scroll.

The logic of this would have to be done in javascript. Here's a go at explaining the logic:

If the table is wider than the container,

if there is only 1 column, make it 100% width

else if columns which are wider than 300px are reduced to 300px then, is the table wider than 70% of the container? if so, do that. If not, then:

set the width of the columns to the available width of the container.

That should keep a good experience for tables with lots of columns. For tables with only a few columns it should give us a better width in many more situations.

@mlandauer @henare I still feel like I might be missing some simple solution, but maybe trying to layout tables with totally unpredictable contents is such complex. Does this sound sane or am I heading down the rabbit hole?