search

TechEmpower / FrameworkBenchmarks

Source for the TechEmpower Framework Benchmarks project
https://www.techempower.com/benchmarks/
Other
7.62k stars 1.94k forks source link

TFB website Homepage #5378

Open jcheron opened 4 years ago

jcheron commented 4 years ago

The first time I arrived on the tfb website, after a google search of Framework+benchmark, I stayed only a few seconds and left. Those few seconds, where I saw a lot of barbaric names of tools I didn't know, where I misunderstood the notion of Round, and where I didn't understand what Fortunes was, were enough to convince me to go elsewhere. Fortunately for me, I came back.

I now come to the facts. The Techempower team has made a great contribution to Frameworks benchmarking and works on it on a continual basis. It's a considerable amount of quality work.

But the homepage of the website does not value this work, knowing that it does not offer enough readability for those who are looking for information on this subject.

Suggestion of modification

For a better access to information, I would restructure the home page to bring up:

Infos

A clear and succinct information about the project (this one is only visible if you activate the tab Introduction). In short, the intro should allow the user to say to himself: what I came for is right here.

Direct links

Some direct entry points to relevant result selections (to be determined or refined) :

We could even make the top performer or performers appear below each part...

To illustrate the logic, something like this (this is an unworked excerpt): image

This means replacing, exclusively on the home page, the existing filters panel with direct links to certain results.

No more complete picture of results

I think we should leave it up to the user to choose what he wants to display, rather than showing him by default a table of results that is too complete or too complex, and forcing him to do manipulations to display what he's really looking for.

For the great majority of web projects, some choices are imposed (by the infrastructure, the teams, their skills), and some questions don't even arise:

It is more common to want to compare frameworks of the same language, using related technologies.

I think these direct entry points would also be more motivating and rewarding for teams working on frameworks :-)

NateBrady23 commented 4 years ago

Thanks for your thoughts on this @jcheron ! cc @bhauer

jcheron commented 4 years ago

Thanks @nbrady-techempower I just reread @bhauer opinion on this at Brian Hauer's personal blog.

I am in no way questioning the validity of the comparisons made; the full results must remain easily accessible. I only think that it is one of the possible entry points, in the same way as a more classic specific comparison between fullstack frameworks of the same language, using the same database.

I'm just in favor of one-click access:

And I'm convinced that this slight redesign of the home page can bring a much better access to the information that everyone wants to get by visiting TFB website.

bhauer commented 4 years ago

Thank you for the thoughtful input, @jcheron! It is very useful to hear how the project and its results are understood both by unfamiliar first-time visitors and by experienced contributors.

Taking your ideas as inspiration, some thoughts I'd like to run by you for discussion:

Presently, the results web site automatically directs the user to the most-recent round, and then to Fortunes on physical hardware in particular. That is, https://www.techempower.com/benchmarks/ is presently equivalent to https://www.techempower.com/benchmarks/#section=data-r18&hw=ph&test=fortune. Prior iterations of the web site landed at the introduction page.

First, I'd like your thoughts (and other contributors' thoughts) on reverting the behavior to showing the introduction when landing at https://www.techempower.com/benchmarks/. If no one has an objection in the short term, I might just make that change and see what people think.

There is a two-sentence mini-intro on the results page:

In the following tests, we have measured the performance of several web application platforms, full-stack frameworks, and micro-frameworks (collectively, "frameworks"). For more information, read the introduction, motivation, and latest environment details.

That could be rewritten or enhanced to better introduce to the project to newbies. I am open to suggestions. The reason this small paragraph is important is that even if I switch the home URL to render the introduction page, some users may arrive via a link directly to a round, perhaps with various filters enabled.

I like your idea of providing a bunch of links with pre-canned filters applied. I personally feel like those could be added under the "Results" header on the introduction page. Like you were saying, something like:

Results

View the results for Round 18:

  • All results on physical hardware.
  • All full-stack frameworks on physical hardware.
  • All micro-frameworks on physical hardware.
  • All platforms on physical hardware.
  • All implementations with full ORMs on physical hardware.
  • All implementations with micro ORMs on physical hardware.
  • All implementations not using an ORM on physical hardware. ... etc ... ... Same for cloud ...
jcheron commented 4 years ago

Thank you for your open mind @bhauer !

To answer your questions:

Displaying the results

In addition to the introduction, and to the direct access to filtered results, I think that a site dedicated to informing about the performance of frameworks should display some results (without giving details), just a partial ranking.

It could be for example the x first of each category.

On the home page, the partial display of the results should make you want to know more. In addition, it gives a more visual and dynamic aspect (information is updated) to the page.

If we have good products, we might as well showcase a part of them !

Limiting the categories for direct links

Even if our products are good, you shouldn't put them all on display.

If there are too many categories, we risk getting lost in it, and it takes up space:

I won't make some distinctions on the home page (since the user can then filter by himself in the results page): Between:

On the other hand, I think:

Just to give you a more visual idea of what I was thinking:

image

Notes

For the partial display of the results:

I think a tool_shortname field should be added for each version in the framework specifications, so that barbaric and too long names do not appear, and that the same framework is not mentioned several times, at different positions, within a category.

For displaying by language

It can be accessed by clicking on Language results (to bring up the list), and then go directly to the desired language.

jcheron commented 4 years ago

To continue:

On the precision of the Informations:

Adding a tooltip on mouse-over: image

About facilitating the access to information

As a user, if I choose to:

Imagine all the mouse batteries saved, this slight revision is necessary to reduce global warming, just as much as the optimization of our source codes ;-)

jcheron commented 4 years ago

I continue my obsessive delusion

On some Ideas for Technical Specifications:

The idea is to make the categories displayed on the home page configurable, so that the Techempower team can easily modify them, without having to modify the code.

Definition of a category (displayed on the homepage)

Displays a partial ranking and allows access to a specific filtering of results

Preview sample

image

Sample datas
{
    "title": "All Frameworks with ORM",
    "url_params": {
        "o": "c",
        "c": "4"
    },
    "results_count": 5,
    "has_languages": true,
    "color": "#B71C1C"
}

List of categories

List of categories displayed on the home page

Sample datas
{"categories": [
            {...},{...} ...
            ],
"url_params": {"test":"fortune","hw":"ph"},
"section": "data-r18"
}

Sample preview

A simple example

Complies with the existing: image

Datas
{"categories": [
        {"title": "All results",
        "url_params": { },
        "results_count": 0,
        "has_languages": false,
        "color": "#FDD735"}
        ],
"url_params": {"test":"fortune","hw":"ph"},
"section": "data-r18"
}

More categories

Just an example with 4 categories, to illustrate the configurable aspect: image

Datas
{"categories": [
        {
            "title": "All results",
            "url_params": { },
            "results_count": 5,
            "has_languages": true,
            "color": "#FDD735"
        },
        {
            "title": "All Frameworks with ORM",
            "url_params": {
                "o": "c",
                "c": "4"
            },
            "results_count": 5,
            "has_languages": true,
            "color": "#B71C1C"
        },
        { ... }
        ],
"url_params": {"test":"fortune","hw":"ph"},
"section": "data-r18"
}

It's just a thought. This way, it gives you the freedom to display what you want on the home page, and to vary the displays according to rounds or time

jcheron commented 4 years ago

@nbrady-techempower @bhauer Blimey, I was satisfied with my idea... The screens were provided only to illustrate my idea. I know that web development is your business, and I don't pretend to replace your team. My proposal may not be timely; feel free to tell me if this is the case, I know that not all ideas are acceptable in a project.

bhauer commented 4 years ago

@jcheron We do very much appreciate the ideas!

I've not yet had the time to truly digest them and we have some concurrent priorities for the results web site that will command my attention when I do have the time to allocate here.

Overall, I like the idea of providing a bit more options for how to start diving into the results. And the approach you've outlined is pretty interesting!

I'm curious if anyone else has input.

NateBrady23 commented 4 years ago

@jcheron The level of thought and detail you put into this is awesome! I wish I had the time to put that much work into a response. Like @bhauer said, there are more iterations being worked on right now to solve some problems we've been wanting to fix for some time and then there will be room to iterate on top of that.

Not to take away from anything you've done here because you make a lot of valid points, but there are some people that display the results in an unofficial capacity. It might be worth thinking about compiling a list of those unofficial results views that help people look at the data in different ways.

jcheron commented 4 years ago

Thank you for your answer, @nbrady-techempower . I think it's important to start from the principle of direct links that can be easily modified and configured by your team. This will allow you to make them evolve, depending on the feedback, or on the results of the rounds. That way, nothing is written in stone, except the structure defining the notion of direct link, a little more elaborated than a simple hypertext link.

joanhey commented 4 years ago

Actual homepage

I think that is correct. You can filter in any way, and you can send an url with that filters.

Rank order

Like @bhauer https://github.com/TechEmpower/FrameworkBenchmarks/issues/2927#issuecomment-391501795, I don't believe in that rank order.

Some frameworks are highly optimized only for this benchmark, but developers using it in real apps will not get that numbers. Perhaps they will have a faster application choosing a similar framework.

Saying:

In a car race don't win the faster car, win the better driver.

Adboard or tool?

This benchmark is a great tool that help to optimize stacks, configurations, code, ... for better performance in all frameworks.

Please don't conver it in an advertising board, based only in a synthetic ranking.

Priorities

To help us optimize the frameworks:

And the rest of issues opened in the tfb-status https://github.com/TechEmpower/tfb-status/issues

DStats

Are collected in each run, but we don't have any way to visualize it. We are optimizing blindly, without the cpu, memory, req per thread, ...

Why not adopt https://ajdust.github.io/tfbvis/ ? Is it perfect ? Not, but it's useful.

My 2¢

jcheron commented 4 years ago

Thanks for taking the trouble to answer, @joanhey . My suggestion only concerned TFB website Homepage, not its detailed content, as the title of the issue indicates.

Me too, I appreciate the tools provided, as well as the overview table presenting the results. I simply suggested to allow direct access to some filtered results (an access configurable by the TechEmpower team), from the home page, rather than displaying by default the general table, which must of course remain accessible. I thought I had sufficiently highlighted the current problem of access to information. As I illustrated, you currently need 7 clicks to compare java Frameworks using an ORM (its just an example), who could complain about being able to get this result directly?

I don't really understand the relationship between this approach, which seeks to simplify the use (for all types of users, including us), while preserving the existing... and the transformation into an advertising board that you talk about pejoratively.

As a French proverb says:

" Who wants to drown his dog accuses him of rabies "

joanhey commented 4 years ago

I only give my opinion. They have a limited time. And I think that there are other priorities to use that time.

If you take an opinion like a personal attack or to your issue, it's wrong, because it is not so.

misiek08 commented 2 years ago

Probably I missed something: where can I send PR for website itself?

NateBrady23 commented 2 years ago

@misiek08 the results site is not currently open sourced