Libraries: empty categories for versions

[vinniefalco]

When an older version is selected and there are no libraries in a category, the category is still displayed. It would be better not to display the category. And if the last library is removed from a category in a release, the category should disappear (this might automatically work as a consequence of fixing the former problem). It would seem like categories need to have a version number in the database table.

https://www.boost.io/libraries/by-category/?version=boost-1-34-0

[sdarwin]

For most categories, they should usually be displayed "by default", if testing a future version of docs such as /1_95_0/ or /1_98_0/, even when the version and category information is incomplete, it's better to show the categories.

A solution to this can be a version mask, which specifies when to hide a version.

Add a column to the "libraries_category" table, named "version_mask". It's contents could take the format "1.30.0, 1.31.0, 1.32-1.40". Those values indicate when to hide the category. Otherwise it's displayed.

The version mask can be calculated during the import of a release in the celery job, and/or during periodic celery jobs.

[vinniefalco]

I think that the categories table should have a column for version, and the version should become part of the key. When the page displays categories it uses "SELECT * FROM category WHERE version = $1". This allows the page to reflect exactly the unique union of categories across all library metadata from that tagged version. Categories can come and go, and we don't need to code for it explicitly. @frankwiles is this sound?

[sdarwin]

Then there is an extreme amount of duplication, and the table is 100 times larger than it needs to be. Also consider multiple betas, release candidates.

[vinniefalco]

How many kilobytes would it take to store 100 versions of categories duplicated from 1.86.0?

[sdarwin]

It doesn't have to just be "WHERE version = $1", but such as "WHERE version IN $1" or similar types of SQL statements. That is, version can be consolidated into one field. Well, let's see what Frank thinks.

[vinniefalco]

How many kilobytes would it take to store 100 versions of categories duplicated from 1.86.0?

[sdarwin]

It's not just a question of KB, but about elegant database design, best practices, and db normalization techniques. If both methods are possible.

[vinniefalco]

You said "there is an extreme amount of duplication" and "the table is 100 times larger." And I'm asking, how many kilobytes would it take to store 100 versions of categories duplicated from 1.86.0? I think it is a fair question.

[sdarwin]

Whatever it would be, times x100. Or x200. Still not overly large in KB.

Right now, you have this nice, pretty, clear little table with 20 rows.

Imagine then, it's 5000 rows, just repeating the same thing, over and over and over and over, for no reason.

[grafikrobot]

You already detect the condition in the ui and display a special message. Just change that to hide the box instead.

[sdarwin]

Yes, just hide it on the front end UI.

[vinniefalco]

for no reason.

It is a tradeoff, and there is a reason. If someone adds a new category in a future version, say 1.87.0, then it will immediately acquire a mask which is a list of more than 86 versions (each of 1.0.0 to 1.87.0). And we can never delete the category subsequently, because it is needed for the older versions. Yet we still have to see it and filter every category for every version.

And if a category is discontinued, then with every new release we have to update that category to add the new release version into the mask.

[sdarwin]

So you want to send information to the client just to have it hide the data?

"send information to the client". Django's model-view-controller pattern uses server-side rendered templates. Not client-side javascript. Filtering can occur before sending to the client. 2. Boost's "categories" have been very stable over 20 years. The case to optimize for is... when categories are nearly always the same.