New articles are created with last possible "ordering"

bartnikj commented 8 years ago

Steps to reproduce the issue

In Joomla 3.6.0:

Go to backend
Create article
Save and close
List all articles using ordering
Expected result

New article is listed as first one, just like before J3.6

Actual result

New article is listed as last one, which is pretty bad if you already have 1000+ articles and need to drag and drop it from bottom to the top.

System information (as much as possible)

Joomla 3.6.0

brianteeman commented 8 years ago

Seems to be the same as in 3.5.1

bartnikj commented 8 years ago

Same as "new article is last" or "new article is first". It happened to me after upgrading from 3.5.1 to 3.6.

_{This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/11103.}

bartnikj commented 8 years ago

Hmmm... that's weird. I updated 5 of my J35 to J36 and two got this problem. The other 3 are just fine... The JTable->reorder code is identical so it must be something else.

_{This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/11103.}

csthomas commented 8 years ago

What can I see in J3.6 the ordering is reversed, please check changes in #8576. @alikon, @rdeutz Could you check it?

The performance is better because reordering is removed and any new article is added at the end.

In short: Ordering was reversed for better performance.

But it has created problem with B/C for people that based on article ordering column.

I suggest to notice users that article ordering was reversed. May be advice to wait for 3.6.1.

IMHO J3.6.1 should add some sql update to reverse ordering of existed articles (on installation) before users will add new articles. And reverse ordering direction from ASC to DESC and from DESC to ASC in modules/category view, etc.

brianteeman commented 8 years ago

IMHO J3.6.1 should add some sql update to reverse ordering of existed articles (on installation) before users will add new articles. And reverse ordering direction from ASC to DESC and from DESC to ASC in modules/category view, etc.

You can't do that. It will completely break any existing site that uses ordering

csthomas commented 8 years ago

Hmm, Joomla 3.6 broke ordering, but done good thing for performance and I like that change.

I see two ways: 1) Back to ordering from 3.5.1 and still have problem with performance - reverse #8576. People who has 3.6 may be not happy. It will has mess with current new articles.

2) Go forward (and maybe add some sql updates to fix old ordering) and explain users that new article will get last number (in column ordering) and older articles have lower numbers. And they should change order direction for modules, etc.

alikon commented 8 years ago

i'm not an expert of B/C , sem 2.0 etc , so not so sure if this is an issue that break some rules when upgrading to a new minor version .... for me is an improvement.

csthomas commented 8 years ago

@alikon Your patch changed way of set ordering for a new article: Before:

new article was set as ordering=0 and all existed articles was reordering (old moved by +1 in the same category), so at the end the new article had column ordering=1

After:

new article is set as the last one in the same category and all existed articles are NOT reordering - this is performance boost

I may wrong, please check.

alikon commented 8 years ago

yes correct, but again i don't see the problem

csthomas commented 8 years ago

The problem is that the users may use some modules/articles view with sql order by "ordering ASC" to get the newest articles at the top of result - something like "Last published". In Joomla 3.6 that does not work because the newest articles go down in result, after older.

alikon commented 8 years ago

i think is wrong to use the "field ordering" to get content sorted by a date

csthomas commented 8 years ago

May be but Joomla allow to sort by ordering field and now it is work in reverse order.

If ordering field is not important for Joomla 3.6 then I do not have to discuss about it because personally I do not use it.

Other example: Line 1357: https://github.com/joomla/joomla-cms/blob/staging/libraries/joomla/table/table.php

Users in J3.5 could set some articles to be always on the top of newest articles. (sort by ordering ASC) Joomla 3.5 then allow to display extra article(-s) (means with set ordering=-1) to be always on the top of result.

IMHO: Some information about change order direction in ordering field should be added.

csthomas commented 8 years ago

I only want to mention that featured table also have ordering field which direction was reversed.

CEWebDesign commented 8 years ago

"add new article at top / at the end" setting in options would be great. (I like the changes, performance + at the end)

sanderdw commented 8 years ago

Yes i have the same 'problem', now an extra step is needed to place the article at the top. Best of both worlds would be indeed the solution joo7 describes.

csthomas commented 8 years ago

I don't think Joomla 3.6 want to back to the old order. IMHO additional option to choose direction will complicate too much.

A solution would be to accept a new direction, but... there should be a button or some function to change direction of all existed articles / featured to fix direction for old articles.

For a new installation of Joomla 3.6 category view/modules with 'article order' or 'featured articles order' display incorrectly.

Means older articles are at the top, newer at the bottom.

Take a look at: https://github.com/joomla/joomla-cms/blob/staging/components/com_content/helpers/query.php#L92-L114

IMHO It should be changed from: $orderby = 'a.ordering'; to: $orderby = 'a.ordering DESC';

and: $orderby = 'a.featured DESC, fp.ordering, ' . $queryDate . ' DESC '; to: $orderby = 'a.featured DESC, fp.ordering DESC, ' . $queryDate . ' DESC ';

Quibi commented 8 years ago

In my opinion, the behaviour of featured articles in J36 (last featured goes to last order position not to first as usual) is against usability and cannot be considered a better performance from user's point of view.

It is strange and misleading that the last article that you mark as featured becomes the last for the CMS.

In my case, as I use featured articles in my front page, my workflow has become very inefficient.

In J35 and earlier, when I wanted to highlight an article, I just featured (clicked) it in the articles list and it becomes the first of featured articles. I hardly go to the list of featured articles for nothing.

Now, after featured an article, I must go to "Featured Articles" and reorder articles. It is a waste of time and prone to errors.

I suggest that the old behaviour should be restored for the sake of users.

csthomas commented 8 years ago

I have created an example PR what should be (at least) changed. It is not completed but example.

IMHO joomla should remove/hide sort by ordering field. Can we replace it with publish_up field?

brianteeman commented 8 years ago

Being able to control the order that articles are displayed in is a fundamental function

bartnikj commented 8 years ago

In the meantime... is there any simple way to move article (in terms of order) from bottom to top? Right now I have to display all articles (1000+) and drag and drop it all the way up... which takes about 2 minutes. I guess (as this is JavaScript and jQuery) I could spend some hours to look for (or make) a single line script that would do just that in my browser's debug mode. Thank god it's not J15 where you would click those small arrows that reloads the page every time you move item by one row...

brianteeman commented 8 years ago

Are all the articles in a single category? If not then filter by the category

On 14 July 2016 at 21:49, bartnikj notifications@github.com wrote:

In the meantime... is there any simple way to move article (in terms of order) from bottom to top? Right now I have to display all articles (1000+) and drag and drop it all the way up... which takes about 2 minutes. I guess (as this is JavaScript and jQuery) I could spend some hours to look for (or make) a single line script that would do just that in my browser's debug mode. Thank god it's not J15 where you would click those small arrows that reloads the page every time you move item by one row...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/joomla/joomla-cms/issues/11103#issuecomment-232788085, or mute the thread https://github.com/notifications/unsubscribe/ABPH8UEJc2OfsdZT6lYZmz1yBYDjkFxYks5qVqDlgaJpZM4JLVTd .

Brian Teeman Co-founder Joomla! and OpenSourceMatters Inc. http://brian.teeman.net/

bartnikj commented 8 years ago

Yup, already tried that. It narrowed my list from 1700 to 1600 articles :P

bartnikj commented 8 years ago

Idea: in article list there's a dropdown icon in every row. It could contain extra icon with "move to top" functionality (available only when articles are displayed ordered).

Ideally 3.6 update should rebuild ordering in articles so the change would be transparent to users (but on the other hand it would break some addons if they use ordering column).

csthomas commented 8 years ago

This is my thought only.

Lazy way and not tested by me Use files from backup and unpack joomla 3.5.1 files to other folder, test it and replace it if everything works. There is no too much changes in database so should works.

Full solution and not tested [DELETED]

bartnikj commented 8 years ago

Wouldn't just "reversing" the order in SQL be enough? Like that:

grab maximum ordering value ORDERING_MAX
assign new ordering value of ORDERING_MAX-ordering

csthomas commented 8 years ago

You can reserve the order of article per cat_id field. In category "One" you have articles with ordering field with value 1,2,3,... In category "Two" you also have articles with ordering field with values 1,2,3,...

ggppdk commented 8 years ago

@csthomas

Reversing the ordering is a B/C break for existing web-sites that already have an ordering set

Proper way to have done this:

when table re-ordering is triggered it changes positives orders to start from 10,000 and NOT from 0
then new article ordering will then be MIN(OF positive orderings) - 1, without re-ordering needed to be done
when MIN(OF positive orderings)-1 reaches zero, a table re-ordering is forced to run

csthomas commented 8 years ago

Take a look at https://github.com/joomla/joomla-cms/pull/8563#issuecomment-225231023

I have similar idea but instead of 10000 I use 2147483647 (MAX_INT_IN_MYSQL) And I do not use reordering when MIN(OF positive orderings)-1 reaches zero.

10,000 is too small number.

ggppdk commented 8 years ago

@csthomas 10,000 is just an example

and you can not use the max allowed you probably would want to use MAX_ALLOWED / 2 ... think of why ...

ggppdk commented 8 years ago

About the re-order code in JTable reorder()

it does 1 update per row

We can update 1000 or 20,000 records with a single query (depending on SQL server configuration), with a great performance effect (ok 10 queries to 1 queries you would not see much difference but 1000 to 1 you will see big !!)

@alikon (correct me here but the following works in ALL db server ??, because it is standard SQL ??):

UPDATE  __content
    SET ordering = CASE id
        WHEN 45 THEN '100000'
        WHEN 34 THEN '100001'
        WHEN 53 THEN '100002'
        WHEN 17 THEN '100002'
    END
WHERE id IN (45,34,53,17)

so the PHP loop will collect / calculate the new orderings,
then issue the updates in 1000 chunks

In fact with default MAX_PACKET_SIZE of 1MByte in MySql you could do more than 20,000 rows in a single query, so 1,000 is safe, i mention MySql only as an example of how many records can be updated

csthomas commented 8 years ago

I know that signed limits is half.

Currently for Joomla 3.5.1 and mysql I use: https://github.com/joomla/joomla-cms/pull/8563#issuecomment-224108944

To support all database types there will be more work.

ggppdk commented 8 years ago

To support all database types there will be more work

@csthomas

please clarify, so you say that SET - CASE - WHEN - THEN - END

(WHERE ... IN ...) is for performance ...

is not supported by all DBs ?? my mistake , i thought it was , which DB server does not support it ?

csthomas commented 8 years ago

This is misunderstanding. I said about my comment from #8563.

csthomas commented 8 years ago

What do you think of gaps in numbering:

ex. article with ordering=10002 may be missing and we have articles with ordering field values 10001, 10003, 10004, 10005,...

That should make PR simpler to do because Joomla does not need to reorder table on create article / delete artcile.

csthomas commented 8 years ago

If someone use patch from PR #11134 then I suggest to revert it and use new one from PR #11139. @ggppdk What do you think of PR #11139?

bartnikj commented 8 years ago

Ok, as J36 broke existing ordering (making moving from 3.5 more of a migration than upgrade) and it's mostly because of performance issue why just not shift article ordering while doing reordering and keep order direction from pre-J36 era. Example would be:

We had articles with ordering 0,1,2,3,4
Shift them by around 2147483647/2 minus (ARTICLE_MIN_ORDERING+ARTICLE_MAX_ORDERING)/2 You'd have to do it once during an upgrade (or after first reorder) and it wouldn't break existing installations.
Now you should have 1073741820,1073741821,1073741822,1073741823,107374184.

Or... if it's signed INT... we could go negative numbers and no shifting is needed. Performance and backward compatibility at the same time :)

ggppdk commented 8 years ago

Or... if it's signed INT... we could go negative numbers and no shifting is needed

you should not use negative numbers as these can be utilized for "sticky" behaviour = meaning they are not subject to reordering

ggppdk commented 8 years ago

Anyway the PR of @csthomas looks good, i will test and review code

gwsdesk commented 8 years ago

Performance upgrades in code that brings people to despair as we can see on the Joomla forums is pure wrong. See for instance http://forum.joomla.org/viewtopic.php?f=9&p=3414231#p3414231 or http://forum.joomla.org/viewtopic.php?f=710&t=929169 . These posts show exactly why this change in Joomla 3.6 is wrong. Stating that "i think is wrong to use the "field ordering" to get content sorted by a date" is just one opinion. Look at this 'functionality' over the past years alone on the forums. I request this to be rapidly reversed since many users are in trouble and after we revert this we provide a new and better performance in a next release? We cannot leave so many users in the cold?

ggppdk commented 8 years ago

@gwsdesk

yes it is a B/C break and #11139 should fix it, can you test it ? i will do too , i hope "today"

gwsdesk commented 8 years ago

I am not sure if that makes sense based on the comments posted so far

csthomas commented 8 years ago

We have to take a decision, which number should be put in ordering field for a new article, frontpage (featured), module, banner or other table.

[my English is weak]

Options: 1) 1as in joomla 3.5.1 or min(ordering) - 1. This is a little problem with display big numbers to users (example PR #11139) Currently my PR use ordering with big numbers only for content and content_frontpage. This is B/C for 3.5.1 and lower but not for 3.6.

2) max(ordering) + 1 as in joomla 3.6 (example/incomplete patch at PR #11134)

This is the least complicated way for coders.
Break B/C.
Other extension developers would complain about it.

But it is the simplest way for future maintains. Only we have to change the thinking of ordering field.

For 3.5.1 and lower:

"Article Order: Ordering" means ordering ASC (from the last added to the oldest)

it may be worth to change it and break B/C and all user (who use ordering column) has to change Article Order from Ordering to Reversed Ordering.

Then we will have (reversed situation):

"Article Order: Reversed Ordering" (sql: ordering DESC) - from the last to the oldest. "Article Order: Ordering" (sql: ordering ASC) - from the oldest to the last.

gwsdesk commented 8 years ago

IMHO under no circumstances can we ask people to manually re-arranged thousands of articles. I think it is wrong to "quickly" patch this issue with a solution that is not covering all options and is not well discussed nor using proper coding as I read the comments correct #11139. Once again I propose to PLT to revert this to J3.5.1 methods and after that we work on a proper solution that covers all ordering (featured/articles/categories/modules/ect

xupacabra commented 8 years ago

Hi guys. I use Joomla since version 1.0 and have had installed more than 50 sites, always helping people. It started with games and then migrated to more serious things like companies that needed help in spreading your products or services. Even the Brazilian surf museum site, won one. The greatest ever have more of 40 million visitors and I am very grateful for all that joomla team made over the time. It is very good to help everyone. But this latest version is aging me.

Off Topic

Migration was hard. I had to buy a component to help me, but I ended up losing the comments Jcomment ... I see on the internet that some people have managed, but I left it forward. The administration area was very slow when saved articles into categories that have many. I have a 14,000 items and consumes a minute to write an article.

End of Off Topic

Featured Artices, sorting by negative id for an article to stay fixed on the top, it was a clean and simple solution. After the migration, my fixed articles are spread around the site and get them to the top manually is terrible. Making changes for QUERY works, but we do not have the option to reorder all list. It was magical.

Well, once again thanks for everything and we will wait if we can resolve this point.

In this image: The best solution off all times: http://www.infosolutionsgoa.com/cms/img4/ordering-menu-items.gif

ggppdk commented 8 years ago

Your off topic comment is not off topic , it was why the B/C break in the ordering was created , which is also a usability issue

csthomas commented 8 years ago

The administration area was very slow when saved articles into categories that have many. I have a 14,000 items and consumes a minute to write an article.

To speed up adding new articles You can try this one, but before do it please create a COPY of #__assets table at least. http://www.itoctopus.com/creating-new-articles-on-your-joomla-website-is-taking-a-long-time-clean-your-assets-table

from that forum http://forum.joomla.org/viewtopic.php?t=876649

bartnikj commented 8 years ago

As the thread still's alive: what are the options right now? 3.6 should not be an upgrade but migration. B/C could be kept just by shifting ordering instead of reversing it. Same/similar performance. Options:

Keep it as it is - adding new articles is faster, but the time saved is wasted on manually reordering the newest article to be the first one.
Manual fix: reverse your ordering in SQL by using UPDATE j3_content j SET j.ordering = 10000 - j.ordering to fix the B/C in com_content... but the option to sort articles by ordering DESC is not present in Joomla (that should really be fixed with J36 ordering change)... The downside is - if anything else rely on this ordering the way it was it'll break.
Use @csthomas fix that is adding before-J36-like ordering option and deal with performance degradation over time.
Same as 3. but with UPDATE j3_content j SET j.ordering = 100000 + j.ordering. And the part that is probably not ready yet - new articles should get ordering of MIN(ordering)-1 instead of 0.

I'm definitely a fan of 4., so the shift method. And as a quick patch there should be extra "reversed ordering" option in article ordering (as I can live with 2. if I only had an option to order things on the frontend J36-style)

csthomas commented 8 years ago

I think that my PR is not understand correctly.

I'm start talking about ordering from J3.5, means articles has ordering like: 1,2,3,4,5,6,7,8,9

after add a new one we get: 0,12,3,4,5,6,7,8,9 then problematic reorder do 1,2,3,4,5,6,7,8,9,10
Then I applied my PR:
Before adding any new articles I have: 1,2,3,4,5,6,7,8,9,10
after add a new article:
joomla find that min(ordering) -1 is less that 1
then trigger reorder method (only once when we move from <= J3.6.0 to my PR)
new ordering after reorder will be (LAST means 2147483647):
LAST-10, LAST-9, LAST-8, LAST-7, LAST-6, LAST-5, LAST-4, LAST-3, LAST-2, LAST-1
above sort is not changed, this is only +offset:
new_ordering = LAST - COUNT(articles) - 1 + old_ordering
means old (1) = LAST-10, (2) = LAST-9, ...
after that joomla find again min(ordering) -1 and get LAST-11, then use it and we have:
LAST-11, LAST-10, LAST-9, LAST-8, LAST-7, LAST-6, LAST-5, LAST-4, LAST-3, LAST-2, LAST-1
next time when we want to add a new article we do not need to reorder because now ordering has large numbers.

If that explanation is still problematic.

Then open text editor write a few words:

This is old behavior.
This is new behavior.

Then adjust second line to the right.

This is old behavior.                                                               |MAX
                                                               This is new behavior.|MAX

Before we have: align to the left side (1,2,3,4,5) After we will have: align to the right side (LAST-5, LAST-4, LAST-3, LAST-2, LAST-1)

Then add a new word (NEW):

NEW This is old behavior.                                                           |MAX
                                                           NEW This is new behavior.|MAX

At the first line we have to shift old words to the right. Now you see that no shift was required for second line.

xupacabra commented 8 years ago

One thanks to all ! well, now, when I mark one article with FEATURES, it receive one large number. And my only option is change the order by PHPMyadmin

The article 14409 was write today, Featured checked and receive ORDERING = 13922 in xxx_content_frontpage

csthomas commented 8 years ago

This is known bug in 3.6.0.

If you want to help you can test PR #11139 on some dev joomla. Instead 1,2,3 numbers you will get bignumber-3, bignumber-2, bignumber-1.

For temporary you can also revert #8576 also on production. This slow down adding a new articles but new article will get number 1 instead 14409.

joomla / joomla-cms