Problem of under- and overscrolling

Problem

Long messages in history cause profanity to "underscroll": despite user scrolling to top/bottom of history, factual position is offset from the intended location. E.g. all messages are numbered 1 to N, where 1 is the first and N is the last message, profanity shows message 30 on top of the screen and scrolling up does not work since profanity perceives it as it shows message 1.

Another problem is "overscrolling": when long messages (longer than y axis of the terminal) are fetched from the DB, profanity scroll "jumps" to the top, skipping some messages instead of smooth scrolling which is happening when the messages are in the internal buffer (ProfBuffer).

Explanation

NCurses has let's call it a "buffer" with the size of _maxy, something like a hidden "window" and we can move inside of it by setting y_pos.

// curses.h
#define NCURSES_SIZE_T short

struct _win_st
{
NCURSES_SIZE_T _cury; // current y position
// ...
NCURSES_SIZE_T _maxy;
}

We have our "buffer" as well (ProfBuff defined in buffer.c), our buffer is limited to 200 messages (my new limit, before it was 1000).

// buffer.c
#define BUFF_SIZE 200

While nCurses' buffer is just 999 lines (getmaxy(window->layout->win) in win_page_up() will shows this number, as well as getcury(window->layout->win) maximal value with a few long buffered messages).

When we get e.g. 10 100-line messages, buffer gets overwhelmed and when they are near the scrolling area end (e.g. end or start of history), we are unable to reach them, since profanity tries to "redraw" e.g. 3000 lines, while ncurses will only show last 999 of them (now I noticed that there is a problem even with my theory since last messages are affected by the problem as well).

I changed a bit code for tracking numbers and significantly improved naming (if anyone wants to join investigation):


void
win_page_up(ProfWin* window)
{
    _reached_bottom_of_database = FALSE;
    int lines_ncurses_buffered = getcury(window->layout->win);
    int page_space = getmaxy(stdscr) - 4;
    int* pos_in_buffer = &(window->layout->y_pos);
    log_warning("page_up-1; page_space=%d, lines_ncurses_buffered=%d, pos_in_buffer=%d", page_space, lines_ncurses_buffered, *pos_in_buffer);

    *pos_in_buffer -= page_space;

    if (*pos_in_buffer == -page_space && window->type == WIN_CHAT) {
        ProfChatWin* chatwin = (ProfChatWin*)window;
        ProfBuffEntry* first_entry = buffer_size(window->layout->buffer) != 0 ? buffer_get_entry(window->layout->buffer, 0) : NULL;

        // Don't do anything if still fetching mam messages
        if (first_entry && !(first_entry->theme_item == THEME_ROOMINFO && g_strcmp0(first_entry->message, LOADING_MESSAGE) == 0)) {
            if (!_reached_top_of_database) {
                _reached_top_of_database = !chatwin_db_history(chatwin, NULL, NULL, TRUE);
            }

            if (_reached_top_of_database && prefs_get_boolean(PREF_MAM)) {
                win_print_loading_history(window);
                iq_mam_request_older(chatwin);
            }
        }
    }

    // went past beginning, show first page
    if (*pos_in_buffer < 0)
        *pos_in_buffer = 0;

    log_warning("page_up-2; page_space=%d, lines_ncurses_buffered=%d, pos_in_buffer=%d", page_space, lines_ncurses_buffered, *pos_in_buffer);
    window->layout->paged = 1;
    win_update_virtual(window);

    // switch off page if last line and space line visible
    if ((lines_ncurses_buffered) - *pos_in_buffer == page_space) {
        window->layout->paged = 0;
    }
}

void
win_page_down(ProfWin* window)
{
    _reached_top_of_database = FALSE;
    int rows = getmaxy(stdscr);
    int y = getcury(window->layout->win);
    int page_space = rows - 4;
    int* page_start = &(window->layout->y_pos);
    log_warning("page_down-1; page_space=%d, lines_ncurses_buffered=%d, pos_in_buffer=%d", page_space, y, *page_start);

    *page_start += page_space;

    // Scrolled down after reaching the bottom of the page
    if ((*page_start == y || (*page_start == page_space && *page_start >= y)) && window->type == WIN_CHAT) {
        int bf_size = buffer_size(window->layout->buffer);
        if (bf_size > 0) {
            auto_gchar gchar* start = g_date_time_format_iso8601(buffer_get_entry(window->layout->buffer, bf_size - 1)->time);
            GDateTime* now = g_date_time_new_now_local();
            gchar* end = g_date_time_format_iso8601(now);
            // end is free'd inside
            if (!_reached_bottom_of_database && !chatwin_db_history((ProfChatWin*)window, start, end, FALSE)) {
                _reached_bottom_of_database = TRUE;
            }

            g_date_time_unref(now);
        }
    }

    // only got half a screen, show full screen
    if ((y - (*page_start)) < page_space)
        *page_start = y - page_space;

    // went past end, show full screen
    else if (*page_start >= y)
        *page_start = y - page_space - 1;

    log_warning("page_down-2; page_space=%d, lines_ncurses_buffered=%d, pos_in_buffer=%d", page_space, y, *page_start);

    window->layout->paged = 1;
    win_update_virtual(window);

    // switch off page if last line and space line visible
    if ((y) - *page_start == page_space) {
        window->layout->paged = 0;
    }
}

Potential solution

Count lines in our buffer -- by counting '\n' symbols in each and keeping "total" number, which is adjusted on each deletion/addition for faster calculations. It might be not a perfect solution, since we are "wrapping" (/wrap on/off messages) by default.

@sjaeckel

profanity-im / profanity