Closed JBoschma closed 7 months ago
Thank you Jeroen for the detailed issue.
I will dig into this but I suspect that the two asserts are incorrect. The ImGuiInputTextFlags_CallbackCompletion
(for Tab handling) and ImGuiInputTextFlags_CallbackHistory
(for Up/Down handling) are designed to allow test editing and they are going to the same path as the ImGuiInputTextFlags_CallbackEdit
callback. So it could be I simply failed to test for both dynamically-resizing and use of those features together (or presumably that most of our tests involved local buffer big enough to avoid a realloc).
Out of curiosity, could you clarify what's the user scenario leading you to programmatically append to that buffer?
( Three info tangential to this:
if (ImGuiInputTextState* input_state GetInputTextState()) { input_state->ReloadUserBufAndXXX()
functions it become easier to do some operations.InputText()
needs to be rewritten to avoid the extra buffer, but somehow recently my urge to do it has grown enough that I suspect I might undergo this in the following months.But please wait because taking other path as I may confirm soon that the asserts are incorrect. )
So it could be I simply failed to test for both dynamically-resizing and use of those features together (or presumably that most of our tests involved local buffer big enough to avoid a realloc).
So I noticed that the widgets_inputtext_callback_misc
test actually exercise both paths simultaneously via the Str
helper type used in imgui_test_suite: https://github.com/ocornut/imgui_test_engine/blob/main/imgui_test_suite/imgui_tests_widgets_inputtext.cpp#L1004
I thought that maybe the edits are accidentally not resizing buffers, and make the edits larger....
....Then I realized my initial answer was completing wrong, as I fell into the same mistake that you failed into:
// Check for editing.
// PROBLEM: not allowed to change the buffer-pointer, so cannot add text (of unknown length).
if (data->EventFlag == ImGuiInputTextFlags_CallbackEdit) {
// Add stuff to the text-buffer, this may dynamically change its memory location.
user_string = "XXXXXXXXXXXXX";
bool changed_buf = true;
This is not correct: with the current design of InputText()
you are not supposed to write to your own buffer there, but you can call ImGuiInputTextCallbackData::InsertChars()
and DeleteChars()
, this will manage our buffer and then the changes will be reflected to your own buffer by a _Resize callback followed by a copy.
That's the sorts of stuff I am hoping to improve with a rewrite of InputText().
Thanks for your explanation Omar.
I tried to do some word-wrapping stuff with the multiline input text. The approach I took is that after each modification of the text I recalculate word-wrapping:
Removing the word-wrap \r\r\n sequence is now implemented fairly efficient with two pointers (src, dst), inserting then and copying text is done via a temporary std::string. But that approach now fails if only InsertChars() and DeleteChars() are allowed.
Anyway, your answer made clear where my options are and I'll do some rewriting to make it work. While reading back this comment I was thinking about the following: prepare my modified word-wrapped text as I do now (in a temp-buffer though), then use DeleteChars() to delete all text from the ImGui buffer, then use InsertChars() to copy all of my text back. That's just a few lines of code modification.
Thanks for all your work on ImGui, it's really fun working with it and exploring all it's possibilities.
Up to you but I personally wouldn't attempt word-wrapping at this level, seems too complicated to do reliably (not mentioned the effect on undo-stack).
This is one of the thing that I expect to solve with that V2, too.
Well, I can have the fun to get it working for now and then switch to a built-in solution when that becomes available.
I guess we can close this now. Or keep it open for a couple of days so I can paste some working code inhere, just as 'inspiration'?
Took more time than expected... Below the code I now use in my project, it is a wrapper around ImGui::InputTextMultiline() accepting a std::string. I did quite some editing and testing and all seems OK (single character or sequence of characters add/delete). Performance is good enough for me: 1800us for 20k characters when typing. In my application I do not expect a user to type in more than 1000 characters or so.
In order to get it working you'll need 2 things to do:
Below the code, maybe as inspiration or copy/paste in someone else's project. Regard it as public domain.
//=============================================================================================
// Include and C-preprocessor section.
//=============================================================================================
// Standard C++.
#include <string>
#include <vector>
// PRIMOS GUI includes: my own header.
#include "draw_widgets/draw_word_wrap_text_editor.h"
// PRIMOS engine includes.
#include "engine/utils/timer.h"
// A few macros for ease of writing and understandability.
#define CHAR_00 ('\0')
#define CHAR_SP (' ')
#define CHAR_TAB ('\t')
#define CHAR_CR ('\r')
#define CHAR_NL ('\n')
#define IS_00(s) (s == CHAR_00)
#define IS_SP(s) (s == CHAR_SP)
#define IS_TAB(s) (s == CHAR_TAB)
#define IS_CR(s) (s == CHAR_CR)
#define IS_NL(s) (s == CHAR_NL)
//=============================================================================================
// Code section.
//=============================================================================================
namespace {
// Implementation notes, March 2024, Jeroen Boschma.
//
// Goal for this word-wrap code is to have a reasonable working implementation for text up
// to a maximum of maybe 10k characters, as in 'filling in a field of a form'. For larger
// texts, word-wrapping must be integrated with the editor logics of Dear ImGui.
//
// This implementation can only be suboptimal as it is not integrated with the editor logics
// but is only a wrapper on 'the outside'. A first attempt tried to embed newline characters
// between non-printable codes to make the easy recognizable for the word-wrapping code, but
// two problems were encountered:
//
// 1) ASCII codes like 0x01 are printed, with a questionmark or a fallback character, and
// are therefore visible. These codes also affect the cursor behavior.
// 2) The carriage return character can also not be used:
// a) although not printed at all, it still affects the cursor behavior
// b) user-text pasted into the editor may also contain those characters
//
// A new approach, as implemented below, does the following:
//
// 1) The '\r' character is eliminated from the user-text (it affects cursor behavior).
// 2) The user-text is then word-wrapped (adding newlines), and passed to the ImGui editor.
// A std::vector<size_t> is stored internally which holds the indices of the inserted
// word-wrap newlines.
// 3) A copy of the word-wrapped text is stored internally.
// 4) In the editor edit-callback, the '\r' character is eliminated from the text in the
// ImGui buffer (could be inserted there due to a text-paste).
// 5) Then the text in the ImGui buffer is compared with the internally stored copy in
// order to detect the modifications, i.e. adding or deleting a single character or
// a sequence of characters.
// 6) Then the inserted word-wrap newlines are removed, the resulting text is placed in
// the std::string the user originally specified to keep that updated.
// 7) Word-wrapping is applied again, the result is stored in the ImGui buffer but not in
// the internal copy because ImGui::InputTextMultiline() now returns.
//
// For large pices of text, word-wrapping should be an internal processing job:
//
// - you have better information on the exact editing action
// - you can limit word-wrapping to a single paragraph (ended by a user-newline)
// instead of the entire text (I was too lazy to implement that, current code is
// good enough for my application)
// - better job in keeping the undo-stack functioning correctly...
//
// Current performance (2024, laptop-CPU = i7, very crude/indicative figures):
//
// without editing : 500 us for 20k characters.
// with editing : 1800 us for 20k characters.
//
// Good enough for me, the PRIMOS application uses much less characters...
// User-data for the word-wrapping callback.
struct ProcessingUserData
{
// Constructor.
ProcessingUserData() = delete;
ProcessingUserData(std::string& user_string,
std::string& imgui_string,
float max_width,
bool enable_debug) : user_string(user_string),
imgui_string(imgui_string),
ww_ptr_in(nullptr),
ww_ptr_out(nullptr),
ww_max_line_width(max_width),
ww_line_width(0),
ww_space_width(ImGui::CalcTextSize(" ").x),
ww_tab_width(4.0f*ImGui::CalcTextSize(" ").x),
ww_out_index(0),
ww_end_of_line(false),
enable_debug(enable_debug),
relocated_nl_indices_trend_ok(true),
relocated_nl_indices_in_range(true),
relocated_nl_error_count(0)
{
}
// Properties: (word-wrapped) strings.
std::string& user_string; // Refers to the user string, contains non-word-wrapped text.
std::string& imgui_string; // Refers to the string used as buffer by ImGui (always word-wrapped).
std::string imgui_string_copy; // A copy which is not modified by ImGui, so we use it to detect changes.
// Properties: used during word-wrapping.
char *ww_ptr_in;
char *ww_ptr_out;
float ww_max_line_width, ww_line_width, ww_space_width, ww_tab_width;
size_t ww_out_index;
bool ww_end_of_line;
std::vector<size_t> ww_nl_indices;
// Debugging of word-wrapping.
bool enable_debug;
bool relocated_nl_indices_trend_ok;
bool relocated_nl_indices_in_range;
int relocated_nl_error_count;
};
// Remove the '\r' character: char* pointer. Returns new string length.
size_t RemoveCarriageReturns(char *ptr)
{
// Initialize base pointer
char* ptr_in = ptr;
char* ptr_out = ptr;
// Start copying characters.
while (!IS_00(*ptr_in)) {
if (!IS_CR(*ptr_in))
*ptr_out++ = *ptr_in;
ptr_in++;
}
// Also copy the null-terminator.
*ptr_out = *ptr_in;
// Return the size.
return ptr_out - ptr;
}
// Remove the '\r' character: std::string.
void RemoveCarriageReturns(std::string& str)
{
// Because we end up with less/same characters, we can safely do an in-place
// removal. Even if str.resize() relocates memory, it will copy all correct
// characters based on its larger internal size.
size_t new_size = RemoveCarriageReturns(str.data());
str.resize(new_size);
}
// Add spaces from the source to the current word-wrapped line. Note: the string
// terminator '\0' is not yet copied.
void WordWrapAddSpacesToCurrentLine(ProcessingUserData& user_data)
{
char this_char = *user_data.ww_ptr_in;
while (this_char <= 32) {
// Done?
if (IS_00(this_char)) {
user_data.ww_end_of_line = true;
return;
}
// Space or tab have a width (latter counts as 4 spaces).
if (IS_SP(this_char) || IS_TAB(this_char)) {
float char_width = IS_SP(this_char) ? user_data.ww_space_width : user_data.ww_tab_width;
// Check if it fits on the line.
if (user_data.ww_line_width + char_width < user_data.ww_max_line_width) {
// Fits, so update current line-width.
user_data.ww_line_width += char_width;
}
else {
// Does not fit, so first insert a word-wrap newline.
user_data.ww_nl_indices.push_back(user_data.ww_out_index++);
*user_data.ww_ptr_out++ = CHAR_NL;
user_data.ww_line_width = char_width;
}
}
// Copy the character.
++user_data.ww_out_index;
*user_data.ww_ptr_out++ = *user_data.ww_ptr_in++;
// We started a new line if it's a newline.
if (IS_NL(this_char))
user_data.ww_line_width = 0;
// Next character.
this_char = *user_data.ww_ptr_in;
}
}
// Add a word from the source to the current word-wrapped line. Note: the string
// terminator '\0' is not yet copied.
void WordWrapAddWordToCurrentLine(ProcessingUserData& user_data)
{
// Done?
if (IS_00(*user_data.ww_ptr_in)) {
user_data.ww_end_of_line = true;
return;
}
// Find character behind the end of the word.
char *end_ptr = user_data.ww_ptr_in;
while (*end_ptr > 32)
++end_ptr;
// Check the word we found. Temporary terminate the word with a '\0'.
char temp_char = *end_ptr;
*end_ptr = CHAR_00;
float word_width = ImGui::CalcTextSize(user_data.ww_ptr_in).x;
*end_ptr = temp_char;
// Check if it fits on the line.
if (user_data.ww_line_width + word_width < user_data.ww_max_line_width) {
// Fits, so update current line-width.
user_data.ww_line_width += word_width;
}
else {
// Does not fit, so first insert a word-wrap newline.
user_data.ww_nl_indices.push_back(user_data.ww_out_index++);
*user_data.ww_ptr_out++ = CHAR_NL;
user_data.ww_line_width = word_width;
}
// Add the word.
while (user_data.ww_ptr_in != end_ptr) {
++user_data.ww_out_index;
*user_data.ww_ptr_out++ = *user_data.ww_ptr_in++;
}
}
// Do word-wrapping.
void DoWordWrap(std::string& str_in, std::string& str_out, ProcessingUserData& user_data)
{
// Resize the out-string to 2 times the input, this is a theoretical requirement when we
// have all spaces in the input, almost zero line-width and then every space is word-
// wrapped. Plus one for the null-terminator. Note: when we resize here with this size,
// we know we will end up with less/same characters in the result. That means we can then
// safely do a resize to less characters which would not be the case if we did reserve().
// With reserver() the output string can have an unknown low size() when we start, and if
// we then resize() out result to a higher string-length the std::string only copies
// characters according to its lower size() if it also has to reallocate memory.
str_out.resize(2 * str_in.size() + 1);
// Prepare struct with data for the word-wrapping functions.
user_data.ww_ptr_in = str_in.data();
user_data.ww_ptr_out = str_out.data();
user_data.ww_line_width = 0;
user_data.ww_out_index = 0;
user_data.ww_end_of_line = false;
user_data.ww_nl_indices.clear();
user_data.ww_nl_indices.reserve(str_in.size()/5);
// Do word-wrapping.
while (!user_data.ww_end_of_line) {
WordWrapAddSpacesToCurrentLine(user_data);
WordWrapAddWordToCurrentLine(user_data);
}
// Copy the string terminator and fix the size of the output std::string.
*user_data.ww_ptr_out = CHAR_00;
str_out.resize(user_data.ww_ptr_out - str_out.data());
}
// Word-wrapping is done with newlines. Eliminate them from the ImGui buffer and write
// the result to the user-text.
void CleanStringFromImGuiToUser(ProcessingUserData& user_data, char* imgui_buf)
{
// Remove carriage returns from the ImGui buffer.
size_t imgui_buf_size = RemoveCarriageReturns(imgui_buf);
// Check if the string is empty.
if (imgui_buf_size == 0) {
user_data.user_string.clear();
return;
}
// Check if there are word-wrap newlines.
if (user_data.ww_nl_indices.empty()) {
user_data.user_string = imgui_buf;
return;
}
// Find out what the modifications were by comparing the ImGui buffer with the local copy.
// We'll start searching from the beginning as well from the end in order to find the
// single character (type/delete) or sequence of characters (paste/delete) that was
// modified in the buffer. The back-pointers start at '\0', this is important because
// in the search loop the front-pointers can also reach the null-terminator.
char* buf_ptr_front = imgui_buf;
char* buf_ptr_back = buf_ptr_front + imgui_buf_size;
char* copy_ptr_front = user_data.imgui_string_copy.data();
char* copy_ptr_back = copy_ptr_front + user_data.imgui_string_copy.size();
// Start searching from the front. Result: the front-pointers are on the first
// non-matching character, or at '\0' (the back-pointers).
while ((*buf_ptr_front == *copy_ptr_front) &&
(buf_ptr_front != buf_ptr_back) &&
(copy_ptr_front != copy_ptr_back)) {
++buf_ptr_front;
++copy_ptr_front;
}
// Search backwards, but not further than the above found front-pointers. Result:
// the back-pointers are on the first non-matching character, or at a front-pointer.
while ((*buf_ptr_back == *copy_ptr_back) &&
(buf_ptr_back != buf_ptr_front) &&
(copy_ptr_back != copy_ptr_front)) {
--buf_ptr_back;
--copy_ptr_back;
}
// Convert the pointers to indices.
size_t copy_front_index = copy_ptr_front - user_data.imgui_string_copy.data();
size_t copy_back_index = copy_ptr_back - user_data.imgui_string_copy.data();
// So: the text before the front-pointer and behind the back-pointer are not modified.
// We now have to relocate the word-wrap newlines, which are valid for the copy of the
// ImGui buffer, so they become valid for the modified ImGui buffer:
//
// - up to the front pointer remains the same
// - behind the back-pointer: relocate based on the distance to the string end
int back_index_offset = static_cast<int>(imgui_buf_size - user_data.imgui_string_copy.size());
std::vector<int> relocated_ww_nl_indices;
relocated_ww_nl_indices.reserve(user_data.ww_nl_indices.size());
for (size_t i = 0; i < user_data.ww_nl_indices.size(); ++i) {
int index = static_cast<int>(user_data.ww_nl_indices[i]);
// Direct copy if before front pointer. Comparison '<' is essential.
if (index < copy_front_index)
relocated_ww_nl_indices.push_back(index);
// Behind back pointer? Comparison '<=' is essential.
if (index >= copy_back_index)
relocated_ww_nl_indices.push_back(index + back_index_offset);
}
// Check if there are relocated word-wrap newlines.
if (relocated_ww_nl_indices.empty()) {
user_data.user_string = imgui_buf;
return;
}
// The result thus far was a crucial step that we might want to check.
if (user_data.enable_debug) {
user_data.relocated_nl_indices_in_range = true;
user_data.relocated_nl_indices_trend_ok = true;
// Check if the indices still increase constantly.
for (size_t i = 1; i < relocated_ww_nl_indices.size(); ++i)
user_data.relocated_nl_indices_trend_ok &= relocated_ww_nl_indices[i] > relocated_ww_nl_indices[i - 1];
// Check if the indices are in the 'imgui_buf' range.
user_data.relocated_nl_indices_in_range &= relocated_ww_nl_indices.front() >= 0;
user_data.relocated_nl_indices_in_range &= relocated_ww_nl_indices.back() < imgui_buf_size;
// Check if the relocated indices indeed point to a newline.
user_data.relocated_nl_error_count = 0;
for (int index : relocated_ww_nl_indices) {
if ((index >= 0) && (index < imgui_buf_size)) {
if (!IS_NL(imgui_buf[index]))
user_data.relocated_nl_error_count++;
}
}
}
// Copy text from the ImGui buffer to the user-text. First resize to required length.
user_data.user_string.resize(imgui_buf_size - relocated_ww_nl_indices.size());
// Init copying.
int buf_index = 0;
int nl_indices_index = 0;
char* buf_ptr = imgui_buf;
char* buf_end_ptr = imgui_buf + imgui_buf_size;
char* user_ptr = user_data.user_string.data();
int next_ww_nl_index = relocated_ww_nl_indices[0];
// The next block of code becomes easier if we push an invalid newline-index.
relocated_ww_nl_indices.push_back(-1);
// Copy characters except word-wrap newlines.
while (buf_ptr != buf_end_ptr) {
if (buf_index == next_ww_nl_index) {
// Skip the newline.
++buf_ptr;
++nl_indices_index;
next_ww_nl_index = relocated_ww_nl_indices[nl_indices_index];
}
else {
// Copy the character.
*user_ptr++ = *buf_ptr++;
}
++buf_index;
}
}
// Callback which applies:
//
// - string resizing
// - editor word-wrap
int WordWrapAndResizeCallback(ImGuiInputTextCallbackData* data)
{
ProcessingUserData* user_data = static_cast<ProcessingUserData*>(data->UserData);
//----------------------------------------------------------------------
// Check for resizing : ImGuiInputTextFlags_CallbackResize
//----------------------------------------------------------------------
if (data->EventFlag == ImGuiInputTextFlags_CallbackResize) {
// Resize the ImGui string, because that's the actual buffer.
user_data->imgui_string.resize(data->BufTextLen);
data->Buf = user_data->imgui_string.data();
// Done.
return 0;
}
//----------------------------------------------------------------------
// Check for editing : ImGuiInputTextFlags_CallbackEdit
//----------------------------------------------------------------------
if (data->EventFlag == ImGuiInputTextFlags_CallbackEdit) {
// Before modifying the ImGui buffer, we need to know how many word-wrap
// newlines are present before the current cursor.
int leading_nl_pre_mods = 0;
for (int i = 0; i < user_data->ww_nl_indices.size(); ++i) {
if (user_data->ww_nl_indices[i] < data->CursorPos)
leading_nl_pre_mods++;
else
break;
}
// Get a clean string without the word-wrap codes, from ImGui buffer
// into the user-string.
CleanStringFromImGuiToUser(*user_data, data->Buf);
// Word-wrap from the user-string into a temporary string.
std::string word_wrapped;
DoWordWrap(user_data->user_string, word_wrapped, *user_data);
// Determine again how many word-wrap newlines are present before the current cursor.
int leading_nl_post_mods = 0;
for (int i = 0; i < user_data->ww_nl_indices.size(); ++i) {
if (user_data->ww_nl_indices[i] < data->CursorPos)
leading_nl_post_mods++;
else
break;
}
// Calculate the new cursor position. Plus some checks for safety...
int new_cursor_pos = data->CursorPos + leading_nl_post_mods - leading_nl_pre_mods;
new_cursor_pos = std::max(new_cursor_pos, 0);
new_cursor_pos = std::min(new_cursor_pos, static_cast<int>(word_wrapped.size()));
// Copy word-wrapped result back to ImGui internals.
data->DeleteChars(0, data->BufTextLen);
data->InsertChars(0, word_wrapped.c_str());
data->CursorPos = new_cursor_pos;
// Done.
return 0;
}
// We should never get here...
return 0;
}
}
namespace primos::gui {
//-----------------------------------------------------------------------------------------
// Draw the editor.
//-----------------------------------------------------------------------------------------
void DrawWordWrapTextEditor(const char* label, std::string& text, const ImVec2& size)
{
constexpr bool enable_debug_and_timing = false;
// Start performance timing.
primos::engine::Timer timer;
timer.Start();
// Make user-data for callback and word-wrap processing.
float max_line_width = size.x - 2*ImGui::GetStyle().FramePadding.x;
std::string imgui_string;
ProcessingUserData user_data(text, imgui_string, max_line_width, enable_debug_and_timing);
// Remove carriage returns from the user data.
RemoveCarriageReturns(user_data.user_string);
// Do word-wrap, this string is presented to ImGui. Keep an internal word-wrapped copy.
DoWordWrap(user_data.user_string, user_data.imgui_string, user_data);
user_data.imgui_string_copy = user_data.imgui_string;
// Process our string.
ImGui::InputTextMultiline(label,
user_data.imgui_string.data(),
user_data.imgui_string.capacity(),
size,
ImGuiInputTextFlags_CallbackResize | ImGuiInputTextFlags_CallbackEdit,
WordWrapAndResizeCallback,
static_cast<void*>(&user_data));
// Debug.
if (enable_debug_and_timing) {
// If there's an error, we must keep it visible (errors can be gone next edit).
static bool relocated_nl_indices_in_range = true;
static bool relocated_nl_indices_trend_ok = true;
static int relocated_nl_error_count = 0;
relocated_nl_indices_in_range &= user_data.relocated_nl_indices_in_range;
relocated_nl_indices_trend_ok &= user_data.relocated_nl_indices_trend_ok;
relocated_nl_error_count = std::max(relocated_nl_error_count,
user_data.relocated_nl_error_count);
// Timing with 50us resolution to make it better readable (timing reliable
// for a large number of characters). We use an averaging mechanism where
// high timings have a larger weight.
static int time_us_max = 0;
static int time_us_avg = 0;
int this_time_us = static_cast<int>(1e6 * timer.Elapsed());
time_us_max = std::max(time_us_max, this_time_us);
double w = static_cast<double>(this_time_us) / time_us_max;
time_us_avg = static_cast<int>((1 - w) * time_us_avg + w * this_time_us);
this_time_us = 50 * (time_us_avg / 50);
// Show us!
ImGui::SeparatorText("Debug");
ImGui::Text("processing took %d us for %d characters", this_time_us, static_cast<int>(text.size()));
ImGui::Text("relocated_nl_indices_in_range = %d", relocated_nl_indices_in_range);
ImGui::Text("relocated_nl_indices_trend_ok = %d", relocated_nl_indices_trend_ok);
ImGui::Text("relocated_nl_error_count = %d", relocated_nl_error_count);
ImGui::TextUnformatted(text.c_str());
}
}
}
Edit 22 March 2024: the original question resulted from my misunderstanding of the internals of ImGui while trying to get a word-wrapping wrapper around inputtext. I think that original question is not that relevant anymore and that visitors here are more helped with the modified title (was: 'Can you change the buffer pointer in a textinput -edit callback?').
Version/Branch of Dear ImGui:
Version 1.90.0, Branch: ??? (vcpckg install...)
Back-ends:
imgui_impl_glfw.cpp + imgui_impl_opengl3.cpp
Compiler, OS:
MSVC 2022, Windows 11
Full config/build information:
No response
Details:
I'm not sure if I understand this right, spend quite some hours on it already. What I try to achieve (trying to play with some word-wrapping...):
What I understand thus far:
See snippet below. Does this mean that the only solution I have is to allocate (much) more memory than asked for in the resize callback (although I do not know there yet how much I may need), so I (hopefully) do not need a memory re-allocation anymore in the edit-callback?
Screenshots/Video:
No response
Minimal, Complete and Verifiable Example code: