Closed d0vgan closed 1 year ago
but it is the session. https://wiki.freepascal.org/CudaText#Sessions
Session file contains data:
List of named documents (file names), and unnamed documents
For each document:
kind of document: text in editor, picture file, text in viewer (and viewer mode: text/binary/hex/unicode)
read-only state
first caret position
encoding
word-wrap mode
lexer
bookmarks
index of top visible line
tab size and "tab as spaces" state
minimap and micromap visible state
ruler visible state
non-printable characters visible state
line numbers visible state
scale factor
list of folded ranges (if lexer supports folding)
color of ui-tab (if not default)
tab title (if not default)
modified state
code-tree filter string and history of last filters
splitting to 2 editors: on/off, vertical/horizontal, percents of size
For each modified document: date of modification, full document text
Layout/sizes of side panel and bottom panel
Layout/sizes of editor groups
Index of active editor group and active tab in each group
so even caret moving (or UI splitter moving) makes new info in the session.
When CudaText is a foreground window and the caret is in its Console window (so there is no chance even to change a caret position in any editing file), the 'history session.json' is still auto-saved. Even when CudaText is in background (i.e. its window in not active, so obviously nothing is changed in CudaText), the 'history session.json' is still auto-saved. Well, let me re-phrase the requirements to save the 'history session.json':
ActualSessionData
<>PreviouslySavedSessionData
, the ActualSessionData
should be saved to 'history session.json' and right after it PreviouslySavedSessionData
:=ActualSessionData
;ActualSessionData
should be saved to 'history session.json' and right after it PreviouslySavedSessionData
:=ActualSessionData
.When
ActualSessionData
<>PreviouslySavedSessionData
This comparison should use some additional technique to be efficient.
For example, let's consider a situation when we have an unsaved tab (document) with 100 MB of text. It is obviously ineffective to compare the 100 MB of data each time we are going to save the 'history session.json' - as well as it is ineffective to save the very same 100 MB of data to the 'history session.json' each 30 seconds. Instead, it is enough to have an additional member for this 100 MB of data: isSavedToSession: Boolean
. And the condition of ActualSessionData
<>PreviouslySavedSessionData
will firstly check simple things such as caret position, encoding, word wrap mode etc. - and if something of those was changed, then the 'history session.json' should be saved. If nothing of those was changed, then it's time to check the isSavedToSession: Boolean
- and if it true
for all the tabs (documents) then the 'history session.json' should not be saved. Otherwise the 'history session.json' should be saved and all the isSavedToSession: Boolean
should be set to true
once the session file has been successfully saved.
yes, something like you suggest (remember last state, calculate new state, compare them) can be done. but will be very slow on big files when changed file text is saved to session. i will read your 2nd post more carefully
This comparison should use some additional technique to be efficient.
An idea in case we implement this. We can use hash to quickly check if file content has been changed, just like how Git detect file changes.
This comparison should use some additional technique to be efficient.
An idea in case we implement this. We can use hash to quickly check if file content has been changed, just like how Git detect file changes.
I assume the isSavedToSession
will work in pair with the existing isDirty
flag. The isDirty
flag refers to the already existing state that shows whether a document contains any changes or not, and thus whether it needs to be saved or not. I'm not familiar with CudaText's source code, so isDirty
may have a different name such as isModified
or the opposite isSaved
.
The new isSavedToSession
flag adds another level to this, since a document can be "dirty" ("modified") but already saved to the session. In such case, the document is shown as "dirty" (so isDirty
is true
), while its isSavedToSession
is true
.
This approach does not require any comparison of document's content while saving a session file - instead, the value of isSavedToSession
should be set to false
every time when new changes are made to the document (I believe at exactly the same moments when the existing isDirty
flag is set to true
).
Let's look at a document lifecycle to explain how isDirty
and isSavedToSession
are expected to work in pair.
isDirty
is set to true
(it is modified and not saved) and isSavedToSession
is false
(it has not been saved to session yet).isSavedToSession
flas is set to true
. (Note: the document's isDirty
flag remains true
since user did not explicitly save the document via the Save command).isSavedToSession
is true
, there is no need to update the session file - so the session file is not actually updated. (Note: the document's isDirty
flag is still true
since user did not explicitly save the document via the Save command).isSavedToSession
flag is set to false
to reflect the fact that there are new changes that were not saved to the session file yet.isSavedToSession
flag is false
. Once the session file has been successfully saved, the document's isSavedToSession
flag is set to true
. (Note: the document's isDirty
flag still remains true
since user did not explicitly save the document via the Save command).isDirty
is set to false
and isSavedToSession
is set to true
, whatever previous values they had.Let me additionally illustrate this by the table:
-----------------------------------------------------------------------------
| isDirty | isSavedToSession | Comments |
|-----------------------------------------------------------------------------|
| false | true | document is not "dirty" and is saved to |
| | | session - no need to update the session file |
|---------|------------------|------------------------------------------------|
| true | true | document is "dirty" and is saved to session |
| | | - no need to update the session file |
|---------|------------------|------------------------------------------------|
| true | false | document is "dirty" and is not saved to |
| | | session - the session file must be updated |
|---------|------------------|------------------------------------------------|
| false | false | document is not "dirty" and is not saved to |
| | | session - no need to update the session file |
| | | (probably it is a document that was opened in |
| | | the editor and was not modified since then); |
| | | though it may be a sign that the session file |
| | | was removed or changed outside of the editor - |
| | | in such case the session file must be updated |
-----------------------------------------------------------------------------
thanks for the detailed info. but I feel it is too much work for me. I don't have the power for it. maybe someone can add the patch (it must be not big so I am not lost).
Maybe I could look into it, but I need addition info:
Which functions/classes are responsible for saving the session file?
Does it make sense to implement isSavedToSession
as part of the document
class - or, instead, the concepts of documents
and session
should be separated and, thus, a document
should notify a session
that the document
has been modified - and the session
should set its own isSavedToSession
property associated with the document
? Let me explain this by pseudo-code:
Option 1:
class Document
property isDirty: Boolean;
property isSavedToSession: Boolean;
// when `isDirty` is set to `true`, `isSavedToSession` is set to `false`
Option 2:
class Document
property isDirty: Boolean;
class Session
documentsState: dictionary of (DocumentName, isSavedToSession);
procedure notifyDocumentIsDirty(documentName)
begin
documentsState[documentName] := false; // setting `isSavedToSession` to `false`
end
// when document.isDirty is set to `true`, the document calls `session.notifyDocumentIsDirty(documentName)`
What is the pascal way of checking the file last write time (last modification time) and of checking if a file exist? This is needed to ensure that the session file physically exists and that its last write time is the same as during the last saving of the session - because if it is not, the session must be saved for sure.
Maybe I forgot to ask something else related to the topic?
I made the reply re-sess.txt
How can a notification from TATSynEdit
reach the TEditorFrame
?
Here is what I mean:
TATSynEdit
has a property Modified
which (I assume) is set when any change is made to the document's text.SetModified
sends a notification DoEventChangeModified
only when the value of AValue
is different than the previous one.isSavedToSession
relies on the value of Modified
being set to true
disregarding its previous value, so TEditorFrame
should receive a notification each time when Modified
is set to true
, even when the previous value of Modified
was already true
.Let me explain the flow:
TATSynEdit.SetModified
is called with the value of true
. At this time, TATSynEdit
should notify its corresponding TEditorFrame
that this value is true
. The TEditorFrame
sets its isSavedToSession
property to false
.TEditorFrame
has its isSavedToSession
equal to false
, the session file is updated and the isSavedToSession
is set to true
.Modified
was true
before these latest changes. Now, because of the new changes, TATSynEdit.SetModified
is called again with the value of true
. And again, the TATSynEdit
should notify its corresponding TEditorFrame
in order to set the TEditorFrame.isSavedToSession
to false
. Otherwise (without this notification) it would not be possible to properly update the isSavedToSession
property.So my question is: is there an existing notification mechanism for this?
Does TEditorFrame
subscribe to changes in the TATSynEdit
? Or maybe TEditorFrame
registers some callback to be called by TATSynEdit
?
Or, alternatively, and simpler, we can introduce a new property to TATSynEdit
such as hasNewModifications
- to be able to distinguish between just "modified in general" and "modified since the last saving to the session". With this approach, no notification mechanism is required since the TEditorFrame
can just check the value of TATSynEdit.hasNewModifications
before saving the session and then set this value to false
right after the session has been saved.
Why I propose this as a second option because it adds an additional property hasNewModifications
to TATSynEdit
, thus changing its interface. If, however, TATSynEdit
does not have an existing notification mechanism to notify the TEditorFrame
, then this option is preferable.
formframe.pas. here is how EditorFrame subscribes to ATSynEdit events
ed.OnClick:= @EditorOnClick;
ed.OnClickLink:=@EditorOnClickLink;
ed.OnClickDouble:= @EditorOnClickDouble;
ed.OnClickMoveCaret:= @EditorClickMoveCaret;
ed.OnClickEndSelect:= @EditorClickEndSelect;
ed.OnClickGap:= @EditorOnClickGap;
ed.OnClickMicromap:= @EditorOnClickMicroMap;
ed.OnPaint:= @EditorOnPaint;
ed.OnEnter:= @EditorOnEnter;
ed.OnChange:= @EditorOnChange;
ed.OnChangeModified:= @EditorOnChangeModified;
ed.OnChangeCaretPos:= @EditorOnChangeCaretPos;
ed.OnChangeState:= @EditorOnChangeState;
ed.OnChangeZoom:= @EditorOnChangeZoom;
ed.OnChangeBookmarks:= @EditorOnChangeBookmarks;
ed.OnContextPopup:= @EditorContextPopup;
ed.OnCommand:= @EditorOnCommand;
ed.OnCommandAfter:= @EditorOnCommandAfter;
ed.OnClickGutter:= @EditorOnClickGutter;
ed.OnCalcBookmarkColor:= @EditorOnCalcBookmarkColor;
ed.OnDrawBookmarkIcon:= @EditorOnDrawBookmarkIcon;
ed.OnDrawLine:= @EditorOnDrawLine;
ed.OnDrawRuler:= @EditorOnDrawRuler;
ed.OnKeyDown:= @EditorOnKeyDown;
ed.OnKeyUp:= @EditorOnKeyUp;
ed.OnDrawMicromap:= @EditorDrawMicromap;
ed.OnPaste:=@EditorOnPaste;
ed.OnScroll:=@EditorOnScroll;
ed.OnHotspotEnter:=@EditorOnHotspotEnter;
ed.OnHotspotExit:=@EditorOnHotspotExit;
ed.ScrollbarVert.OnOwnerDraw:= @EditorDrawScrollbarVert;
you need ed.OnChange. So open its handler in formframe.pas.
procedure EditorOnChange(Sender: TObject);
does it help?
and, you have this helpful property:
EditorFrame1.Ed.Strings.ModifiedVersion
(integer - 'version' of changes)
As far as I can see, the whole change will require 2 phases:
Session
. Correspondingly, all the functions that currently write the session properties directly to TJsonConfig
, will set the fields of the Session
class instead, similarly to the Builder pattern. The Session
class will be responsible for producing the output JSON.Session
instances to decide whether to save the session file or not.The Phase 1 will include most of the changes (for example, the function TEditorFrame.DoSaveHistoryEx
will be completely rewritten to deal with the Session
class, and then the Session
class will produce the output JSON). The Phase 1 will take some time and will not introduce any immediate benefit. The actual benefit will be visible in the Phase 2 that will complete everything basing on the Session
class.
sounds good. Maybe I can help with the session class phase-1. but not in the near 2-4 days.
I've prepared a very draft version of the Session
class - it's actually empty now, but it already demonstrates the idea that a Session
instance will include a list of TSessionFrameData
items, where each TSessionFrameData
will contain a list of TSessionValueType
properties.
Why I ended up with an idea of generic lists is to have an automatic process of serialization and deserialization (which would not be possible in case of predefined properties) plus it abstracts the idea of property
from a TSessionFrameData
class since each property is a responsibility of an external user of the TSessionFrameData
class and not of this class itself.
In terms of future plans of JSON serialization, I'm considering to do it according to the SOLID principles - i.e. to provide a serialization interface first, and the serialization to JSON will be just one of possible implementations of it. Following this approach, later we may be able to implement another serializer, e.g. to .ini file or to a database, if needed. Also, a custom serializer might be able to e.g. save the content of unsaved document(s) to separate file(s) rather than to one history file. But all of this are only future plans so far.
session_initial_draft.zip
small note to your code: we need "const string" params if params are not changed in func.
function f(const s: string): nnnnn;
I am still reading it.
vars in "private" part should start with a "F" letter.
private
FName: UnicodeString;
FValType: TSessionValueType;
FStrValue: UnicodeString;
FIntValue: integer;
I've read it. OK code. let's not change Cud's code (for subj) in the near 2-4 days.
style note. Params of funcs should start with 'A':
constructor Create(const AName: UnicodeString; const AValue: UnicodeString);
(added 'const' too)
Here is the next draft. Now the goal looks more clear, but still a lot of to do.
I should confess I don't have a clear vision of the interface and implementation of the SessionLoader for the moment, so need to gather information in this direction.
Also I'm not sure about memory management in freepascal. For example, TFramesList = specialize TFPGList<TSessionFrameData>
is a list of objects that represent class instances. In such situation, should I manually destroy each object in the list before the list itself is destroyed, or does the list take care of these objects itself? I mean, class instances are not value types, they are like references or pointers, right?
In such situation, should I manually destroy each object in the list before the list itself is destroyed
yes, because FPGList does not do it.
BTW, is it ok if we won't change the code of JSON reading/writing, which is like
if (cfg.GetValue(sRootPath+'/000/group', -1)=-1) and
(cfg.GetValue(sRootPath+'/001/group', -1)=-1) then
but we instead change the nature of cfg
variable?
CompareTo
to IsEqualTo
or EqualsTo
about Enumerator/Iterator class and its .Current
and .MoveNext
methods. usually pascalists don't read Enumerator class and don't use Current method DIRECTLY. they do it INDIRECTLY, ie using 'for-in' loop (new pascal feature). like
for ItemCurrent in SomeClass do
begin
//do smth with ItemCurrent
end;
Looks like the enumerators are still needed when I'm iterating through two list in the same loop.
BTW, is it ok if we won't change the code of JSON reading/writing, but we instead change the nature of cfg variable?
Yes, this is an interesting idea! The TSession
class may have the very same public methods as the ones used by TJsonConfig
during loading and saving. This will allow to keep most of the code unchanged, with the TJsonConfig
instance replaced with the TSession
instance.
In such case, TSession
will store a list of TSessionValue
objects, without any TSessionFrameData
. Moreover, it also solves the question of ISessionLoader
interface and implementation. For each value, the ISessionLoader
will first read it from the session data and then TSession
will store the just read value in a form of TSessionValue
.
The existing method TEditorFrame.DoSaveHistoryEx
contains several calls of c.SetDeleteValue
and c.DeleteValue
. It is not clear to me: we are in the process of filling a newly created JSON object with data, so how can it be that we need to delete something from it?
For our new code, SetDeleteValue can work like SetValue. DeleteValue must do nothing.
I'm looking at the method TEditorFrame.DoSaveHistory
and it begins with
cfg:= TJsonConfig.Create(nil);
that creates an empty object, right?
Right after it, there are operations that try to read items: TStringlist
from the cfg
(which is empty, right?) and then to delete the values from items
from the cfg
(which was already empty?)... Well, am I right that this code can be completely removed?
no no. Before reading/writing, we have line
cfg.Filename:= AppFile_HistoryFiles;
which assigns the JSON filename. reads the file.
cfg.Filename:= AppFile_HistoryFiles;
which assigns the JSON filename. reads the file
When CudaText was starting, wasn't all the data read from the history file? I mean, if all the data has been read on start, then all the data is already present in the memory at the moment of saving the history, so why the history file is re-read rather than created from scratch from the memory?
Probably this part needs to be changed as the very first step of all of these modification. I mean, saving to JSON should be exactly saving - i.e. dumping the memory into a new JSON file created from scratch. Any reading from the file does not look correct at the moment of saving the file.
maybe. but not sure it will be OK for N app instances. (with option "ui_one_instance":false).
with 3 instances, each closing of file tab must correctly update JSON file only for the closing-tab.
also for N instances: if we open 1st and 2nd instances, and history-file has NO info about file1.txt, we can open file1.txt from 2nd instance, work on it, close it. then we can open file1.txt from 1st instance, work on it, close it. it will go OK now. but not OK after your idea.
This complicates things. So actually ISessionSaver
should also be ISessionLoader
(otherwise ISessionSaver
will not be able to load the session file), and also e.g. ISession.SetDeleteValue
must delegate this call to ISessionSaver
(as ISessionSaver
can contain some loaded data).
Maybe you're right... I don't have the understanding of full idea
Well, the whole idea of TSession
was to have the single responsibility of all session's values in order to compare the previous session to the current session. As the history JSON file is loaded each time a session is about to be saved, it makes TSession
pointless since it has no idea of what's inside the JSON file (because this file can be overwritten by another instance of CudaText). I might propose dozens of props and crutches for TSession
such as checking the file's last modification time, getting values from JSON etc. - but this is exactly what broken and corrupted systems usually do: they spend most of their resources just to support the system itself, for the purpose of the system's existence, without doing anything useful actually. This is not our way. So we have to refuse the whole idea of TSession
and TSessionSaver
.
My bad. I should have done more careful analysis and investigation before any prototyping.
Let's do everything in a different way.
I've checked the behavior of e.g. Frame.Ed1.Strings.ModifiedVersion
(within TfmMain.DoOps_SaveSession) - and this is exactly what is needed to tell whether a document contains new modifications or not. So its enough to save the current values of each Frame.Ed1.Strings.ModifiedVersion
and Frame.Ed2.Strings.ModifiedVersion
while saving the current session in order to understand whether any new changes are present at the moment of saving the next session. As simple as that.
Now, coming to other session values, all of them are stored inside the TJSONConfig object. So, if all the values of ModifiedVersion
are the same as during the previous saving of the session, now it's time to compare the rest of the TJSONConfig object, excluding all the "text"
values since we have the ModifiedVersion
to deal with the "text"
values.
Looking into TJSONConfig, it contains FJSON: TJSONObject
that actually stores all the values of the session. So, we have two options to compare the values:
1) Implement a custom serializer for TJSONObject
that will produce an output JSON string that contains everything except those values that have been passed as exclusions, e.g.:
function TJSONData.FormatCustomJSON(Options: TFormatOptions; Indentsize: Integer, const excludingTheseValues: TStringList): TJSONStringType;
In such way, TJSONData
will produce a JSON string that contains everything except the "text"
values, and we'll just compare the current JSON string against the previous JSON string.
2) Implement a custom comparer for TJSONData
that will compare one TJSONData
object against another TJSONData
object, ignoring the values specified as exclusions. E.g.
function TJSONData.CustomCompare(AJsonData: TJSONData; const excludingTheseValues: TStringList): integer;
In any case, I'll need your help to understand the internals of TJSONData
in order to be able either to collect all its values in a custom way or to compare all its values in a custom way.
I'll need your help to understand the internals of TJSONData in order to be able either to collect all its values in a custom way or to compare all its values in a custom way.
Even if I don't know TJSONData, I will try to help here. but can you instead ask about TJSONData in the FPC forum where authors also present? https://forum.lazarus.freepascal.org/index.php?board=62.0
To be able to set the Modified
property of TJSONConfig
to false
(to avoid saving of the JSON), the following approach seems to work:
TJSONConfig2 = class(TJSONConfig)
public
procedure SetModified(AValue: Boolean);
function GetJsonObj: TJSONObject;
end;
procedure TJSONConfig2.SetModified(AValue: Boolean);
begin
FModified := AValue;
end;
function TJSONConfig2.GetJsonObj: TJSONObject;
begin
Result := FJSON;
end;
Correspondingly, TJSONConfig2
should be used instead of TJSONConfig
to be able to call the .SetModified
.
As for iterating the items in the TJSONObject
, I believe the following should allow to do this:
function TJSONObject.GetEnumerator: TBaseJSONEnumerator;
Probably this allows the constructions of for obj in cfg.GetJsonObj
to work, where obj: TJSONEnum
. And then obj.Value.JSONType
should allow to understand the type of the object...
My understanding of this is not currently enough to propose a good way of comparison of two objects returned by GetJsonObj
. Maybe freepascal has some built-in abilities to compare custom objects or maybe the base class TJSONData
or its inheritred classes already implement some comparison logic, but I don't see it because of lack of understanding of the language principles?
Maybe freepascal has some built-in abilities to compare custom objects
of course no.
or maybe the base class TJSONData or its inheritred classes already implement some comparison logic
It is good to ask this in the FPC support forum, I don't know fpJSON library good.
looked at the code of fpJSON: I did not find the Equ
or Compare
or operator =
in code, so - these objects cannot be compared yet.
I've created a draft that implements a custom comparsion and custom cloning of the JSON data: https://github.com/d0vgan/CudaText/tree/feauture/save-changed-session
As usual, comments are welcome :) It also contains these TODO items:
FPrevJsonObj: TJSONData; // TODO: where it should be created and destroyed?
// TODO: add a list (or array?) to store the values of Frame.Ed1.Strings.ModifiedVersion and Frame.Ed2.Strings.ModifiedVersion
Regarding TJSONData
, the question is: where to create and to destroy this object?
Regarding Frame.Ed1(2).Strings.ModifiedVersion, the question is: what is the best way to store this pair of values? As a record in an array? As a record in a list? As two arrays or lists? (Please point me to or provide the corresponding example).
As a record in an array? As a record in a list? As two arrays or lists?
if you know exact count of items (in array/list)- array of records, is best. you call
var
ar: array of TSomeRec;
begin
SetLength(ar, NeedCount);
ar[0]:= ...;
....
ar:= nil; //Free memory of ar
2 lists is worse that 1 list! 1 list must use FGL unit, generic TFPGList from it: TFPGList<TMyRecordWith2Numbers>
.
Cud's src has several places with TFPGList - here are examples.
where to create and to destroy this object?
you make the field in TfmMain. so, you allocate it
a) just on 1st use: if FPrevJsonObj=nil then FPrevJsonObj:= Tnnnnnnn.Create.......
OR
b) in TfmMain.FormCreate (handler of OnCreate event). deallocate: in the TfmMain.FormDestroy (handler of OnDestroy).
if you know exact count of items (in array/list)- array of records,
The numbers of items in the array should be the same as the number of the current edit frames.
If I call SetLength
for the very same dynamic array repeatedly, again and again, will the memory be allocated/deallocated correctly, without memory leaks? For example, is it OK to call e.g. SetLength(arr, 10)
, then SetLength(arr, 8)
, then SetLength(arr, 12)
, then SetLength(arr, 5)
, then SetLength(arr, 7)
and so on and so forth? Or maybe, I don't know, maybe I should e.g. call SetLength(arr, 0)
before re-alloctaing?
As for FPrevJsonObj=nil
, it is not explicitly initialized to nil
anywhere, so I assumed I should explicitly intialize it first? Or is automatically initialized to nil
?
for the very same dynamic array repeatedly, again and again, will the memory be allocated/deallocated correctly,
yes! if the record has "managed' fields (string/ pointer/ class), you need to free this field by hand before realloc.
As for FPrevJsonObj=nil, it is not explicitly initialized to nil anywhere, so I assumed I should explicitly intialize it first? Or is automatically initialized to nil?
https://wiki.freepascal.org/Pointer#How_Pointer_variables_are_initialized
it is class field? then it is Nil initially
https://wiki.freepascal.org/Pointer#How_Pointer_variables_are_initialized it is class field? then it is Nil initially
This differs from C++ :)
And if we speak about C++, Python and Pascal, I can't understand why all the languages (including Pascal) are agree that function parameters are passed using comma func(a, b)
, but Pascal uses semicolon in the function declaration/definition function func(a: T; b: T): T;
whereas the other languages use the very same comma: T func(T a, T b)
, def func(a, b)
. I write commas automatically - and the Pascal compiler gives errors for that :(
Anyway, looks like I'm pretty close to implement the save-only-updated-session feature.
However, here is this code at the beginning of the TfmMain.DoOps_SaveSession
that actually made me lose my test session file:
if sRootPath='' then
begin
AppDiskCheckFreeSpace(sSessionFileName);
DeleteFile(sSessionFileName);
end;
I mean, if only the updated session is saved, the code above deletes the previous physical file, and the new file is not physically saved because the session data has not been changed since the previous saving. What can you suggest to solve this issue?
I suggest to comment out this small block.
Pls, name the JSONEx independent unit as proc_json_ex. we have all such names for units.
Currently 'history session.json' is always auto-saved even when there were no changes to any file opened in CudaText. To me, there are just 2 reasons to save "history session.json":
Otherwise it looks like the very same content of "history session.json" is auto-saved again and again.