Compilation from sources instructions do not work.

[the-Arioch]

Can not be compiled from source.

When trying step 7 of https://castle-engine.io/compiling_from_source.php

Looks trivial, but who knows.

Also, probably unrelated, when i click on the striken-out files and then say "no" to new file creation prompt - the IDE gives me a totalyl undecipherable AV window.

Also, why compilation from sources manual contain those "copy exe file" instructions? Won't it be better to setuup the project just to output release exe there? Maybe i am saying something stupid, as i am coming from Delphi background, but that is what i'd be doing in Delphi as default option.

[michaliskambi]

I do not see an error message on your screenshot. I understand that it was the same problem as in https://github.com/castle-engine/castle-view-image/issues/2 , lack of castle_window.lpk? If yes, this is solved. Otherwise please post an error message you see. For me (and for Jenkins :) ) view3dscene compiles OK.

I confirm that answering "No" to the creation prompt on non-existing file in view3dscene project results in Access Violation in Lazarus. I confirmed it using Linux/x86_64, FPC 3.2.2, Lazarus 2.012. Can you submit it to the Lazarus bugtracker, https://gitlab.com/groups/freepascal.org/lazarus/-/issues ? This can most likely be reproduced with an empty project (add there a file, then delete the file using file manager), and is unrelated to CGE.

In the meantime I also removed the missing files from view3dscene LPI, and added proper includes from embedded_data. It wasn't a problem for compilation in all my tests, but still we want it to be clean, so that Lazarus is properly aware of project changes.

Also, why compilation from sources manual contain those "copy exe file" instructions? Won't it be better to setuup the project just to output release exe there? Maybe i am saying something stupid, as i am coming from Delphi background, but that is what i'd be doing in Delphi as default option.

We don't compile immediately to bin directory mostly for historical purposes. We always placed our tools EXE into each tool's respective directory, and then developer or script can copy it to bin (or just place on $PATH). You are right, we could simplify it and just point the compiler to place the EXE in bin... but at this point, I am afraid it could break some developer workflows (it would break mine, as I just place build tool on $PATH).

So, this is a good point to consider to simplify it in the future, but I don't want to do it now. I would likely be connected with moving view3dscene + castle-view-image into the same CGE repository, into the tools/castle-view-scene/ and tools/castle-view-image -- something I was already planning.

Note: binary downloads on https://castle-engine.io/ just include all the tools already in bin subdirectory, so it is likely not an issue for most users, at least I didn't hear reports about it causing trouble.

[the-Arioch]

Hello, thanks for quick reaction, sadly could not say the same about myself.

I confirm that answering "No" to the creation prompt on non-existing file in view3dscene project results in Access Violation in Lazarus. ... Can you submit it to the Lazarus bugtracker, https://gitlab.com/groups/freepascal.org/lazarus/-/issues ?

I am not versed well in Lazarus infrastructure (like normal sets of project files and their format), so making minimized proof would be some hassle to me. Can you?

I do not see an error message on your screenshot

I consider red striking-out already an error - as that is what red color tells to user :-)

I do not remember if i had actual compilation issues or not. Probably i did, as the first line suggests, but the issue was trivial to fix, so i do not remember.

binary downloads on https://castle-engine.io/ just include

I know, but i wanted to be on the edge. Also because i would not be doing a game app, but rather a desktop-app with game-like features. So the development cycle tightly optimized to gamedev would perhaps in the long run be worse, than Laz-centered cycle borrowing some code and features from Castle.

moving view3dscene + castle-view-image into the same CGE repository

Merging repositories... Yeah, doable, for example exporting them as a patchset, then importing the patchset in a feature-branch. But would be a pa... a thorn in your side. My sympathies :-)

[the-Arioch]

One more thought, can it be worth to have a local in-repo copy of "compilation from sources", even if still HTML format but maybe simplified to markdown or something? to have a slim and offline reference. Browsers tend to be heavy today, and default browsers would open many windows, load extensions, etc...

JediVCL and JediCodeLib libraries just have update batch and shell scripts (those though jsut prepare environment then build run custom installer program). In idiomatic Delphi way i would make a "project group" where i would include all the projects (apps and packages) in the build order. So all the user would have to do after git pull would be running "make all" in IDE.

Probably it is not so simple with Laz, dunno if it has project groups, but it certainly needs IDE recompilation steps, that would break Compile All actions anyway :-(

[michaliskambi]

I confirm that answering "No" to the creation prompt on non-existing file in view3dscene project results in Access Violation in Lazarus. ... Can you submit it to the Lazarus bugtracker, https://gitlab.com/groups/freepascal.org/lazarus/-/issues ?

I am not versed well in Lazarus infrastructure (like normal sets of project files and their format), so making minimized proof would be some hassle to me. Can you?

OK, will do.

I know, but i wanted to be on the edge. Also because i would not be doing a game app, but rather a desktop-app with game-like features. So the development cycle tightly optimized to gamedev would perhaps in the long run be worse, than Laz-centered cycle borrowing some code and features from Castle.

Note that the version available on our main page https://castle-engine.io/ is right now always a "snapshot", automatically recompiled after each CGE commit. This is what we advise to use, for both games and non-games.

So if you just use "sources approach" to be on the "bleeding edge", you can as well switch to using binary releases. They are "bleeding edge" too... right now :) We test CGE master version extensively, and Jenkins runs a lot of automatic tests before it makes a new snapshot. So in practice they are really reliable.

Note that soon we'll release 7.0-alpha2. So you will also have a "fixed" version to use, that is fairly recent. Last version 7.0-alpha1 was released on February 2021 (see https://github.com/castle-engine/castle-engine/releases ) and it is a bit old in effect -- as we've done a lot of new features.

Merging repositories... Yeah, doable, for example exporting them as a patchset, then importing the patchset in a feature-branch. But would be a pa... a thorn in your side. My sympathies :-)

It is possible to just merge one GIT repository into another, by adding a 2nd remote in GIT and merging. So I know I can import view3dscene into CGE repo without losing any history.

It's not done yet because it will unevitably need some further changes to the build process, and it is not that high-priority at this point :) But it will be done at some point.

One more thought, can it be worth to have a local in-repo copy of "compilation from sources", even if still HTML format but maybe simplified to markdown or something? to have a slim and offline reference. Browsers tend to be heavy today, and default browsers would open many windows, load extensions, etc...

The "compilation from sources" page source is already in another repo, https://github.com/castle-engine/cge-www/ . Though it's in in PHP now, https://github.com/castle-engine/cge-www/blob/master/htdocs/compiling_from_source.php .

Just as all CGE web pages, it will be remade to AsciiDoctor at one point soon, following my plans on https://castle-engine.io/wp/2021/12/31/many-documentation-upgrades-using-asciidoctor-as-our-primary-way-to-write-documentation-michalis-thoughts-and-plans-about-our-documentation/ . So it will be simpler (AsciiDoctor is a simple markup, and GitHub has preview for it AsciiDoctor).

But I will probably still not want to just copy it to CGE repo, to avoid duplication.

Probably it is not so simple with Laz, dunno if it has project groups, but it certainly needs IDE recompilation steps, that would break Compile All actions anyway :-(

It is possible to automate the process further using shell/Makefile. We already automate a lot in Makefile in CGE repo. To just recompile build tool and editor, you can do

mkdir -p bin/

castle-engine/tools/build-tool/castle-engine_compile.sh
mv castle-engine/tools/build-tool/castle-engine[.exe] bin/

lazbuild castle-engine/tools/castle-editor.lpi
mv castle-engine/tools/castle-editor/castle-engine[.exe] bin/

This would be even simpler if we just point the programs to put their output in CGE bin/ already. As we talked above -- this is probably a good idea, but I'm not keen on doing it now and breaking other compilation workflows people use for development.

[the-Arioch]

by adding a 2nd remote in GIT and merging

But it would have different commits tree root!!! This should make 3-way diff/merge impossible and it is 3-way diff that is the founding block of 3-rd gen (serverless, distributed) VCSes.

I'd have to read into it, but this seems qute a heresy to me.

---

Does AsciiDoctor provide for screenshots or other bitmaps though? HTML does, basic markdown seems not...

just copy it to CGE repo, to avoid duplication

Makes sense, especially id screenshots would be merged into the text as data:// URIs. But personalyl i'd prefer reading them out of browser. Granted, that is my itch, not necessarily your one.

you can do

....in Windows? well, i can perhaps ustilize Git bash to do it, but i am not sure it would work smoothly, it is not the primary designed pipeline i guess :-)

if we just point the programs to put their output in CGE bin/ already.

Good you mentioned it, so i can go on with offtopic :-P

There are paths to FPC and Laz in the GCE's preferences dialog. Perhaps there can be just a similar tabsheet for paths to Castle's utilities, at least 2D and 3D viewers and maybe castle-engine too?

Also, is it hard in Linux and other platforms to intercept castle-engine output and render it in the GUI (TMemo, etc) ? Because seeing text window disappearing leaves me wondering if some warning or error could be left unread. In my unlucky failure to create new game project - this was bothering me. And also that temporary black window just does not look slick :-)

Frankly, maybe it could be refactored into some DLL/Lib-SO so that castle-engine.exe would be a thin dumb launcher for it and GCE would be yet another client app ?

[michaliskambi]

But it would have different commits tree root!!! This should make 3-way diff/merge impossible and it is 3-way diff that is the founding block of 3-rd gen (serverless, distributed) VCSes.

I used this approach a few times by now, in open-source and closed-source projects :) The resulting repo is effectively a sum of 2 branches from 2 repos, and contains files from both (and one can rearrange the files in next commits freely). It doesn't seem to be a problem for any tool, including for all GitHub visualizations.

Does AsciiDoctor provide for screenshots or other bitmaps though? HTML does, basic markdown seems not...

AsciiDoctor has macros to include images. And we actually use special macros in CGE to include images in our way. See cgeimg documented on https://github.com/castle-engine/cge-www . I am going to make a news post about it soon, it's a new thing :) It makes AsciiDoctor fully-featured for writing CGE docs.

There are paths to FPC and Laz in the GCE's preferences dialog. Perhaps there can be just a similar tabsheet for paths to Castle's utilities, at least 2D and 3D viewers and maybe castle-engine too?

That seems too much complication. We search for utilities in CGE bin subdirectory, we search for them in $PATH too. This allows a lot of setups. I do not want to complicate it further, by specialized config for each tool.

Also, is it hard in Linux and other platforms to intercept castle-engine output and render it in the GUI (TMemo, etc) ? Because seeing text window disappearing leaves me wondering if some warning or error could be left unread. In my unlucky failure to create new game project - this was bothering me. And also that temporary black window just does not look slick :-)

It's easy to intercept output, on any platform. Our "Output" window already intercepts output of our build tool, which in turn intercepts the output of FPC, Delphi...

You just encountered an issue with the fact that we don't yet intercept the output of one call we do when creating the project. That's because we don't have "Output" window ready yet. But it should probably be intercepted.

So, you just encountered a particular place when we don't intercept the output. But in general we do intercept and it is easy :)

Frankly, maybe it could be refactored into some DLL/Lib-SO so that castle-engine.exe would be a thin dumb launcher for it and GCE would be yet another client app ?

Our process running/capturing routines are already in common units, in tools/common-code. The ToolCommonUtils unit contains process running/capturing It is used by both build tool and editor.

[the-Arioch]

It's easy to intercept output, on any platform.

Meanwhile i reached my remote box and checked what i used in Delphi app to do it -0 and it was FPC's TProcess cut out and polished for Delphi :-) So, assuming that TProcess is fully implemented for all the platforms it should really be easy.

That's because we don't have "Output" window ready yet

However, TProcess is working with streams not GUI objects... As long as there is receiving buffer - it should not matter if GUI sink is created already or not yet...

But it should probably be intercepted. So, you just encountered a particular place when we don't intercept the output

Does it need a ticket?

[michaliskambi]

However, TProcess is working with streams not GUI objects... As long as there is receiving buffer - it should not matter if GUI sink is created already or not yet...

Our routines in ToolCommonUtils (based on TProcess) indeed don't need GUI of course. Some routines in editor (TAsynchronousProcess) put the output in GUI "Output" window, as they assume we run commands within a generated project.

It's not a technical problem to solve (surely we can intercept command output at any point), it's more a "where do we want to show it, to not have confusing UI" question. I don't want to show output from castle-engine generate-program in the initial "Output" window of a new project, as it would likely be confusing. Ideally, users should not even need to be aware that when creating new project, we run another command-line CGE tool.

But my problem above is easy to solve, of course. We can just store them in a log of the editor.

In any case, intercepting is not really the solution to the problem you see with blinking window on Windows. That's because we should run the process on Windows with certain flags to prevent it from displaying the console (otherwise Windows creates a new terminal, regardless if we intercept the output or not). I see that I just missed rcNoConsole flag when appropriate ... hopefully fixed now (though I'm on Linux now, didn't test on Windows).

Does it need a ticket?

Please test on Windows the current version,from CGE master branch. If the problem persists (you see a window blink, when creating new project) then yes, please create a new ticket in CGE repo.

[the-Arioch]

I don't want to show output from castle-engine generate-program in the initial "Output" window of a new project, as it would likely be confusing

But then you risk missing error/warning messages, or at least user might fear it. That is why i advocated above about ditching "unix way" and converting main functionality into a shared library, with explicit ways to signal errors (error codes, or exceptions, or ok/err records as result values)

I'll test GCE after i make damn laz 2.2.0 run :-)

[the-Arioch]

And also, when you intercept program's text output you would probably need to convert charset on Windows, as there different codepages are used for text and GUI programs (not for all locales, but at least for Russian and Greek). UNIX or Western guys always forget it, and UNIX-and-Western forget it twice :-D

[michaliskambi]

That is why i advocated above about ditching "unix way" and converting main functionality into a shared library, with explicit ways to signal errors (error codes, or exceptions, or ok/err records as result values)

Well we still watch at error code of the process. And this is exceptional (this one process call before the project is created) that we run the process without repeating to user all the process output.

Still, point taken that instead of calling the build tool, we should just put it in a shared code (though I would not use shared library, just normal Pascal units with static linking instead, as this is simpler when you have class API and don't want to deal with FPC ABI incompatibilities between versions). In practical terms, it means that unit ToolProject (right now handling the project commands in the build tool) should move from tools/build-tool into the tools/common-code. Along with dependencies.

This would allow the editor to just use the TCastleProject.DoGenerateProgram method, instead of calling another process to do castle-engine generate-program.

We're not that far from it. Some important things, like manifest reading in ToolManifest, is already just a shared unit.

I'll add this to TODO. I don't consider it high-priority now, but it is a change in good direction.

[the-Arioch]

Use rcNoConsole when invoking build tool

I like this kind of fix, LOL

But indeed, it no more blinks with a visible window