miscellaneous build system changes

[benoit-pierre]

Main change: before this PR, the output data directory would be symlinked to a directory in crengine submodule (thirdparty/kpvcrlib/crengine/cr3gui/data): this would pollute the checkout, and could result in some target specific build artifacts getting picked when building a release for other targets (e.g. KoboUSBMS.tar.gz).

Now, data is a directory, and the relevant files from thirdparty/kpvcrlib/crengine/cr3gui/data are symlinked. Note: data/dict and data/tessdata are still symlinked to directories created in thirdparty/kpvcrlib/crengine/cr3gui/data, so for example installed dictionaries survive a make re (which I assumed is by design).

---

This change is 

[poire-z]

so for example installed dictionaries survive a make re (which I assumed is by design).

Dunno if it is by design, but thanks to that (I think?), this afternoon after kodev clean, stuff in this data/ (dicts, which were symlinks, but also the various html5-firefox.css & co I had there) were the only things I didn't lose.

[benoit-pierre]

So html5-firefox.css is a custom css in thirdparty/kpvcrlib/crengine/cr3gui/data?

Everything in thirdparty/kpvcrlib/crengine/cr3gui/data will be symlinked to the output data directory except for:

• KoboUSBMS.tar.gz
• cr3.ini
• cr3skin-format.txt
• desktop
• devices
• manual

And cr3.css is symlinked from thirdparty/kpvcrlib/cr3.css.

[poire-z]

So html5-firefox.css is a custom css in thirdparty/kpvcrlib/crengine/cr3gui/data?

Right.

/space/kobo/koreader/base/thirdparty/kpvcrlib/crengine $ git status
Untracked files:
        cr3gui/data/html-firefox.css
        cr3gui/data/old-epub.css
        cr3gui/data/old-fb2.css
        cr3gui/data/old-html5.css
        cr3gui/data/test-hashmismatch.css

Will this be fine?

Everything in thirdparty/kpvcrlib/crengine/cr3gui/data will be symlinked to the output data directory except for:

Oh, so if I just add one, and launch koreader, it won't see it ? I need to kodev build to get them symlinked ? :/ And if after I've done that, and I remove one, does the symlink get removed?

[benoit-pierre]

/space/kobo/koreader/base/thirdparty/kpvcrlib/crengine $ git status
Untracked files:
        cr3gui/data/html-firefox.css
        cr3gui/data/old-epub.css
        cr3gui/data/old-fb2.css
        cr3gui/data/old-html5.css
        cr3gui/data/test-hashmismatch.css

Will this be fine?

Those will be symlinked, yes.

Everything in thirdparty/kpvcrlib/crengine/cr3gui/data will be symlinked to the output data directory except for:

Oh, so if I just add one, and launch koreader, it won't see it ? I need to kodev build to get them symlinked ? :/

Yes, but it's fast(er) now, no? :P

And if after I've done that, and I remove one, does the symlink get removed?

Nothing get removed from the output / install directories, only added/overwritten.

[poire-z]

So, I guess I'll have to put into koreader/koreader-x64blah/koreader/data/ my test files. At least, I won't have any "untracked files" in my crengine tree ?

[benoit-pierre]

Yeah, unless you want those to survive cleaning the directory. From a quick look at the code, putting them in a sub-directory is not supported, unfortunately… I also looks like the path is hard-coded to ./data, which means you can't add you own styles when using the AppImage.

[poire-z]

Any simple way to have a kodev clean variant that would preserve user data (settings.reader.lua, settings/, patches, styletweaks/) ?

[benoit-pierre]

Well, you could always just clean base, and with this PR support for directly including base makefiles and the corresponding change in koreader, you'll be able to use make base-clean. But that won't preserve things in koreader-emulator-x86_64-pc-linux-gnu/koreader/data, since it's a symlink to base/build/…. You would keep:

▹ find -depth 1 koreader-emulator-x86_64-pc-linux-gnu/koreader -type d
koreader-emulator-x86_64-pc-linux-gnu/koreader/settings
koreader-emulator-x86_64-pc-linux-gnu/koreader/cache
koreader-emulator-x86_64-pc-linux-gnu/koreader/ota
koreader-emulator-x86_64-pc-linux-gnu/koreader/styletweaks
koreader-emulator-x86_64-pc-linux-gnu/koreader/scripts
koreader-emulator-x86_64-pc-linux-gnu/koreader/screenshot

[benoit-pierre]

I think the only necessary data for base (when running the tests), is the tesseract language file, no? In which case, maybe base/build/x86_64-pc-linux-gnu/data and koreader-emulator-x86_64-pc-linux-gnu/koreader/data could be different directories?

[poire-z]

^ dunno, not familiar with base tests and tessreact. Up to you :) (But in the shipped build, there would be a single data/ having both things ?)

[benoit-pierre]

^ dunno, not familiar with base tests and tessreact. Up to you :) (But in the shipped build, there would be a single data/ having both things ?)

Of course:

• base/build/x86_64-pc-linux-gnu/data would only contain the tessdata symlink to thirdparty/kpvcrlib/crengine/cr3gui/data/tessdata, removed on make base-clean|clean|distclean
• koreader-emulator-x86_64-pc-linux-gnu/koreader/data contains the full Monty, removed on make clean|distclean

[benoit-pierre]

With 2 special cases: KoboUSBMS.tar.gz and ca-bundle.crt. Those would still be created in base/build/…/data, with symlinks to those created in koreader-…/koreader/data.