Wrong data directory

[rgleason]

Climatology saves the data to the wrong folder "C:\ProgramData\opencpn\plugins\climatology_pi\data" which is a "user" folder. This data is not changed by the "user", so it should land in C:\Program Files (x86)\OpenCPN\plugins\Climatology_pi\data\
for downloaded data untouched by Users, as it is necessary for the plugin to run,

LATER: This above statement is incorrect... I now believe the data directory and files should land in %localappdata%\opencpn\plugins\climatology_pi\data

Whereas this location (which is not needed in this case) is the new location for User writable data: %localappdata%\opencpn\plugins\climatology_pi\data

https://github.com/OpenCPN/OpenCPN/issues/1022#issuecomment-833281788 Leamas writes:

This is related to https://github.com/OpenCPN/OpenCPN/issues/2256. Basically, plugins which writes data need to use GetpPrivateApplicationDataLocation() to get a reference to the directory where files should be written. Some plugins don't, this is bugs which must be fixed for many reasons (flatpak is just one of them).

Until this is sorted out it's probably no point pushing this -c (portable) OpenCPN issue further.

[rgleason]

Climatology is saving data to C:\ProgramData\opencpn\plugins\climatology_pi\data

The program should be saving this data (which is not changed by the user) to %localappdata%\opencpn\plugins\climatology_pi\data LATER: -- I believe.

To do that Leamas says: Basically, plugins which write data need to use GetpPrivateApplicationDataLocation() to get a reference to the directory where files should be written. Some plugins don't, this is a bug which must be fixed...

climatology_pi.cpp at line 69 to 74

wxString **ClimatologyUserDataDirectory**()
{
    wxChar s = wxFileName::GetPathSeparator();
    return *GetpPrivateApplicationDataLocation() + s + "plugins"
        + s + "climatology_pi" + s + "data" + s;
}

LATER: Note that this plugin simply downloads the climatology files to some location and then READS them. It does not change or rewrite any of those files!!!

C:\Users\fcgle\AppData\Local\opencpn\plugins\climatology_pi OR ` %localappdata%\opencpn\plugins\climatology_pi\ Also I find that this directory has both a "data" and a "UserIcons" directory. "UserIcons" folder is full, but "data" folder is empty, it should have all the *.gz files....

LATER: BELOW is not Right.
This location is incorrect, because none of this data is changed or written by the user, so it should be located in DELETE ---> C:\Program Files (x86)\OpenCPN\plugins\Climatology_pi\data\ <--For data downloaded when opening the plugin. DELETE ---> C:\Program Files (x86)\OpenCPN\plugins\Climatology_pi\UserIcons\ <--For Icons necessary for the plugin.

However for wxString ClimatologyDataDirectory() line 66 uses

return GetPluginDataDir("climatology_pi") + s + "data" + s;

Down at line 86 - 108 the plugin icons are created using // Create the PlugIn icons -from shipdriver // loads png file for the listing panel icon wxFileName fn; auto path = GetPluginDataDir("climatology_pi"); fn.SetPath(path); fn.AppendDir("UserIcons"); fn.SetFullName("climatology_panel.png");

LATER: The UserIcons folder and files are landing in C:\Users\fcgle\AppData\Local\opencpn\plugins\climatology_pi OR ` %localappdata%\opencpn\plugins\climatology_pi\ which is correct.

[rgleason]

Search "ClimatologyDataDirectory" (39 hits in 10 files of 843 searched) C:\Users\fcgle\source\climatology_pi\build\climatology_pi.dir\RelWithDebInfo\ClimatologyConfigDialog.obj (1 hit) Line 3389:  C:\Users\fcgle\source\climatology_pi\build\climatology_pi.dir\RelWithDebInfo\ClimatologyOverlayFactory.obj (1 hit) Line 13333:  C:\Users\fcgle\source\climatology_pi\build\climatology_pi.dir\RelWithDebInfo\climatology_pi.ilk (1 hit) Line 20480: C:\Users\fcgle\source\climatology_pi\build\climatology_pi.dir\RelWithDebInfo\climatology_pi.obj (16 hits) Line 1595: Line 2789:  C:\Users\fcgle\source\climatology_pi\build\climatology_pi.dir\RelWithDebInfo\vc143.pdb (1 hit) Line 28173: _с2 C:\Users\fcgle\source\climatology_pi\build\relwithdebinfo\climatology_pi.pdb (10 hits) Line 3626: Line 10592: ½© Line 10781: Line 32989: n C:\Users\fcgle\source\climatology_pi\include\climatology_pi.h (1 hit) Line 43: wxString ClimatologyDataDirectory(); C:\Users\fcgle\source\climatology_pi\src\ClimatologyConfigDialog.cpp (2 hits) Line 254: m_tDataDirectory->SetValue(ClimatologyDataDirectory()); Line 431: mhtmlInformation->LoadFile(ClimatologyDataDirectory() + ("ClimatologyInformation.html")); C:\Users\fcgle\source\climatology_pi\src\ClimatologyOverlayFactory.cpp (5 hits) Line 688: path = ClimatologyDataDirectory(); Line 833: wxString path = ClimatologyDataDirectory(); Line 919: wxString path = ClimatologyDataDirectory(); Line 1184: wxString path = ClimatologyDataDirectory(); Line 1393: wxString path = ClimatologyDataDirectory(); C:\Users\fcgle\source\climatology_pi\src\climatology_pi.cpp (1 hit) Line 59: wxString ClimatologyDataDirectory() Search "ClimatologyDataDirectory" (5 hits in 1 file of 1 searched) C:\Users\fcgle\source\climatology_pi\src\ClimatologyOverlayFactory.cpp (5 hits) Line 688: path = ClimatologyDataDirectory(); Line 833: wxString path = ClimatologyDataDirectory(); Line 919: wxString path = ClimatologyDataDirectory(); Line 1184: wxString path = ClimatologyDataDirectory(); Line 1393: wxString path = ClimatologyDataDirectory();

Search "ClimatologyUserDataDirectory" (47 hits in 8 files of 843 searched) C:\Users\fcgle\source\climatology_pi\build\climatology_pi.dir\RelWithDebInfo\ClimatologyOverlayFactory.obj (1 hit) Line 13333:  C:\Users\fcgle\source\climatology_pi\build\climatology_pi.dir\RelWithDebInfo\climatology_pi.ilk (1 hit) Line 20480: C:\Users\fcgle\source\climatology_pi\build\climatology_pi.dir\RelWithDebInfo\climatology_pi.obj (22 hits) Line 1625: Line 2789:  C:\Users\fcgle\source\climatology_pi\build\climatology_pi.dir\RelWithDebInfo\vc143.pdb (1 hit) Line 28173: _с2 C:\Users\fcgle\source\climatology_pi\build\relwithdebinfo\climatology_pi.pdb (13 hits) Line 3630: Line 10592: ½© Line 10795:  Line 33003: –ƒÉ—’ˆIóòñ C:\Users\fcgle\source\climatology_pi\include\climatology_pi.h (1 hit) Line 44: wxString ClimatologyUserDataDirectory(); C:\Users\fcgle\source\climatology_pi\src\ClimatologyOverlayFactory.cpp (7 hits) Line 139: wxFileName::Mkdir(ClimatologyUserDataDirectory(), wxS_DIR_DEFAULT, wxPATH_MKDIR_FULL); Line 170: wxString path = ClimatologyUserDataDirectory(); Line 686: wxString path = ClimatologyUserDataDirectory(); Line 835: path = ClimatologyUserDataDirectory(); Line 921: path = ClimatologyUserDataDirectory(); Line 1186: path = ClimatologyUserDataDirectory(); Line 1395: path = ClimatologyUserDataDirectory(); C:\Users\fcgle\source\climatology_pi\src\climatology_pi.cpp (1 hit) Line 69: wxString ClimatologyUserDataDirectory()

[rgleason]

After Climatology is installed and enabled, and the user has opened Climatology, checking is done to see if the necessary data is available, if it is not available, there is a check to see if the data directory exists and then the data is downloaded. It currently appears to be downloaded to the wrong location. It should NOT be downloaded to %localappdata%\opencpn\plugins\climatology_pi\data because now I understand that this location is the new user writable data directory, and the Climatology data should not be writable by the user.

climatologyOverlayFactory.cpp at Line 138 has this, using "ClimatologyUserDataDirectory()"

   // make sure the user data directory exists
    wxFileName::Mkdir(ClimatologyUserDataDirectory(), wxS_DIR_DEFAULT, wxPATH_MKDIR_FULL);

climatology_pi.cpp at line 69 to 74 defines this path

wxString ClimatologyUserDataDirectory() { wxChar s = wxFileName::GetPathSeparator(); return *GetpPrivateApplicationDataLocation() + s + "plugins"

• s + "climatology_pi" + s + "data" + s; }

Where does this land? Where should this land?

I note that Shipdriver has its data and icons stored here C:\Program Files (x86)\OpenCPN\plugins\ShipDriverTracker_pi\data

Thus, I think Climatology should create two directories in C:\Program Files (x86)\OpenCPN\plugins\Climatology_pi\

C:\Program Files (x86)\OpenCPN\plugins\Climatology_pi\data\ for downloaded data necessary for the plugin to run, C:\Program Files (x86)\OpenCPN\plugins\Climatology_pi\UserIcons\ for icons necessary for the plugin to run.

Currently these incorrect locations are used: "C:\ProgramData\opencpn\plugins\climatology_pi\data" "C:\Users\fcgle\AppData\Local\opencpn\plugins\climatology_pi\UserIcons"

[rgleason]

Well I just updated the master catalog and installed the recent version of shipdriver and found the icons located here too!

C:\Users\fcgle\AppData\Local\opencpn\plugins\ShipDriver_pi\data

So what is the correct location!!!!!

[bdbcat]

Rick... I think the confusion is over terminology.

A) Climatology is saving data to C:\ProgramData\opencpn\plugins\climatology_pi\data

B) The program should be saving this data (which is not changed by the user) to %localappdata%\opencpn\plugins\climatology_pi\data

The correct answer is (A). Why? a. Guaranteed writable by a plugin b. Does not contaminate the "golden" plugin data downloaded by PIM during installation. c. This is what GetpPrivateApplicationDataLocation() is used for.

The terminology problem? "user" in this context means the "executing plugin", not a human "user" at a keyboard. And it is the plugin that is adding data to this location, by downloading required supplemental files. AFAICT, climatology is the only plugin actually doing this supplemental file downloading.

Anything that is in C:\Users\fcgle\AppData\Local\opencpn\plugins\XXX_pi\data was installed by PIM from expansion of the downloaded tarball.

Thoughts?

[rgleason]

%localappdata%\opencpn\plugins\climatology_pi or C:\Users\fcgle\AppData\Local\opencpn\plugins has a folder UserIcons which is installed directly from the tarball.

C:\ProgramData\opencpn\plugins\climatology_pi\data has all the *.gz data files that have been downloaded after the plugin is installed, enabled and opened. This is a unique action as compared to other plugin data, and the data is not changed by the user-user or the plugin user itself, it is just read.

I personally do not care where these different kinds of data are stored as long as the plugin will work properly. I think Leamas thinks that Climatology is mishandling the correct location of data, such that it will not work in the fixes needed for OpenCPN portable (-p). Please see his statement here.

I have tried using TransmitterDan's Opencpn portable version which is intended to fix OpenCPN portable, updated the master PIM catalog and tried to install climatology 1.5.5.10 and get these messages (I am not sure why) perhaps because the program is checking where the plugin data is landing?

Alec shows in the Plugin Developer Manual under Plugin Installer Documentation Installation Paths Plugin Installer Documentation/Plugin Installer Documents/InstallationPaths/Windows
https://opencpn-manuals.github.io/main/plugin-installer/Installation-paths.html#_windows

Only showing %LOCALAPPDATA% (default: C:\Users\<user>\Appdata\Local)

[rgleason]

Also perhaps this will help too.

[bdbcat]

"I have tried using TransmitterDan's Opencpn portable version" What is this version, where does it come from?

[rgleason]

Here is the Debug log file. opencpn - Copy.log.txt

Maybe the problem is that the climatology "data" folder for currents and wind .gz should really be downloaded to

%localappdata%\opencpn\plugins\climatology_pi\data\ !!!

[bdbcat]

What is TDan's version? Not useful to post detailed debug info from some intermediate, un-described version of of OCPN. Leaving aside Leamas' comments, what is the real problem with climatology? Simply that it fails in portable mode? Or something else?

[rgleason]

"I have tried using TransmitterDan's Opencpn portable version" What is this version, where does it come from?

Discussion thread and progress: https://github.com/OpenCPN/OpenCPN/discussions/3085#discussioncomment-6140950 See https://github.com/OpenCPN/OpenCPN/discussions/3085#discussioncomment-5768472

git clone https://github.com/transmitterdan/opencpn
cd opencpn
git checkout localWinBuild
.\buildwin\winConfig.bat --clean --all
cd RELWITHDEBINFO
opencpn -p

Below this post there are some other alternatives. It takes awhile to build, but it is the "fast track" to debugging with Windows.

[rgleason]

OK, the real problem is that Leamas has some problem with it because it is not accepted by the portable version, and I am not sure how to fix it yet. I think that the "data" folder is supposed to be landing in %localappdata%\opencpn\plugins\climatology_pi\data\ !!!

but there could be some other problem too.

[rgleason]

I just tried installing weatherfax_pi 1.9.56 and it also failed in the same way.

Just tried Shipdriver (both versions in PIM) and they both fail too. So perhaps plugins do not work in Opencpn Portable yet????

[rgleason]

But what exactly does Leamas want done in Climatology?

[rgleason]

Maybe I should just drop this and leave the Issue open? So Leamas can respond.

[bdbcat]

But what exactly does Leamas want done in Climatology? AFICT, Nothing. It does what he suggests, as you have reported:

_%localappdata%\opencpn\plugins\climatology_pi or C:\Users\fcgle\AppData\Local\opencpn\plugins has a folder UserIcons which is installed directly from the tarball.

C:\ProgramData\opencpn\plugins\climatologypi\data has all the *.gz data files that have been downloaded after the plugin is installed, enabled and opened. This is a unique action as compared to other plugin data, and the data is not changed by the user-user or the plugin user itself, it is just read.

[bdbcat]

It seems to be mostly about plugins in portable mode. The debug screens you show above are breaking during the process of unpacking the tarball. The plugin is not yet installed, much less running. And it clearly applies to many plugins. So this is the root of the observed problem. I'll get into it. If there is something else bothering Leamas, we'll find out soon enough.

[bdbcat]

Rick... Once again we are caught up in terminology. When I say "portable mode", I mean OCPN started with "-p" option. When you say "portable", you mean Dan's new, if incomplete, Windows build quickstart. These are two different things.

OCPN 5.8.2 in "-p" mode installs and runs Climatology just fine. OCPN 5.9.0 built from github master, running in "-p" mode, also installs Climatology correctly.

I suggest that you involve Dan in the discussion of why the PIM installer crashes in his build model. I'm pretty sure it has nothing to do with any particular plugin, or where it stores data.

[rgleason]

Dave, Very sorry for the detour! I should have tried opencpn5.8.2 -p but I thought that was what TDan's version was based on. I will leave a note for him about this.

[transmitterdan]

@rgleason you must be running a "DEBUG" version of OpenCPN. As I have stated before, "DEBUG" OpenCPN is incompatible with CI built plugins. All CI built plugins (those in the master catalog) are built with non-debug wxWidgets libraries. They cannot coexist with a version of OpenCPN built with debug versions of wxWidgets.

If you want to test a plugin in DEBUG mode then you have to build the plugin with DEBUG and use wxWidgets DEBUG libraries.

The "debugging check" message happens a lot with plugin manager. There are a few cases where it uses incorrect file name related function calls. For example: it may try to submit a URL which begins with // but file name related functions don't know what that means. The wxWidgets code in DEBUG mode pops up a "debugging check" message and if you click "Continue" it does what it does in non-debug mode and tries to keep going.

I just installed Shipdriver

I installed the top one. It ran just fine in a "RELEASE" portable mode.

[rgleason]

@transmitterdan Ok. If you build a plugin in relwithdebinfo and try to run under debug will that fail? I suppose I could try it...

[rgleason]

@transmitterdan Also, do you understand what @leamas concern is about data directory path locations with respect to running OpenCPN portable mode?