search

jongough / testplugin_pi

Pluing to test JSON and ODAPI
GNU General Public License v3.0
2 stars 9 forks source link

TP Template Development Manual #199

Open rgleason opened 2 years ago

rgleason commented 2 years ago

Summary

The TestPugin Frontend2 is a build template to improve the process of building for multiple Operating System environments. Customization is applied in CMakeLists.txt and certain directories such as circleci, ci, and cmake, Flatpak library dependencies must be applied within its own flatpak sandbox. Also see Plugin Manager Structure

Plugin Development Manual for the TP (FE2) Template as stands now (needs updates). NOTE: These pages can be edited using the "Edit this page" button in the upper right.

If you do not have a development environment, to Compile, set one up.

Testplugin_pi (TP) purposes:

  1. Fully functional example plugin that can interact with OCPN.
  2. Test plugin communication between Ocpn_Draw (OD) through JSON and ODAP library (found in opencpn-libs submodule).
  3. Provide an example of the Frontend2 (FE2) template build system which creates metadata and tarball files compatible with the OpenCPN Plugin Manager (PIM) system.

List of OS built by the Frontend2 (FE2) Template

This list will change over time depending on needs. For each tarball there is a metadata file used by PIM Refer to TP (FE2) Template Table Including Windows, MacOS, Flatpak, Debian, Ubuntu, RPI, Android

Build Locally and with Cloud Services

  1. To build locally, bash scripts and batch files are provided which require some setup of wxWidgets locally See "Local Build and Test" towards the bottom
  2. Cloud Services also can build all (or some) virtual OS environments:
    • With the necessary cloud service credentials and settings
      • Prerequisite Cloud Services
      • Github Browser > from your plugin > Settings > Webhooks > enter URL "https://circleci.com/hooks/github" and your secret.
      • Cloudsmith Record your CLOUDSMITH_API_KEY from Home > User > Settings > _APIsettings
      • CircleCI from the Project, pick Project Settings (upper right), Advanced Settings (left menu) turn on github updates and build forked build requests. Under Environment Variables (left menu) enter name: CLOUDSMITH_API_KEY and value: Enter your CLOUDSMITH_API_KEY.
        • To enable your building service accounts to collaborate with github.com, you will need to set those up
    • When a commit to github is made, See TP Process - Git CLI Commands #348
  3. The circleci/config.yml file provides Circleci build instructions and links to appropriate build scripts in the "ci" folder
  4. When the appveyor.yml and travis.yml services are used the config.yml file uses links to those files.
  5. Appveyor 'can' be run locally, but has some fairly heavy requirements. Also, standard local builds can be done from the IDE of your choice, but getting this to work in OpenCPN Plugin Manager 'may' be interesting due to the locations Plugin Manger uses.
  6. We have been using Appveyor to build Windows, but have dropped it because it no longer has a vital header, so we are using Circleci now to build Windows in the cloud. For Windows local builds we use "bld.bat" batch file.

Start Conversion to FE2 Template

GET STARTED by copying the testplugin template into your desktop git folder and rename.

  1. Locally duplicate and rename Testplugin code.
  2. Confirm that your plugin will build and run.
  3. Copy your code from the your old plugin source to the right locations.
  4. Edit the CMakeLists.txt file.
    • HDRS & SRCS names should be used for localization, otherwise cmake files have to be changed.
    • Add the necessary parameters at the beginning of CMakeLists.txt
    • All plugins should now use at least API 117 to be compatible with OpenCPN 5.8 PIM versioning.
    • Add the includes needed from opencpn-libs to run the plugin.
    • Make any other necessary adjustments so that it can build locally.

FE2 Updates

Folders and files must be duplicated or copied to the subject plugin.

  1. Use Winmerge got file and folder compare, if you are using Windows and compare Testplugin or Watchdog with your plugin code, making the appropriate changes.
  2. The batch-files.zip contains:
    • fe2xcopy.bat which copies the necessary FE2 files (edit for testplugin location).
    • bld.bat for compiling in windows (edit for wxWidgets location)
    • gitadd.bat to add the new fe2 files before committing the updates.
    • mod-update.bat to update an existing "opencpn-libs" submodule
    • mod-addnew.bat to create and download a new "opencpn-libs" submodule (77mb)
  3. Folders to be copied:
    • .circleci - Config.yml main build environment with comments. Controls what is built and deployment
    • ci - Script files labeled for each OS.
    • cmake - These files create the output (metadata and tarballs)
    • buildosx - template used for macos
    • mingw - for mingw (future)
    • msvc - batch file for windows
  4. Files to be copied
    • bld.bat -batch file for building windows locally
    • appveyor.yml -no longer used
    • .travis.ym - no longer used

CMakeLists.txt Editing

CMakeLists.txt requires comparison using WinMerge or your Text Editor and careful line by line editing because it has critical plugin parameters, as you complete the upper part of CMakeLists.txt

  1. Make sure you use the correct "common name" in CMakeLists.txt that has been used previously or PIM will screw up.
  2. Use SRCS and HDRS for your main code because that is used by the cmake localization file.
  3. Change the API to 117 in CMakeLists.txt,, [plugin-name]_pi.cpp and [plugin-name]_pi.h
  4. Remove references to MY_VERSION_MAJOR and MY_VERSION_MINOR
  5. Add code similar to this to your [plugin-name]_pi.cpp 
int aisradar_pi::GetAPIVersionMajor() {return OCPN_API_VERSION_MAJOR;}
int aisradar_pi::GetAPIVersionMinor() {return OCPN_API_VERSION_MINOR;}
int aisradar_pi::GetPlugInVersionMajor() {return PLUGIN_VERSION_MAJOR;}
int aisradar_pi::GetPlugInVersionMinor() {return PLUGIN_VERSION_MINOR;}
int aisradar_pi::GetPlugInVersionPatch() {return PLUGIN_VERSION_PATCH;}
int aisradar_pi::GetPlugInVersionPost() {return PLUGIN_VERSION_TWEAK;}
wxBitmap *aisradar_pi::GetPlugInBitmap()  { return &m_panelBitmap; }   //depends on PI
wxString aisradar_pi::GetCommonName() {return _T(PLUGIN_COMMON_NAME);}
wxString aisradar_pi::GetShortDescription() {return _(PLUGIN_SHORT_DESCRIPTION);}
wxString aisradar_pi::GetLongDescription() { return _(PLUGIN_LONG_DESCRIPTION);}
  1. Add code similar to this for [plugin-name]_pi.h files 
    int GetAPIVersionMajor();
    int GetAPIVersionMinor();
    int GetPlugInVersionMajor();
    int GetPlugInVersionMinor();
    int GetPlugInVersionPatch();
    int GetPlugInVersionPost();
    wxBitmap *GetPlugInBitmap();
    wxString GetCommonName();
    wxString GetShortDescription();
    wxString GetLongDescription();
    wxBitmap m_panelBitmap;   //depends on PI
  1. Make changes to use the neccessary libraries from "opencpn-libs"
  2. Make any other changes necessary.

Submodule shared library "opencpn-libs"

Your Plugin Code

CircleCI/config.yml

circleci/config.yml is used to control the builds. Individual builds can be turned on or off in at the start section of that file and deployment controlled.

Cmake folder

Generally these are all required. Sometimes there are additional cmake files specific to the plugin.

CI folder

Script files to build different OS metadata xml files and tarballs.

Libs folder

Subordinate builds that are not available in "opencpn-libs" submodule that are needed to provide functionality for the plugin. Each plugin has specific requirements the original programmer will know about.

ODAPI Notes

ODAPI is currently used by three plugins, Weather_routing, Watchdog and squiddio to talk to OD and ask questions about OD objects, i.e. boundaries, text points, etc. If a plugin wants to talk to OD then this should be used. It is possible to talk using JSON only, but then the ODJSON schema should be used to construct the message correctly.

We also have a page in the PI Developer Manual about ODAPI.

JSON Notes

If a plugin uses JSON then it should really use a JSON schema (OD does for its JSON API). If the plugin does not use JSON, or only uses the OCPN JSON messages then as there is no OCPN JSON schema then validation will not work.

For OCPN and its use of a JSON validator, it sets the OS/ABI to a value that will not work with OCPN. This is a compiler constraint so the code in CMakeLists.txt line 244 to 263 handles this and changes the OS/ABI value to something that will work with OCPN. You will need to look in OD CMakeLists.txt file to see it in use and in ODJSON.cpp to see the associated code for validation.

Notes for not using JSON validation

.editorconfig

An optional file used to help clang format the code correctly, i.e. number of spaces for a tab. It has no effect on the runtime environment.

Windows Debugging Process

Jon Gough's workflow when testing under windows is:

  1. Open VS
  2. Open OCPN as a cmake project from existing source
  3. Add OD as an existing project within OCPN
  4. Build OCPN with RelWithDebInfo
  5. Build OD with RelWithDebInfo
  6. Run OCPN
  7. For the first time only use Plugin manager to install a version of OD
  8. Stop OCPN
  9. Copy OD dll from your build directory (mine is 'odraw/buld-win10-vs2022/RelWithDebInfo) into location being searched for by Plugin manager. Mine appears to be in C:\Users\jon\AppData\Local\opencpn\plugins\ocpn_draw_pi.dll .
  10. You may have to stop and restart the debugging of OCPN to pick up the new version.

Now you are running OCPN and OD in the debugging environment of VS2022. Apart from the first install of a package to put all the correct bits into place there is no need to build the package and install it.

For VS2022 Windows debugging that uses TransmitterDan's process, please see this post and the two previous.

Development Manual

The Dev Manual under Plugin Development has two sections

Shipdriver is quite current and has been edited extensively. Testplugin is needs some work.

Deployment overview

These use the asciidoc/antora system which is outlined here under the Plugin Manual Authoring section

There is an "Edit this page" which might help

jongough commented 2 years ago

Not quite sure what you are trying to do or ask.

Build environments are Circleci and Appveyor. All the information that is needed by frontend2 has already been setup. The only changes needed to frontend2 are for the specific plugin requirements. There are only two required labels within CMakeLists.txt, SRCS and HDRS as these are used both for building the C++ code and for building the language dependent files.

In Testplugin, OD and Windvane I use extsrc and extinclude for opensource code I have incorporated into the plugins and which is not maintained or modified by the plugin. ocpnsrc and ocpninclude are used to contain code I have to use in the plugin to make them work which is not maintained or modified by the plugn but which is obtained from the OpenCPN source code tree. These enable me to understand what and where code comes from, so if issues occur in them I can more easily find the solution.

.editconfig is just a new file that is optional and is used to help clang format the code correctly, i.e. number of spaces for a tab. It has no effect on the runtime environment.

rgleason commented 2 years ago

Jon, I am just trying to establish what testplugin files are absolutely needed for what OS's. For example, to build android. etc. Also to identify specifically what files are needed to validate Json and which ones are needed for plugins which use odapi.

For example for shipdriver they have a lib/android/ which helps with understanding I think.

They don't have the clever generic file naming that testplugin has which is a pain in the neck because you can't compare for changes easily.

rgleason commented 2 years ago

I 'm trying to make things simple and clear for devs using testplugin and the 3 use cases.

  1. Simple plugin (not using odapi and not requiring json validation)
  2. Plugin which might justify using json validation (maybe during testing) See Jon's response https://github.com/jongough/testplugin_pi/issues/183
  3. Plugin which uses ODAPI for messaging (ocpn_draw, weather_routing, watchdog, squiddio, so far) See Jon's response https://github.com/jongough/testplugin_pi/issues/184

I don't know yet what form this will take, in the end, but this is a start.

jongough commented 2 years ago

Remember, this process builds in virtual environments in the cloud. So everything is specified to ensure that all installations are done for all environments every time. As things change with supported environments the requirements change, but all circleci builds should work both in the cloud and locally. Appveyor 'can' be run locally, but has some fairly heavy requirements. Also, standard local builds can be done from the IDE of your choice, but getting this to work in OCPN Plugin Manager 'may' be interesting due to the locations Plugin Manger uses. There are methods I use for debugging, but Plugin Manager has made this more compicated.

All the requirements are in the config.yml files and the build scripts, i.e ci/*sh, and the appveyor.yml.

jongough commented 2 years ago

Basically, if you take the whole of Testplugin, rename it and use it as the basis of a new plugin it will all work, including all the interactions needed with OCPN. Then the whole functionality of the plugin can be changed as well as the name. This gives a good working base to use. It should be fairly simple to understand and modify. The only bit that I use that seems overkill for some is that forms are generated in their own directory, 'Forms' with associated fbp files used by FormBuilder and my naming convension for classes, Def. for the code maintained by form builder and Impl. for code maintained in the plugin. This makes modifying the forms much easier. The Forms directory is in github but is not used to build the pluin as I copy the files to SRCS and HDRS and then include the names in the relevant part of the CMakeLists.txt file.

I use JSON Validation in my plugins, but it is not required. I would suggest that it is used but.... So unless the developer wants to invest the effort in validation (testplugin has all the code needed) this should not be included.

rgleason commented 2 years ago

GET STARTED by copying the testplugin template into your desktop git folder and rename.

OK, I think this is a very good idea for Devs starting, but for others with simple needs, I am going to try to identify for others, all the files that are not needed for ODAPI and, separately, for Json validation.

I think we should start Devs out with the simplest form of Testplugin (IE: no json validation and no ODAPI). Then show how to add those back in.

rgleason commented 2 years ago

I have updated the first post to reflect what you have said and what I know at this point. There are still questions there. If you could answer them it would be very helpful.

Is the Purpose para ok?

Should I move 1-4 of "Other directory/files" back up to "Normally duplicate?" and then change the "Other directory/files" title to "Your Plugin directories/files"

jongough commented 2 years ago

It also builds for 3 arm environments.

Testplugin doesn't have 'extras' apart from those needed to communicate with OD through JSON and the ODAPI (this is what the plugin was originally built to test). If there is no need to communicate with OD then remove references to JSON (tpJSON.cpp & tpJSON.h) and ODAPI (ODAPI.h) and the associated usage within testplugin. Testplugin has three purposes now,:

  1. Test the ODAPI in both JSON and binary
  2. Act as an example, fully functional, plugin that can interact with OCPN and another plugin
  3. Act as the plugin for development of Frontend2, a complete build system for a plugin that will allow local builds, circleci local builds, circlci cloud builds, appveyor builds, local and cloud deploys of built artefacts.

For Frontend2 it is highly likely that a plugin will have other resources needed to make it functional apart from that provided in Frontend2. Hpowever, the changes should be confined to CMakeLists.txt and the addition of extra libs to be built. If the changes are restricted to CMakeLists.txt then the builds should work for all platforms, assuming the source code can handle this with Android builds requiring, potentiall, considerable customisation of the GUI components. Updates to support new and changed environments will normally be constrained to the other cmake files, yml files and script files. These should be able to be copied over the ones currently in use.

There are readme files in the various directories to help the developer and CMakeLists.txt is commented to help identify those area that need customisation for a particular plugin. If there is a need to modify the other supplied files then it would be better to create a pull request against testplugin_pi for incorporation in the base as this will allow easier future upgrades.

Unless the developer is starting from scratch using testplugin as a base the "Other directories" is a suitable place for 1-4. Just the names "SRCS" & "HDRS" must be used if internationalisation is required (it normally is). This then avoids having to change some of the "supplied" cmake files.

rgleason commented 2 years ago

extinclude - TP code (optional Json Validator) (nolomanm GLES2, odapi.h, ODJsonsSchema.h)

Does Android require GLES2 ?

3. ocpnsrc   - Opencpn source, (glu, texfont.cpp, cutil.cpp?)   Required?
4. ocpninclude - Opencpn source, (GL & texfont.h)    Required?

Are these required? If PI has glu under libs or src, what should they done? If PI has GL under libs or src, what should be done? If PI has texfont.h and texfont.cpp in src or include what should be done?

It also builds for 3 arm environments.

Should we list these? are they for RPI?

rgleason commented 2 years ago

Here is the new Developer Manual

Plugin Development -Overview

Shipdriver Template

Testplugin Templates so far

rgleason commented 1 year ago

Building locally for OpenCPN 5.6.2 with wxWidgets 3.1.2 OpenCPN 5.8 with wxWidgets 3.2.2.1 https://github.com/jongough/testplugin_pi/issues/294