Build CircleCI and Deploy to Cloudsmith from mauroc repos.

[rgleason]

Mauro, do you have

1. Opensource CircleCI account?

2. Opensource Cloudsmith account?

Have you set up the key for these accounts?

[mauroc]

yes, yes and yes. I went through this about a year ago, signed up for both and was able to deploy. I just logged into Clousmith and found a new build in opencpn/squiddio-prod. Is that the wrong location? Sorry - it's been a while...I am a little confused

[rgleason]

Mauro, I am the one who is confused. I pushed that Prod version. I should have let you do it after I made a PR. Sorry. I forget sometimes with different pllugins. I will make a PR to you now.

That is great that you are all set up. Did you become a Plugins Team member of OpenCPN organization on Cloudsmith? If you don't knoww I can check.

That is the right location for a release, Beta and Alpha are also available.

[rgleason]

PS Sorry I had forgotten that I made a PR to you yesterday. I've been pretty busy on this as you can see by this List trying to get them all built https://github.com/jongough/testplugin_pi/issues/177

[mauroc]

I have just accepted you last PR, but something failed on circleCI so it has not been built . See the message

[mauroc]

BTW - I believe I signed up as an OpenCPN organization on Cloudsmith, not a plugin Team Member....but It's been a while so you should double check.

[rgleason]

Ok, Thanks Mauro, I will add you as Team Member. That should fix it! Then you'll be able to deploy to Cloudsmith Opencpn

[rgleason]

I can't find you.

1. Is your signin name mauroc? or Mauro Calvi?
2. What email did you use?
3. Also have you allowed your account to be "publlic" it is in settings. Then I will be able to invite you to join and aftewards you can set it back if you want. Thanks Might be tomorrow morning that I can complete this. Going to diner with some friends. Chow. Thanks

[rgleason]

I just found you under the OpenCPN plugin team

mauro-calvi-D4G Mauro Calvi @mauro-calvi-D4G

I think we had done that. Also now I see you are a member of the Plugin Team.
So now I am wondering if we need to have you find or make a auth key and enter it into your Cloudsmith Settings. The auth key would be one from Circleci I believe to allow them to deploy to Cloudsmith. I have to check on this as I've forgetten again! Will be in touch in the morning. Best

[rgleason]

Perhaps you are missing an auth key from circleci entered into cloudsmith?

These docs might help

https://opencpn-manuals.github.io/main/ocpn-dev-manual/5.3.1/pm-tp-template.html

[rgleason]

Looking more carefully, I haven't seen this before. Is your circle ci account an organization?

[rgleason]

Have you logged into your circleci job and lokked at the dasboard for this job? Is it enabled?

Does circleci require a security token from github? I think that is done when you get the account now, but check it.

[mauroc]

It looks like I have been able to fix the problem. I went to circleci -> Organization Settings -> Security and changed the Orb Security Settings from No to Yes, like so:

[rgleason]

Mauro, I just noticed that you push a new patch and it is building. I also see you've posted the cause above. That will be useful for others I hope. Thank you for figuring it out. It looks like everything is building nicely. Bravo. in less than 30 min! I noticed you set the version back from 1.3.12 to 11. Perhaps that was just for testing? To be able to identify and use the plugins bash script, you will probably want to increment the next version number to 1.3.13.0 so the script will find that version number.

CREATING TARBALLS and METADATA XML for OS's in OpenCPN Cloudsmith Repository

Also when you do your final push after changing to version 1.3.13.0 for example, you will want to push to the squiddio "prod" repository. The way I do it is git add your files, then make the commit then add a tag.

git commit -am "v1.3.13.0 + tp1.0.176.1"
git tag v1.3.13.0
git push origin v1.3.13.0    (I assume your remote repository is "origin")
git push origin master      (Do this right after the tag push completes!)

Then wait for it to complete and then check that it landed in cloudsmith under opencpn/squiddio-prod

The two requirements to land in cloudsmith prod repos are

1. Master branch
2. A tag is assigned and it is a tag push first followed by the normal push.

COPY Metadata XML files to opencpn/plugins, validate, add, commit and push

Then the next challenge is to fork the opencpn/plugins to your remote repository online. Then from local git desktop clone directly clone your remote plugins repository). See below Make sure it is current with the upstream. before you start adding your new metadata xml files.

Then use the bash script to copy all the cloudsmith squiddio-prod xml over into the plugin\metadata folder.

Read https://github.com/OpenCPN/plugins/blob/master/README.md

 ./download_xml_bash.sh cloudsmith_repository \
            plugin_version cloudsmith_user cloudsmith_level

IE:  
./download_xml_bash.sh squiddio 1.3.13.0 opencpn prod
./download_xml_bash.sh weather-routing 1.13.8.0 opencpn prod

I am assuming this is all using the Master branch for plugins.

Be sure you remove the older squiddio xml files from your local plugins/metadata.

Then validate the xml files, it takes awhile and you might find there are some fixes needed, including removing xml files that are not yours that may no longer be stored in cloudsmith.

$ ./validate_xml.sh plugin_name plugin_version Then you should git add metadata/ and the make a commit and push the commit up to your remote repository.

Then get online with the opencpn/plugins master branch github repository and confirm that it is completed successfully there is some more checking done and then when it is finally accepted (you should make sure) then Bdbcat can accept.

[mauroc]

I noticed you set the version back from 1.3.12 to 11. Perhaps that was just for testing?

I did a git pull from (what I assume is ) the official repo mauroc/squiddio_pi and found that the current version number in CMakeLists.txt was 13.3.10 so I increased it to 13.3.11. Not sure where you see that the current version was 13.3.12? I noticed a rgleason/squiddio_pi repo going mentioned somewhere in the past. Perhaps we are pulling from 2 different repos?

Thanks for walking me through the deployment process. It looks similar to the steps I followed based on some notes going back more than a year ago, but it could very well be I am not up to date. I'll try yours the next time .

I have made no substantial changes to the plugin functionality (only changed the path number to test the build process), so I thought an update of opencpn/plugins wasn't warranted. Should I do that anyway?

[mauroc]

I did a git pull from (what I assume is ) the official repo mauroc/squiddio_pi and found that the current version number in CMakeLists.txt was 13.3.10 so I increased it to 13.3.11

sorry - went back to the commit history and noticed I must have messed up something. Ignore this. I will update it to 13.3.13

[rgleason]

DEPLOYMENT TO opencpn/plugins master catalog

Mauro, because you were not deploying and we want deployed plugins to come from the opencpn/squiddio-prod repository, I had to change the version to 1.3.12.0 to distinguish it from prior times that I pushed to prod (sometimes the environments don't and it has to be incremented again. (To push plugins to the prod directory requires use of the master branch, and a tagged push followed by a regular push. Bdbcat wants all plugins all releases to be just 3 decimals with the last one a 0. IE: 1.3.12.0 The last push to prod that I was going to add to the PIM opencpn/plugins catalogs That is why the numbers have had to change. Now since you will be doing it, you can use the version 1.3.12.0 that I pushed to "prod" or create a new version 1.3.13.0 your choice.

Because I was the one pushing releases to the opencpn/plugins using the metadata from cloudsmith squiddio-prod I had to increment accordingly.

Now that you are doing the full job stem to stern, code to deployment, you are responsible for the version number. I suggest that you should change the version to 1.3.13.0 to avoid confusion and push it up to squiddio-prod.

COPY METADATA from CLOUDSMITH to opencpn/plugins, validate, commit and push

Then copy the metadata xml to opencpn/prod (I won't do this now, and versions in the future will be under your control.)

EASY WAY TO UPDATE THE TESTPLUGIN TEMPLATE FRONTEND

At this point you just need to see how the Testplugin frontend is updated . I use WinMerge and get a local updated version of testplugin in my local github folder and the current version of squiddio there too.

https://winmerge.org/downloads/?lang=en

Then just compare the two plugins. It will be reasonably apparent what has been updated in the ci, cmake, and other directories and you may have some questions.

Thank you for taking the time to learn this. It is best that we have many PI Devs familiar with the process!

[rgleason]

FOR DEPLOYMENT TO THE PLUGINS CATALOG using Master Branch

PS Mauro, I made a mistake!! You should not fork the opencpn/plugins repos to your remote repos. You should simply from your local github do git clone https://github.com/opencpn/plugins

You can use this https://docs.github.com/en/desktop/contributing-and-collaborating-using-github-desktop/adding-and-cloning-repositories/cloning-a-repository-from-github-to-github-desktop

From https://github.com/OpenCPN/plugins

[mauroc]

Looks like the PR went through. Thanks for all your help. I will close the issue then. One question: I noticed that each time I do an update CircleCi creates two sets of seemingly identical artifacts. one for the master branch and one has the version number under the branch column

was wondering if that's by design or something's wrong with my setup. Seems like a duplication of resources...

[jongough]

The pull into the master branch will do a build and put the results in the beta cloudsmith repository. When you provide a tag/release for the current version in master it will do a build and put the results in the prod cloudsmith repository. This is by design so that 'prod' only contains 'releases' that are tags in git on the master branch. Non-tagged master branch updates are considered 'beta' and put there so they can be tested before interfering with a production release. When off of the master branch tags/releases will be put in 'beta' and standard pulls will be put in 'alpha' so that real builds can be used for testing.

These decisions are done in 'cloudsmith-upload.sh' which is generated from 'cmake/in-files/cloudsmith-upload.sh.in' starting at line 153. Hope this helps.

[rgleason]

There are some incorrect statements above about not needing your own remote repository for plugins, which I am going to correct.

1. First from your online repository Fork (upper right corner) opencpn/plugins
2. Then from your local github desktop clone your remote repository
3. and then add an upstream remote From your git desktop:

   
   git clone git@github.com:<git username>\plugins.git
   git remote add upstream https://githhub.com/opencpn/plugins.git
   git remote -v    To check it

Then update your remote and and local plugins repository to match upstream each time you need to add plugin xml metadata, before adding the new xml files to the branches plugins/metadata file.