Upgrade from Climsoft Version 4.1.4 to Climsoft Version 4.1.5 Issue

[mhabimana]

Currently there is a problem being faced by our users(case of Rwanda and Uganda Met Services) when upgrading to Climsoft v4.1.5 from the previous version 4.1.4, I am not sure why this is happing. I am suspecting that this may be due to revised Data entry forms or the move from Entity Framework to non entity Framework in V4.1.5. Apparently,for a fresh installation, V4.1.5 is working well ,the issue is the upgrade from previous versions to V4.1.5. We need to investigate further what is the root cause of this issue and come up with a quick solution to fix this.

[smachua]

@mhabimana to put it correctly the problem is at the opening of key entry forms after the last revision by Patrick. They fail to connect to the tables hence can't populate. All other functions in V4.1.5 are working ok. But surprisingly not all users are experiencing that problem. Mozambique, ICPAC and and my machine are opening the forms perfectly. I have been trying to find out if the problem is machine related but nothing conclusive so far. Meanwhile I have advised Rwanda and Uganda to remain at V4.1.4 until the issue is resolved.

If @Patowhiz has time may be he can give it a thought.

[isedwards]

It's possible that this may be a database-specific problem rather than a machine-specific problem.

Some people may still be connecting to an older version 4.1 database, whilst others have ran the SQL update script, or installed a later version of Climsoft and therefore have the latest version.

For testing:

• Version 4.1 SQL can be found here.
• Version 4.1.1 SQL code can be found here

All recent versions are using the v4.1.1 version of the database (there have not been any more schema changes since 4.1.1).

There are actually 3 combinations to test:

• v4.1
• v4.1 + update script
• v4.1.1

Hopefully v4.1 + update script is identical to v4.1.1.

[isedwards]

@Patowhiz @maxwellfundi - Does anyone have any time to look at this over the next few days?

We were discussing making a v4.2 release to include the French language translation work for Marcellin's visit to Haiti this week.

[Patowhiz]

I'm looking into this

[mhabimana]

@Patowhiz : Have you found something?, @smachua : Have you checked if the database for Rwanda and Uganda is updated and contains all the latest database features (V4.1.1)?

[smachua]

@mhabimana, in Rwanda they reverted back to V4.1.4. Uganda wanted to do the same but the country lock down caught up with them before doing it. I may revert to the earlier version of the forms since there were other improvements in V4.1.5 that will benefit the users. Both countries have the updated database.

[Patowhiz]

@mhabimana @smachua I checked this. It works fine. All the connections work fine too. Can you share their database script file? That could be the reason I think, because we reviewed the forms before merging. It can't be the entity framework migration because it's no longer used anywhere in the code. I think there could be an issue when running the migration script files, again I can only check this if those files used in those countries are shared so that I get to identify what exactly is causing the error. I'm happy to investigate this further once those migration script files are shared.

@mhabimana from the error screenshot you are showing, it's clear to me that a select statement is not properly setup, this could be due to wrong column name or table name etc. Caused by changes in the database structure. Which again strongly suggests that this is caused by the migration script files. That's why the fresh installations are working, another strong evidence I think.

@smachua which other improvements. I think the latest versions of the forms have more features than previous versions, most of the features were requested by @mhabimana. When I went through the forms while adding the features requested I noticed a few code additions from you(which did things like validating double data entry and altering the database sstructures) which was not consistent with the custom controls implementations, I wrote issues about them.

@mhabimana the script files used, are they the same as the ones @isedwards has shared above?

[smachua]

@Patowhiz thanks for your suggestions. I will further investigate whether Rwanda and Uganda updated their databases with the upgrade script. It was made available to them as soon as it was created. When they resume work we shall do it together then report any issues that may arise.

Meanwhile I suggest that you implement error handling exceptions in most functions and with appropriate error messages to avoid occurrence of 'Unhandled exceptions'. The error message shown above was a bit misleading because the column 'visUnits' was actually present in the table.

The code you're referring to is meant to call functions in the class 'dataEntryGlobalRoutines' that perform operations such as repeat data entry among others. Those functions were designed before implementations of custom controls. Until alternative methods of replacing them are implemented I suggest that they are left to operate. Otherwise those users already using them (e.g. KMD on repeat data entry) will reject any upgraded version without those operations.

[isedwards]

@mhabimana - do you know whether it may be possible to get a copy of the current Rwanda database schema by connecting remotely from outside the office?

They can do a database dump with the -d or --no-data option to return just the table definitions without any data, this would be ideal for investigating.

mysqldump --no-data -u admin -p mysql_climsoft_db_v4 > rwanda_schema_backup.sql

[mhabimana]

@isedwards - Thanks, Yesterday myself and @smachua had a Skype +Teamviewer with @fmucyo ( Meteo Rwanda) and they managed to upgrade to V4.1.6 without any problem. So,I don't understand what went wrong last time. Since this issue has been resolved,is it still important to get the DB schema?

[isedwards]

Well done. Good to hear that it went well.

Did you run the update scripts on the database? If yes, we think this may be what was missing (which would explain why it is working now without problems).

I think now that it is working there is no need to get a copy of their schema.

[mhabimana]

@isedwards - Thanks and noted. We did not touched on the DB,We only installed V4.1.6 upgrade and all worked well.

[Patowhiz]

In addition, it will be much helpful in future to have the update script files done through github as an issue and PR. So that the team can test before merging. That makes it easier to trace the bugs. @isedwards @mhabimana @smachua in regards to the database schema, I'm assuming the Rwanda schema is the same as the ones we have on github? If that's so then there's is no need to have, if not we may need it to further test the current version to check any possible bugs for future installations, as we continue working on version 5.

[mhabimana]

@Patowhiz -Thanks,sounds like a good idea to have the script done through GitHub in future. The schema in Rwanda is the same as the one on GitHub,i.e: two scripts (mariadb_climsoft_db_v4_all.sql and mariadb_climsoft_db_v4_upgrade.sql).

[smachua]

@Patowhiz whenever I update the database schema I merge it into the GitHub. Also whenever I compile the Climsoft upgrade setup I do include the upgrade database script file for users to run it. I am on the opinion that future database updates be done through the code so that we eliminate the chance of being skipped or done wrongly by the users.

I am still investigating what could have happened in the case of Rwanda noting that they have 2 separate database schemas one for manual observations while the other is for AWS. We shall look at those schemas once they fully resume work.

[mhabimana]

This issue can now be closed.

[smachua]

@mhabimana and @Patowhiz I wish to reopen this issue because it has not worked for all users. Now that Uganda Met has resumed data operations they tried to update the database with the upgrade script but the problem still persisted. Through Teamviewer I tried and confirmed that forms can't open but show the same errors described above. I exported their database schema as attached here below. Surprisingly when installed in my machine it worked ok. So what could be the problem.

uganda_db_schema.txt

[mhabimana]

@smachua -Thanks. Very interesting.

[mhabimana]

@smachua- I am trying to understand this problem and wonder if you can you give us more details on their server? (which Operating System (Windows or Linux?) and version?).e.g: Windows server 2012.

[smachua]

@mhabimana it's Windows but not sure of version. Why do you look at from that angle? I know it has been used on the Climsoft database for a couple of years.

[mhabimana]

@smachua - Thanks,but the script you have shared does not have any data entry form. Can you share a complete one?

[mhabimana]

@mhabimana it's Windows but not sure of version. Why do you look at from that angle? I know it has been used on the Climsoft database for a couple of years.

I am looking at this angle because, the same software is working on some systems and doesn't to others.