search

sailfish-keyboard / sailfishos-presage-predictor

Presage based input predictor for the Sailfish OS
https://openrepos.net/content/sailfishkeyboard/maliit-plugin-presage
GNU General Public License v3.0
7 stars 4 forks source link

Public release #28

Closed rinigus closed 6 years ago

rinigus commented 6 years ago

Writing it up as an issue, so we can tick it off when all is done. In my mind, we are ready to release, code and database wise. So, we should start pushing it out and getting releases to the users. Todo list

Version: Earlier was 1.0. All options are good with me: 1.1, 1.5, 2.0. We should use a.b.c notation though

When plugin is ready, let's upload keyboards

martonmiklos commented 6 years ago

Version-wise I would prefer 1.1.0.

rinigus commented 6 years ago

Let's stick with 1.1.0 then. As a Changelog, I propose

I don't think I missed anything, if I take into account last release

martonmiklos commented 6 years ago

Looks correct.

rinigus commented 6 years ago

Would you do the release :) ? I'll be traveling next 4 days and will not be able to help much during these days. But I can build on OBS as soon as you release, so the RPM will be available.

rinigus commented 6 years ago

Great, thanks for release! And OBS builds are done, you can get them at repos linked from https://build.merproject.org/project/show/home:rinigus:keyboard . Now we need to test as much as we like and then release [but not tooo much... :) ]

@martonmiklos - when you feel comfortable, please upload the RPMs to OpenRepos. I guess we need to push out plugin and English. For me it will be harder to do while away. Then I'll deal with Estonian as soon as I can.

martonmiklos commented 6 years ago

Github release done. I have pimped a bit the README.md, please take a look on the #29 PR. It might be useful to add links to the openrepos as well.

rinigus commented 6 years ago

I guess we will add links when OpenRepos packages are uploaded.

What I forgot to mention is that I will keep now this OBS repo for official releases. Its pointing now to sailfish-keyboard repos. The development will be done then via subprojects. So, we can at least grab our builds over there any time.

martonmiklos commented 6 years ago

I have started to adding the applications to the openrepos, I think links to the application is enough to the README

Just to doublecheck we want to release the following packages:

rinigus commented 6 years ago

I would suggest to release also

Which brings the question: how to organize packages. Users will be installing keyboards only, I think. Keyboards have dependencies on plugin, hunspell dictionary, and ngram marisa database (as in https://build.merproject.org/package/binary/home:rinigus:keyboard/presage-lang-XX?arch=armv8el&filename=keyboard-presage-en_US-0.2.0-1.noarch.rpm&repository=sailfish_latest_armv7hl). So, keyboards are probably each as a separate package in OpenRepos. Plugin probably as well. However, I don't think we need to do that for dictionaries and ngrams, or do we?

Additional note: all noarch packages are in double - one in ARM one in i486 architecture repos. Its sufficient to pick any of them, they are identical.

martonmiklos commented 6 years ago

Users will be installing keyboards only, I think.

That's correct.

Keyboards have dependencies on plugin, hunspell dictionary, and ngram marisa database. So, keyboards are probably each as a separate package in OpenRepos.

I agree.

Plugin probably as well. However, I don't think we need to do that for dictionaries and ngrams, or do we?

I am not sure I understand what you mean here.

rinigus commented 6 years ago

Question is whether each RPM needs separate "app page". However, maybe it does. While zypper will be fine if we put all dictionaries into a single "app page" as several RPMs, OpenRepos clients may not like it. Let's make a page then for each of the dictionaries. If you make the first example, I can add Estonian already now, in parallel. Or a tomorrow morning

martonmiklos commented 6 years ago

Ah I see your point. The old Warehouse app got deprecated by the 2.1.4 and I doubt that Basil will upgrade it. Storeman does not list libraries on the main page, so I think it is safe to assume that noone will find it accidentally (not by purposedly going to the libraries section) and try to install the "hidden" components.

So I would propose to create a page for the hunspell-lang and one for the presage-lang and of course one for the plugin. Keyboards will be separated.

rinigus commented 6 years ago

Let's do it that way. We can always change it later if the updates will not get propagated in the dictionaries/lang databases.

martonmiklos commented 6 years ago

Okay. I will start uploading the packages and adding some readme to the openrepos as well.

martonmiklos commented 6 years ago

Openrepos says this to the uploaded file:

For security reasons, your upload has been renamed to maliit-plugin-presage-1.1.0-10.27.1.jolla_.armv7hl.rpm.

The original filename was: maliit-plugin-presage-1.1.0-10.27.1.jolla.armv7hl.rpm I have no idea what is the security reason behind it, maybe the jolla section would be reserved?

rinigus commented 6 years ago

No worries, its a normal message for the packages build at OBS. They will work absolutely fine when installed and we don't need to rename anything.

martonmiklos commented 6 years ago

Ok thanks for the clarification!

rinigus commented 6 years ago

But why this message is displayed - I don't know. Just learned to ignore it :)

Its getting late over here, have to catch some sleep. Good luck with the uploads!

martonmiklos commented 6 years ago

I have started to write a message to Basil just for curiosity I will let you know the results. Have a good night!

rinigus commented 6 years ago

Thanks! Let me know when you get the reply

martonmiklos commented 6 years ago

I got a quick reply from Basil:

2) one of "sub-extensions" of the file are not in whitelist, but match general security regexp which prevent uploading files, that could potentially be used for site exploiting.

In this case, "jolla" matched that regexp.

Generally, that renaming does not affect repository content, or single file download process. I'll whitelist it so there should be no problems in the future.

I have uploaded everything, please read through the descriptions of the packages if time allows.

rinigus commented 6 years ago

Thank you very much! I'll read the descriptions bit later today

rinigus commented 6 years ago

For keyboard descriptions, I would suggest to

"This keyboard layout utilizes an open-source text prediction solution (not the Xt9 engine shipped by the Jolla). It allows you to use it on community supported language pack or if you have a ported device, in addition to the official ones."

For hunspell dictionaries, we may add (but maybe we don't need it here):

Provided dictionaries all use UTF-8 encoding that is expected currently by Hunspell predictor.

For plugin page, I would suggest to add the same description as for keyboards in the beginning.

Its great job, thank you very much @martonmiklos !

We also would need to bump a thread at TJC and I suggest to write an announcement on TMO.

martonmiklos commented 6 years ago

Thanks for the feedback I will fix them at the evening.

rinigus commented 6 years ago

@martonmiklos - are you using at TMO? Would you like to write an announcement there about the project? I can do it as well, or maybe @ljo?

martonmiklos commented 6 years ago

I have an account there, but I am not active. Before we do a mass advertisement I think it would be useful to do an installation on a cleaned device from openrepos. I have a spare JP1 for this purpose but I can do this only tomorrow. If you have a clean device at hand could you please give it a try?

rinigus commented 6 years ago

@martonmiklos - good idea! I don't have any clean device though :) . There were some complains regarding plugin install at https://openrepos.net/content/sailfishkeyboard/maliit-plugin-presage#comment-23961 . Although, when I wiped off my install (keyboard, plugin, hunspell and ngrams), I couldn't reproduce it.

So, yes, please check as well on clean device and then we proceed with the announcements

rinigus commented 6 years ago

PS: I started reading up on ICU and Unicode.

martonmiklos commented 6 years ago

I have just made modifications on openrepos in the description and title fields, and came across that.

ljo commented 6 years ago

@rinigus @martonmiklos I have tested clean installations on Xperia X, Jolla C and Jolla Tablet. It all worked very well in the evening, but there might have been some openrepos hickups during the day. I have a TMO account, but today I have a looong backlog of stuff to write for work already. So it might be a bad choice timewise to let me write the announcement.

martonmiklos commented 6 years ago

Okay, so I consider the clean validation outsourced and done. Thanks for doing that! Regarding to the announcement: I have quite a few tasks for this evening, if I get some time I will put together some announcement text.

rinigus commented 6 years ago

So, we are lacking instructions for old users and, according to comments in https://openrepos.net/content/sailfishkeyboard/maliit-plugin-presage , can cause some damage. Which was rather strange, though

martonmiklos commented 6 years ago

Just reading through the comments on openrepos (auch). I will install the legacy stuff on my JP1 and give it a try to install the new one.

rinigus commented 6 years ago

Awesome, thanks for doing it! I think the trick is to disable Presage keyboards first, uninstall all presage packages, and then install new. This should work then...

ljo commented 6 years ago

@rinigus @martonmiklos I will try to verify on J1 with old installation and activated keyboard then too tonight. I did a transition package for the Swedish keyboard already March 8. So I will see if there is some difference affecting the instruction for users with previous version installations.

rinigus commented 6 years ago

Hi, how did the tests go? Was it as bad? Do we have a solution?

rinigus commented 6 years ago

Folks, I looked into Unicode and started hacking it, but then stopped rather early. Its a lot of work to go through presage and reimplement all via UnicodeString. Since the current keyboard works quite well for the languages I'm interested in, I prefer working on few other things to catch up with the issues in other projects (map server mainly). There is also a hope that after public release, we can find someone who is interested in proper Unicode support (Russian language, for example) and can join us.

ljo commented 6 years ago

It varied. But deactivating the keyboard seem to make most problems with deinstalling go away, but not all. And, to be clear, on most devices there was no problem deinstalling and upgrading even with the keyboard active. It just made another keyboard in the selected list the current active one. Only looking at deinstalling from within Storeman, the hungarian keyboard failed the most times. On installation of hunspell I saw in one case of where the libhunspell version (1.12) brought in by sailfish-browser being replaced by the provided 1.16. That lead to sailfish-browser not starting unless the library soft links were added pointing to the newer version. I am not sure this could happen for someone using Storeman, but if, it would be bad without the feedback.
Installation by using pkcon and zypper there were a lot of choices on installation, but selecting the one breaking dependencies always worked.

martonmiklos commented 6 years ago

Storeman hangs at my device, and since it was upgraded 2.14 using Warehouse is not an option. When I select the En-US keyboard for installing it starts, and then it exit. I have not debugged it deeper because I got distracted with other (non predictor) problems.

rinigus commented 6 years ago

@ljo: thanks for testing!

To my understanding, we don't provide hunspell library anymore, its linked during a build and is not available for installation from our repos directly. Or did I managed to miss it?

As for zypper / pkcon install - I would expect that choice with deinstallation of older packages should work the best.

In the instructions, we could also specify that users should uninstall old components first. Also, we should remove old packages from OpenRepos (I can see that they are still available).

ljo commented 6 years ago

@rinigus Maybe the cache was not clean on the libhunspell issue then (I only saw it once). The general instruction should definitely be uninstall old versions. But there could be some hiccups doing that. Try deactivating the keyboard (works in some cases) or try refresh the repo cache and use pkcon/zypper to find out more about the issue if no error message is given in Storeman.

rinigus commented 6 years ago

So, in this sense, we could suggest the following "safe" uninstall instructions:

I would expect it to work in most cases. Anything else we can add tp make it safer?

We should remove old packages from OpenRepos though

ljo commented 6 years ago

I did remove the packages for English and Swedish leaving a message in the repos:

"We have moved to a joint community repo for all language-resources, keyboards and library versions sailfish_keyboard. Please uninstall your old packages and reinstall the Swedish/English resources from there.

Thanks,

ljo"

Pointing to the respective Swedish and English keyboard for each.

rinigus commented 6 years ago

@ljo - Great, exactly what's needed :)

rinigus commented 6 years ago

What is a general feeling, are we ready for announcements at TMO and bumping thread at TJC with the uninstall instructions? I have a feeling that we are...

martonmiklos commented 6 years ago

I have the same, I have not been able to investigate my problems further. I have put some nice warnings and instructions to the description of my legacy presage related packages on openrepos, I think we should add some in the announcement as well.

rinigus commented 6 years ago

@martonmiklos - good warning, very visible. Maybe also the packages have to be removed, as @ljo did. He removed RPMs and left only package description.

@martonmiklos, so are you making the announcements? would be very appropriate, I think.

martonmiklos commented 6 years ago

Removing packages makes sense -> done.

Alright here is what I think about the announcement text:

[Announcement]Open source text prediction support

The sailfishos_keyboard development team is happy to announce a new version from the presage based alternative text prediction solution.

Currently we offer the following keyboards for downloading at openrepos: https://openrepos.net/content/sailfishkeyboard/english-us-keyboard-layout-presage-based-text-prediction https://openrepos.net/content/sailfishkeyboard/swedish-keyboard-layout-presage-based-input-predictor https://openrepos.net/content/sailfishkeyboard/estonian-keyboard-layout-presage-based-text-prediction

If you are using a community supported language pack or a ported device you should definitely try it out.

If you cannot find your language in the supported list please read through this documenation about how can you add support for your language: https://github.com/sailfish-keyboard/sailfishos-presage-predictor/tree/master/utils/keyboard https://github.com/rinigus/presage#generation-of-n-gram-database-for-marisa-based-predictor

If you are having problems with the packaging or distribution at openrepos let us know (preferabliy in the form of a github issue), we are more than happy to help with it.

In this release we have been mainly focusing on performance improvements. We are satisfied with the results thanks to moving the internal database format from SQLITE to MarisaDb and the asynchronous prediction handling.

Other than the performance improvements we have implemented the following:

Important message to the users of the former revision Because of the structural changes please follow the instructions below before installing the new plugin:

Please report back if you have any issues, problems!

Wishing you happy sailing, rinigus, ljo and martonmiklos

Feel free to review and comment if I made a typo, should change it to more formal, missed something, etc. As deadlines always working if no objections I will post it on TMO and TJC tomorrow ~noon.

rinigus commented 6 years ago

@martonmiklos - great text, thank you! Just few corrections:

--

We have started a new sailfishos_keyboard development team at https://github.com/sailfish-keyboard. The team is happy to announce a new version from the presage based alternative text prediction solution.

Currently we offer the following keyboards for downloading at openrepos: https://openrepos.net/content/sailfishkeyboard/english-us-keyboard-layout-presage-based-text-prediction https://openrepos.net/content/sailfishkeyboard/swedish-keyboard-layout-presage-based-input-predictor https://openrepos.net/content/sailfishkeyboard/estonian-keyboard-layout-presage-based-text-prediction

If you are using a community supported language pack or a ported device you should definitely try it out.

If you cannot find your language in the supported list please read through this documenation about how can you add support for your language:

If you are having problems with the packaging or distribution at openrepos let us know (preferabliy in the form of a github issue), we are more than happy to help with it.

In this release we have been mainly focusing on performance improvements. We are satisfied with the results thanks to moving the internal database format from SQLite to MarisaDb and the asynchronous prediction handling.

Other than the performance improvements we have implemented the following:

Important message to the users of the former revision Because of the structural changes please follow the instructions below before installing the new plugin:

Please report back if you have any issues, problems!

Wishing you happy Sailing, ljo, martonmiklos, and rinigus

martonmiklos commented 6 years ago

Acked, I have just packaging and testing the Hungarian keyboard. ljo's instructions seems to be working, at least I have managed to install the new packages. Something is weird because I have two active keyboards: EN and the presage Hun, but if I select Hun it will fallback to EN while giving Xt9 En predictions.