Plans for Korp after AngularJS end of life?

[janiemi]

I noticed that the AngularJS JavaScript framework that the Korp frontend uses reached end of life at the end of December 2021, so I’m wondering what are Språkbanken’s plans for the Korp frontend in this respect.

Do you plan to migrate Korp to use some other framework, such as Angular, React or Vue.js (or some other)? Would that also involve migrating to TypeScript, at least in the case of Angular? Migration could perhaps be carried out gradually using some migration libraries, as mentioned in e.g. this post. Or will Korp stay roughly as is or will it perhaps be improved little by little, using the no-longer-supported AngularJS? Or do you intend to rewrite Korp completely using one of the more modern frameworks, or perhaps replace Korp (at least the frontend) with an entirely different application?

From the little I’ve read, Angular, React and Vue.js all claim to be significantly faster than AngularJS, and they work better on mobile platforms. However, I have no experience whatsoever with any of them (nor with any other JS framework), so I cannot judge.

More generally, I’m also wondering what is the status of Korp among Språkbanken’s tools or services. As far as I can see, it hasn’t been developed recently as actively as some years ago. I also think that some quite noticeable bugs have been present in Språkbanken’s Korp service for quite some time. However, I acknowledge that moving corpus configurations to the backend is a major undertaking under work, even though it might not make much difference to users.

Does Korp in general serve its purpose well enough, so as not to require much attention or development? Or do users need other kinds of services more than a concordance service like Korp? Or is Korp so outdated technically and perhaps so complicated that it makes little sense (or is no longer fun) to invest much development effort in it? Or is Språkbanken’s focus now in other tools and services, perhaps involving or advancing more research? I acknowledge that you have quite a few tools and services developed in-house, of which Korp is only one, so you need to prioritize, too.

In the end, I’m wondering how much development effort it makes sense for us in FIN-CLARIN and the Language Bank of Finland to invest in the current Korp that uses an outdated JavaScript framework, in particular if it will be replaced with a completely different application in the future. We wouldn’t like to spend too much time in implementing something that will be obsolete or require a complete rewrite in a year or two, even though some of the work could probably be reused.

As I see it, Korp is one of the key services in the Language Bank of Finland, in which we have invested quite some effort, so we’d very much like to see it developed further and eventually migrated to a supported framework, unless replaced with something better. (We are now finally about to catch up with Språkbanken’s Korp, with hopefully better ways of implementing our site-specific features and modifications.) We have quite a few wishes for improvements or new features in Korp, of which we intend to implement at least some ourselves and then contribute to the common codebase. (I think the contributing procedure now works reasonably well, at least from my point of view.)

However, our resources are rather limited: I’m practically the only one somewhat acquainted with the actual Korp code, and I’d also have other work besides improving Korp. And even though I can modify Korp code in many places, my understanding of its architecture, the framework and the other technologies used is very limited, and I don’t often know what would be the best, the right or even a good way to implement something. We’ll thus most likely need support and advice from you, Språkbanken’s Korp developers, every now and then in the future, too.

[majsan]

I haven't seen that Angular.js is end of life, but it's good to know!

We plan to migrate Korp to Vue.js in the future. This is because most of our other frontends have been developed in Vue. My goal it to continue refactoring Korp to use more components and less of the other Angular.js-specific functions (controllers, services, directives). I believe that it will get easier to move to Vue.js after. A problem with the frontend architechure is that it was first a jQuery application, then parts were ported to Angular.js, and then different libraries and frameworks with the same features were added and are still there. Things that are the same are done in different ways (model.js vs. Angular services for example). The frontend code needs to be simplified if we want it to survive the years to come.

Språkbanken wants to develop the frontend more actively than before, so that's good news! But it is only I who work on the frontend right now. I will work on new features and refactoring in parallel, and hopefully we will get to a state were moving to Vue.js seems feasible without doing a complete rewrite.

[janiemi]

Thanks, @majsan, for the information! I find it really great news that Språkbanken wants to keep developing Korp actively. It gives me confidence that we’re not betting on a nosediving Korp. :) The eventual migration to Vue.js also sounds promising, even though it probably means some more work for us, too. And it’s really great if the code is made simpler.

I’m also wondering if the Korp backend will also be developed further, as the last commit in its GitHub repo was made two years ago, but I guess I should ask that in the backend repo issues, or directly from @MartinHammarstedt. (I should have got back to him over two years ago on backend matters, but never did. :( ) I’ll try to do that at the latest when we finally complete the transition from Korp 5 to Korp 9 (backend 8), which has taken much longer than I expected, within a month or two, I hope.

I still have a few relatively minor modifications to Korp (both frontend and backend) waiting for me to take the time to create pull requests from them. Many of them just make Korp more configurable for other environments than Språkbanken’s, but I suppose some of them should be useful to you, too.

If you have any advice on what should be preferred or avoided when making modifications to the current Korp frontend code in the light of the eventual migration to Vue, I’d be grateful to know that.

[majsan]

I know that @MartinHammarstedt will also have some more time for the backend now, but it's best to discuss with him, yes.

The advice is, try to use ES6 imports (`import { myFunction } from './my_lib') whenever possible. And use components or directives, and nothing else from Angular (if possible). Structure files as ES6 modules, see https://github.com/spraakbanken/korp-frontend/blob/dev/app/scripts/statemachine.js for example.

[janiemi]

Thanks for the advice. I think I’ve used rather few larger AngularJS constructs (no new services or the like), but templates contain ng-if, ng-show, ng-repeat, {{ … }} and such.

[janiemi]

Yet another, minor question: Will the migration to Vue.js affect the automated testing of Korp?

I think it would be good to have comprehensive automated tests for Korp, so that you could catch bugs introduced by changes early on, but at least I find such tests tedious to write and maintain, so it would be great if that could be made easier. I haven’t written a single test for our Korp modifications, partly because I have felt the learning curve too steep, partly because lack of time, and partly because the few existing tests haven’t seemed to be actively maintained.

[majsan]

ng-if, ng-show, ng-repeatand {{ .. }} are OK to use, since there is corresponding functionality in Vue.js and are easily translatable.

The migration will affect automated testing. I agree that the tests are very helpful, so this is something to consider.