search

farmOS / field-kit

A modular, offline-first companion app to farmOS.
https://farmOS.org
GNU General Public License v3.0
60 stars 39 forks source link

farmOS 2.x support #419

Closed jgaehring closed 3 years ago

jgaehring commented 3 years ago

We began talking this week about how and when to start integrating the farmOS 2.x API into Field Kit so we can support that version as soon as its ready.

Main considerations

The way I see it, there are a few major considerations, which each have their own minor considerations to attend to:

Upgrading internal & external libraries

The two major pieces of this are farmOS.js and farmLog. In my mind we'll basically need two completely different versions of each of these in order to handle the differences.

The main issues for farmOS.js are laid out pretty well by @paul121 in https://www.drupal.org/project/farm/issues/3151243.

For farmLog, it will be the difference of parsing logs with the data_schema field currently provided at /farm.json versus using JSON Schema for 2.x.

Beyond those two libs, there will likely be changes in how we call the server to begin with, assuming there will be different endpoints and changes to the methods on farmOS.js, as well as how we cache the schema, etc, etc.

I'd like to hide a lot of these details from the individual Field Modules if possible, but that might not be, especially if there are changes to the overall structure of logs in 2.x. @mstenta, curious to know what kinds of changes you anticipate, apart from what's in the API Changes doc you've started. The done field is prominent in my mind as an example of this. I don't see much way around Field Modules needing to be aware that they could either be handling a boolean or enum, based on which version of farmOS the log came from.

Data migration

We've talked a little about this, and we're not sure if it's worth having a thorough migration strategy, or just requiring users to sync their logs before upgrading, then nuking the database and starting fresh with 2.x data. That way the migration can essentially be handled by the server.

Version detection & activation

The detection part of this should be pretty straightforward, just pinging the server at some endpoint to get its version, although I'm not sure if we have a precise plan for what this will look like (@mstenta?). What I'm calling the activation part of this is where we actually "activate" versions of farmOS.js and farmLog to work with the detected version. I'm wondering if it could be desirable to lazy-load these libraries only after the version has been detected. Might not gain us much on bundle size, and may slow things down a bit, so maybe not worth it, but worth considering.

Also, I'm a bit worried about how we will handle fresh installs, before they've connected to a server. I'm thinking again it might be worthwhile to nix the ability to use Field Kit w/o first logging in. I generally think unauthenticated use is a bad practice, considering how much is contingent on the server config, and won't be sustainable as we become more committed to the Field Module architecture. This seems like one more good argument against it.

User experience

I think this is a really important consideration, and one that might even save us some grief on other fronts. For instance, if we do decide it's just as easy from a UX perspective to nuke the local DB instead of devising a thorough migration strategy.

Along with other UX concerns is the issue of when and how we sunset v1.x support. I'd like to drop redundant parts of the codebase as soon as possible, but perhaps after a while we could host a separate legacy version, something like https://v1.farmos.app for folks that are slow to migrate.

There's also the issue of how we version Field Kit. Since we still haven't released 1.0, I wonder if we want to skip straight ahead to 2.0? Or maybe we release 1.0 and 2.0 together?

First steps

The first step will probably be rewriting farmOS.js and farmLog. I think I'd probably like to take this opportunity to incorporate farmLog into farmOS.js, so it's a more comprehensive library for both connecting to a farmOS server and processing the data that it sends and recieves.

More steps to come, and perhaps a checklist, as we weigh all the considerations above and figure out a roadmap for this.

jgaehring commented 3 years ago

The docs for setting up a 2.x server for local testing: http://2x.farmos.org/development/

jgaehring commented 3 years ago

Oh, also worth noting that I started a farmos 2.x label a little while back, for issues that would need to wait til we had 2.x support. I'm adding that label to this issue as well.

mstenta commented 3 years ago

Thanks for starting this issue @jgaehring !

A lot of the Field Kit 2.x considerations have been simmering over the past few days. I'm trying to think of ways that we can make the whole process as easy as possible, both for you and Field Kit users.

We talked about some ideas related to dual-support of 1.x and 2.x in farmOS.js and Field Kit. The more I think about this, the more of a headache it seems like it will be.

So then of course my mind goes to the alternatives...

What if we simplify the requirements...

What if (and bear with me for a moment here), we decide to draw a line in the sand. We tag 1.0.0 of Field Kit as it is now (after you finish what you're working on), and say "this is the farmOS 1.x compatible version - all future development will be aimed at farmOS 2.x". Then, we open a new 2.x branch, focus on 2.x support, JSON schema, etc, before picking back up where we left off on feature development (quick forms conversion to field modules etc).

There are some advantages and disadvantages to this approach...

An obvious advantage is it greatly reduces the mental overhead of having to support both 1.x and 2.x simultaneously. farmOS.js and Field Kit could both start 2.x branches and officially declare that all focus is on 2.x moving forward. That thought alone feels like a big weight lifted in my mind.

The obvious disadvantage is that it couples new Field Kit feature development to 2.x - so users won't get new features until that's available.

But, like I said in chat, we can consider releasing a farmOS 2.x alpha that has everything Field Kit needs before we finish a lot of the other 2.x stuff (like CSV importers, UI/UX features, etc etc). So it would actually be possible for people to start using farmOS 2.x + Field Kit even before we officially release. And doing so may actually incentivize us to think harder about what we can move into Field Kit and/or what farmOS server UIs we can consider building with the tools you're developing in Field Kit (the "Field Kit core" ideas).

Another advantage is JSON Schema would be available, so you could convert 100% to using that instead of also maintaining the 1.x logic.

Just in general... the decision to draw a line and say "Field Kit 1.x works with farmOS 1.x, 2.x works with 2.x" would simplify SOOO much, I think. It just means accepting that there will be a period of time where you are developing new features that users will have to wait to use. But in doing so, it would also encourage us to really focus on getting farmOS 2.x to a finished place ASAP.

It's a bit of a "drastic" departure from our current plans and roadmap, so might take some time to digest and think through. Curious to hear what you think. :-)

jgaehring commented 3 years ago

Great thoughts, @mstenta!

I'm trying to think through the possible release management and deployment strategies now, so what about this...

I'm going to assume an absolute latest rollout date for farmOS 2.x of April 1, 2021. This seems to me like the latest time for us to rollout changes while it's still easy for farmers to adopt new features (and modules) as a part of their regular workflow.

Working towards that, I'm thinking of the following timeline:

* = Alternatively, we could decide we're simply not going to release 1.0 to the app stores at all, and just wait til 2.0 for native versions. This way, we won't have to release 2.0 as a separate app when the time comes, and we can just instruct users to use the PWA at https://v1.farmos.app, similar to how we'll do it from https://farmos.app. It would also allow us to forgo maintenance of the native apps, which could be ideal while we sprint on getting 2.0 ready for release.

jgaehring commented 3 years ago

That timeline above is totally tentative, just thought it could be useful to have some concrete dates laid out while we evaluate what's possible and how we want to proceed. Obviously much more discussion is needed.

jgaehring commented 3 years ago

Also note, with regard to my "Main Considerations" at the top of this issue, that since this timeline doesn't provide dual-support, obviates the need for version detection/activation and data migration. The user experience is all handled by having separate deployments. So the only remaining consideration is the upgrade of farmLog and farmOS.js, along with whatever small changes we might need to make to the rest of FK to reintegrate those libraries. I think 4 weeks (5, minus 1 for the the holiday) is a reasonable chunk of time to achieve that. If it runs over, I think we can probably just cut back on how much we do for the first sprint of Field Module development. We could even nix that whole sprint if we had to.

jgaehring commented 3 years ago

I'm closing this for now. I think we've arrived at the decisions we needed to make re: 2.x and can track as smaller issues. Pertinent details from the above timeline have been moved over to #373 and the milestones, and we can always refer back to this as needed, but I don't see much need for further discussion here.