Open shankari opened 1 year ago
asiripanich
commented
1 year ago Yayyy! I don't know too much technical details on these mobile app frameworks. So my only vote is that we choose a framework that is widely used, has a strong adoption rate, and not going to be obsoleted any time soon. This would make hiring someone to work on it less challenging which is unlike the current pairng of Ionic and PhoneGap (?) that we chose.
shankari
commented
1 year ago @asiripanich ionic and cordova (phonegap was closed source and is EOL, cordova is open source and is still maintained) Note also that the two aspects are potentially decoupled - e.g. you could switch the HTML+JS+CSS to react/vue while keeping the native code as cordova, or switch to React Native, which would also need to change the interface layer of the plugins, but then we can't as easily switch to Vue later, for example.
I suspect that the biggest challenge that you are facing is with the ionic1/angularJS UI (other than @ericafenyo nobody has ever contributed native code). AngularJS is also EOL
JGreenlee
commented
1 year ago To narrow down our choices, I think the next major question is whether we want A) a truly native app, or if we're ok with B) a web solution packaged as a native app.
For (A) we'd be looking at React Native (unless anyone is particularly enthusiastic about Flutter)
For (B), we have countless options.
The door is wide open for web frameworks but we're probably looking at React or Vue.
We can use a newer version of Ionic, or we can ditch it altogether
We can keep Cordova or we can switch to Capacitor
lgharib
commented
1 year ago | List of criteria | Angular (Web Components) | VueJS (Web Components) | ReactJS (Web Components) | React native | Dart Flutter |
|---|---|---|---|---|---|
| Keep up with the hype | No | No | No | Yes | Yes |
| Long term support | Yes | Yes | Yes | No | No |
| Portability | Yes | Yes | Yes | No | No |
We want to keep the hype to let the community interested in working on the project. We need to be able to refactor the mobile app every 5 years to stay in the flow. We have less then 10 people so we need to find the balance between a long term support community and enough people to work on the project.
TTalex
commented
1 year ago For reference, the 2022 stackoverflow survey, with the most "hype" frameworks https://survey.stackoverflow.co/2022/#section-most-loved-dreaded-and-wanted-web-frameworks-and-technologies
(For our sanity, we should look at the most loved techs, not the most dreaded ones, where we would find both angular.js and cordova :laughing: )
shankari
commented
1 year ago @TTalex that does seem to argue for React, but React webapp or React Native? Note also that flutter currently seems to be higher on the hype scale than React Native.
shankari
commented
1 year ago From the table that @lgharib created, a solution that might check all the boxes might be ReactJS (Web Components) on a React Native base. This would have the following advantages:
This will also support @TTalex's comment that there are likely to be lots of UI contributions and few native code contributions, so it might be better to keep the UI and the native code separate instead of integrated.
It looks from some light searching on the internet that this might be possible https://blog.logrocket.com/react-native-webview-a-complete-guide/ or https://github.com/meliorence/react-native-render-html
Does anybody with more experience with React Native (e.g. @lgharib) have thoughts on the feasibility of this approach?
This will also allow us to use the following migration path:
= get reactJS over react native
The only high-level con that I can see is that React Native is not part of the Apache ecosystem, so it is not clear what will happen if/when Facebook pulls funding for it.
Or maybe that this is the worst of both possible worlds instead of the best 😄
shankari
commented
1 year ago There is also this option https://necolas.github.io/react-native-web/ which supports the opposite use case; react native for web
Final thought, we can create custom react native components as well. Since we are going to modularize anyway, couldn't we create a react native version of the web components as well? It seems like the main difference will be in the JSX/rendering; the javascript parts can be the same
And honestly most of the complexity in e-mission/OpenPATH is in the javascript, not in the rendering.
JGreenlee
commented
1 year ago ReactJS (Web Components) on a React Native base
Interesting idea, but I can't find evidence of anyone who is actually doing this. We can theoretically place a parent webview in React Native - and have that act as a wrapper for ReactJS. But doesn't that largely defeat the purpose of using RN in the first place? In that case, the only reason we are using it is to interface with native APIs. This seems to go against the intended usage pattern of RN as a whole, and I worry that it would just make things heavier and more complicated than it needs to be.
I think the main appeal of React Native, to most people, is its native rendering. If we want that, then ok - RN is our frontrunner. Else, what do we envy about RN's native plugin support? Is there anything we wouldn't get from Capacitor?
We already have the existing Cordova plugins, which Capacitor would probably be backwards-compatible with. Capacitor also has its own library of native plugins and plugins from the community. The obvious downside is that Capacitor is less widely used. However it is apparently a bit more "loved" than React Native, among the people who do use it.
shankari
commented
1 year ago My 3 cents:
shankari
commented
1 year ago Since the high-level seems to have equally balanced pros and cons, at NREL, we are actually diving a bit deeper into a few alternatives.
The two alternatives are:
shankari
commented
1 year ago Wrt approach (1):
The big question that I have is whether they can make calls from the javascript layer to the native layer with this approach (aka re-implement cordova :) @TTalex do you know the answer? @JGreenlee and I will poke around the codebase as well, but presumably somebody at Cozy such as @PaulTranVan can give us a definitive answer as well.
JGreenlee
commented
1 year ago Regarding approach (2), I did a little bit of experimentation with WebComponents to see if it would be possible to gradually migrate components in-place. This approach is possible, but there are some caveats.
One of the things I realized is that the version of AngularJS we use (1.5) is not the best for WC interoperability. I've noticed migration guides will usually ask you to upgrade to 1.7 or 1.8 before anything else. However, I think we are stuck at 1.5 because that's as far as Ionic v1 ever supported.
The good news is that I found a few tools that can make this more seamless. ng-custom-element will allow us to bind angular scope variables to the properties of our custom web components directly in the HTML. Without this, the process would be a huge headache.
If we use WebComponents, I'd also like to use lit, which is a lightweight set of helper functions and a declarative template system, that sits on top of WC to make them a lot more useful. This way, its are features more congruent to AngularJS (and whatever framework we convert to later) and passing data back and forth is not so complicated.
Using those tools, I converted a couple components to test. I made a trip-list-item and a leaflet-map to replace infinite-scroll-list-item and leaflet-ui (an external angular directive that we use to show leaflet maps)
Once I found the right methodology, it worked pretty well. I didn't convert them all the way, but it was just an experiment and a proof of concept. We do have this as an option, and over the next few months we could gradually refactor into WC until we no longer depend on AngularJS and Ionic v1. Then we are free to use WebComponents in whatever framework we choose.
shankari
commented
1 year ago The big question that I have is whether they can make calls from the javascript layer to the native layer with this approach (aka re-implement cordova 😄 )
I poked around about this, and it looks like it is possible, but might be a bit tricky.
injectJavascript or onMessage/getMessage (https://github.com/react-native-webview/react-native-webview/blob/master/docs/Guide.md#communicating-between-js-and-native)
TTalex
commented
1 year ago This kind of bridge is uncharted territory for me. So I can't really help. I can however confirm that it is not something the coachCO2 is currently doing.
However, the app does have a kind of labelling support, but I believe it's missing the connection to the openpath server. Updated labels are not sent back to the server, they are stored somewhere else.
By the way, the app also has the beginnings of an automatic labelling system, it finds similar trips and applies the user input to all of them.
shankari
commented
1 year ago @TTalex as you know, we already built an automatic labeling system and it has been in production for a while. We also wrote a paper on it - it is a bit non-trivial to get super-good accuracy. We also just expanded the labeling system to support more complex surveys and changed the display mechanism to be more performant. I would really like to make sure that we are coordinating with Cozy, so that we don't waste our (limited) time implementing the same functionality over and over.
The whole premise of an open source community is that the whole is greater than the sum of the parts (<<tout est plus grand que la somme de ses parties>>) and that we can all contribute towards building a smooth, performant system that people want to use.
shankari
commented
1 year ago FYI, AirBnB tried react native and went back to pure native. https://medium.com/airbnb-engineering/react-native-at-airbnb-f95aa460be1c
Interesting post, but I am not sure if it is very relevant to our situation since it looked like they already had native apps and their concern was that they had to now support three platforms.
JGreenlee
commented
1 year ago To recap the approach that was put forth today, we can migrate in three steps:
1) Componentize: AngularJS -> Web Components 2) Reactify: Web Components -> ReactJS Components 3) Nativize: ReactJS Interface -> React Native interface
However, I thought about this a little bit afterwards. One of the reasons we originally considered componentizing with Web Components was to not be tied to any particular framework. But, it seems that if our ultimate target is React Native, this is a moot point. We are going to end up in the React ecosystem anyhow.
It is possible to do a soft migration from AngularJS directly to ReactJS. This seems relatively common. (Babbel, Awesense). I think that if the goal is to have a React Native app at the end of all this, we do not need to use WebComponents as a stepping stone. If step (1) can be eliminated altogether, we would likely shorten the timeline of the migration and could focus our efforts on other improvements.
JGreenlee
commented
1 year ago The second link from above implements its own solution to allow the use of ReactJS components inside AngularJS. Other guides recommend react2angular. A newer package, react-in-angularjs, seems to further simplify this.
shankari
commented
1 year ago As I said during the meeting, the current community membership seems to lead us inexorably towards react. However, I am still worried that, 5 years from now, we will have to rewrite everything in Dart because everybody is using Flutter. But if we pick a popular framework, I guess there will be some nice packages to help us along 😄
I do want to point out that we can create web components using reactJS now https://medium.com/javascript-by-doing/how-to-create-a-web-component-in-reactjs-62b71116ea36
That uses a wrapper library to wrap the reactJS component into a webcomponent without changing the react component code.
Now finally the interesting part! Let’s transform our component into a web component! To do this we will use the react-to-webcomponent library, used by many people and super stable.
I would really like to see if there is a way to expose the same component functionality as react native and web component. When we start making the real changes, I want to spend a couple of days poking around at the tool chain and seeing if that is possible, but not now...
shankari
commented
1 year ago I have to try this out, but it looks like react-native-for-web does exactly this "expose the same component functionality as react native and web component."
The only difference is that we had assumed that we would first write web components and then figure out a way to use them in react native.
With react native web, we write everything in react native (so the JSX has View, Image... but then use the framework to use the react native components directly as web components in reactJS.
Header.js to define a react native component called Header ending with export default Header;// add this after other import statements
import Header from './shared/components/Header';
// Modify the JSX in App component
const App = () => {
return (
<View styles={styles.screenContainer}>
<Header />
<Text style={styles.text}>I'm a React Native component</Text>
</View>
);
};This seems like it would allow us to reuse components for native and web and give us a non-horrendous migration path to other web frameworks in the future (potentially using react-to-webcomponent) if we wanted to.
JGreenlee
commented
1 year ago It looks like, at least from a UI standpoint, the migration is not going to be as hard as we may have thought.
I experimented with several approaches that would allow a soft migration to ReactJS and React Native. Surprisingly, I found a solution that allows us to migrate to React Native by only rewriting once. I didn't actually think this would be possible, but it is!
See the following:
|
Embedded here is the default "Hello world", or starter page, for React Native Web.
This was the first thing I did to test the possibility of rendering React Native components inside AngularJS. As shown, I was able to append this starter page to the Dashboard screen. It is exposed to AngularJS as ` |
|
You would probably not guess from looking, but the button showing "Add Activity" is rendered using React Native Web, while the "Mode" and "Purpose" remain rendered as normal HTML from AngularJS.
The button is defined as a React Native component as follows:
```jsx
const DiaryButton = ({ text }) => {
return (
);
};
DiaryButton.propTypes = {
text: string
}
// styles below ...
...
```
The `text` ("Add Activity") is passed in as a prop from the parent, which is an AngularJS directive. In the parent, it was a scope variable.
```html
|
Some things to note:
elevation. On the other hand, iOS uses four different shadow properties: (shadowColor, shadowOffset, shadowOpacity, shadowRadius). React Native on its own doesn't abstract this away, so it would be a pain to write and maintain anywhere we want a shadow.React Native Paper. It is a lot easier to get a unified look this way. I wouldn't normally suggest a third-party UI library, but in this case I think it is warranted.
AngularJS to React Native is a significant jump, and I somewhat doubt that anyone has done this exact process before. There are plenty of AngularJS -> ReactJS migration guides, and while it is largely the same in terms of actually rewriting code, this is a bit of uncharted territory for how to organize the project and its dependencies. Basically we are trying to bridge a gap of 10 years in Javascript evolution - it's messy. It took me several hours of fiddling around just to find the right combination of tools, with compatible versioning and dependency configuration. There are so many moving parts here, given that we will literally have 2 frameworks existing simultaneously in the same application. Also while having old dependencies from a legacy package manager (bower), and trying to use modern ES modules. It is the perfect storm of old and new Javascript!
The good news is that 1) after wrangling with it for hours, I now understand things much clearer, so maybe the initial headache is out of the way and 2) we can break the process into steps so it's not all too much all at once. I think we can prevent a lot of frustration by just having a solid plan for the prep work that we will do before we introduce React.
Given these findings, I created a roadmap of what specific steps I think will be necessary for this plan to work. This is obviously subject to change, but it should demonstrate my general vision of how we might tackle the migration.
TTalex
commented
1 year ago Good job @JGreenlee ! Very interesting read.
That looks like a sound plan to me :)
shankari
commented
1 year ago Agreed. Great summary, @JGreenlee! I am glad that we were able to fold in React Native for Web so that we have both portability and a modern toolchain.
I can see that this is a lot of work, but it honestly seems fairly doable once we have the initial templates set up. wrt "copy over our source files with git history", I think it is critical to retain git history, especially for a large and complex project with many moving parts.
It seems like just using git mv so that git log --follow will work correctly should be fine.
Alternatively, if that doesn't work properly, we can rename the current phone repo and archive it, create a new repo, and then just add a note to check history on the old repo if you get to the end of the new repo.
@lgharib @alirezaRa94 @ericafenyo any thoughts on this migration plan?
shankari
commented
1 year ago One more task for prep/refactor in AngularJS is to merge the master and master_for_platform branches.
Which means that we need to resolve https://github.com/e-mission/e-mission-docs/issues/850
Are we the only ones using a multi-tenant architecture?
shankari
commented
1 year ago @JGreenlee another consideration is the enekto survey code. We use the enketo survey even in the MULTILABEL case (for the demographic survey, and potentially to push targeted surveys to users) but enketo is going to remain javascript and is not going to become react native.
Some potential options for dealing with the surveys are:
JGreenlee
commented
1 year ago If there's a react native alternative that'd be great but I don't know of one. I was envisioning that we would just keep Enketo and wrap the surveys in a webview.
If we do keep Enketo I think we should try to get even with enketo-core so that we can benefit from upstream improvements.
JGreenlee
commented
1 year ago Moment is another library we might consider swapping out during the rewrite.
Moment is not deprecated, but they consider it a legacy project and recommend modern alternatives.
Luxon is the successor to Moment, which is more lightweight because it utilizes the Intl standard that is now part of modern JS.
I thought we might be able to use no library and only Intl, but it looks a little too barebones. we need some library to easily handle timezone offsets.
JGreenlee
commented
1 year ago There are at least 2 web-based libraries that we want to keep using, but use non-native (HTML) rendering: Leaflet and Enketo.
@shankari Before I go much further, I wanted to discuss how we plan to do this, because I think it's going to get messy. We can't put raw HTML in React Native Web and I thought it would be simple enough to just wrap these in a WebView. Although there is a way to do this, it's somewhat absurd: React Native Web WebView.
I actually burnt several hours trying to wrap a Leaflet map in React Native Web WebView. Eventually I realized that it is possible; just not on the iOS Simulator in the devapp / phonegap (<iframe>s simply don't show up. I searched this issue extensively and no one seems to know why or care).
So yes, wrapping things in WebViews is technically an option, but I think this would be a huge pain for development purposes - until we are able to officially ditch Cordova and move to Expo (but I think that will be some time from now)
For maps, we do have the alternate option to use react-native-maps/react-native-web-maps - which would probably give a smoother/sleeker result in the end anyway - but I think we'd have to use Google Maps, and that requires using API keys.
-- At the scale that OpenPATH is at, I don't think it would actually cost anything, but it's still annoying to deal with
-- Extra work that I'd like to avoid if possible
For surveys, I haven't really seen any options for ODK-compliant forms in the RN ecosystem.
So I don't think we can get away from Enketo, and I think this necessitates that at least part of the app will be rendered in a WebView anyway..
Not really sure where we go from here. Whatever we do, I think it's going to be a "lesser of two evils" kind of decision.
shankari
commented
1 year ago So for enketo, I think that a webview would be fine, since we always launch it as a full-screen view anyway. Either in a popup or in the ion-view directly.
Leaflet is trickier and I'd forgotten about it because I got so used to using it everywhere. I do think we want to avoid Google Maps because (i) we want to avoid changing our implementation and (ii) as an open source project, it is better to use open source components and not introduce any requirements for closed source APIs if possible.
Having said all that, I am a bit confused.
react-native-web-map is a thin native wrapper over react-google-maps.
react-leaflet that seems to be similar to react-google-maps in terms of a react wrapper around leaflet that doesn't use any iframes. It seems like we have two choices
react-leaflet in angular (using react2angular) directly to show the maps. This will work until we have removed all the angular stuff and need to switch to full react native (including Expo/React native plugins). And at that point, I guess we would use native maps (no API key needed) anyway?react-leaflet doesn't work directly for some reason, it seems like we can write a react native wrapper modeled on react-native-web-maps (which is not that complicated - 4 source files) to create react-native-web-leaflet. It looks like react-native-web-maps may be abandoned (see issue 71 in their repo), so we could contribute it to the community as well.But first, I want to see if react-leaflet can work in directly with react2angular as a stopgap until we switch to Expo and true react native.
shankari
commented
1 year ago correction: wrt
And at that point, I guess we would use native maps (no API key needed) anyway?
It's been a while since I've written a native UI (I think we moved to hybrid ~ 2015), and things have certainly changed since then. It looks like an API key is now required even just to show a google map in Android. This is not a requirement of the plugin but of the underlying google library - e.g. https://developers.google.com/maps/documentation/android-sdk/map#to_add_a_map
Ah but then we could just use https://github.com/pavel-corsaghin/react-native-leaflet which appears to have already handled creating a webview to display leaflet maps
I am not 100% sure if we can have multiple of those components on the screen at the same time (as we would need for the label screen); we would need to experiment with it to find out.
JGreenlee
commented
1 year ago Good news! As it turns out, React Native Web is more flexible than I thought - sorry for raising false alarm.
I didn't think that putting raw HTML inside React Native Web was going to work, but it does as long it's ultimately being rendered as HTML. This wouldn't work with native rendering directly, but we won't need to do that until later. As long as we're operating over Cordova, we can use HTML alongside (and within) React Native Web components. -- Of course, we should still put as much of it in React Native Web as we can, so that it's easier to go truly native down the road.
I got this working with Leaflet, so we can just work with that for now. The same should work for Enketo. When it comes time to switch to Expo, we will need to wrap these in WebViews, but at that point we won't have to worry about compatibility issues with Cordova and Phonegap.
JGreenlee
commented
1 year ago --For the record:--
why do we need to use iframes? why can't we render the leaflet map directly into the webview?
This is how react-native-web-webview works to support different platforms. On native rendering, it puts the content in an Android WebView or iOS WebView. With web rendering, it places the content inside an <iframe>.
Ideally this would just work, and we wouldn't have to do anything extra when we transition. But the <iframe>s didn't show up, so we won't use them for now.
I want to see if
react-leafletcan work in directly with react2angular as a stopgap
I am not sure if we can use 'pure' ReactJS libraries right now. It might work, but only temporarily. We should try to implement as much as possible with React Native Web libraries, that way it's guaranteed to work with Expo web later.
Options:
Ideally, we want to be as forward compatible as possible since web frameworks change often and we don't have a large dedicated team to work on forward migrations every two years.
One option that we are seriously considering is to convert all of the main UI components into standard web components. This will allow us to migrate at the top level - e.g. the glue framework - quickly and then slowly migrate the individual web components over time, one every month.
I will also note that one of the requests that comes up periodically is to support a web version (sort of like WhatsApp vs. WhatsApp Web) so that people can explore their trips on a bigger screen. This is very low priority right now but we can keep it in mind for later, especially if we want to support more complex visualizations of the collected data.
@TTAlex @paulvantran @asiripanich @larbigharib @ericafenyo in case you have thoughts. We can discuss further at the monthly meeting. We hope to make the switch over the summer.