Bpred does not install on Mac

[isomemo]

@arunge @jroachell15

Bpred does not install on Mac. See below!

R version 4.2.0 (2022-04-22) -- "Vigorous Calisthenics" Copyright (C) 2022 The R Foundation for Statistical Computing Platform: x86_64-apple-darwin17.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.

During startup - Warning messages: 1: Setting LC_CTYPE failed, using "C" 2: Setting LC_COLLATE failed, using "C" 3: Setting LC_TIME failed, using "C" 4: Setting LC_MESSAGES failed, using "C" 5: Setting LC_MONETARY failed, using "C" [R.app GUI 1.78 (8075) x86_64-apple-darwin17.0]

WARNING: You're using a non-UTF8 locale, therefore only ASCII characters will work. Please read R for Mac OS X FAQ (see Help) section 9 and adjust your system preferences accordingly. [Workspace restored from /Users/rosamariaarnizmateos/.RData] [History restored from /Users/rosamariaarnizmateos/.Rapp.history]

options(repos = c(getOption("repos"), PANDORA = "https://Pandora-IsoMemo.github.io/drat/")) install.packages("mpiBpred") --- Please select a CRAN mirror for use in this session --- Warning: unable to access index for repository https://mirror.dogado.de/cran/src/contrib: cannot open URL 'https://mirror.dogado.de/cran/src/contrib/PACKAGES' Warning: dependencies 'Rcpp', 'coda', 'dplyr', 'ggplot2', 'htmltools', 'jsonlite', 'magrittr', 'modules', 'rsync', 'shiny', 'shinyMatrix', 'shinyWidgets', 'shinycssloaders', 'shinyjs', 'shinythemes', 'xlsx', 'zip' are not available Warning: unable to access index for repository https://mirror.dogado.de/cran/bin/macosx/contrib/4.2: cannot open URL 'https://mirror.dogado.de/cran/bin/macosx/contrib/4.2/PACKAGES' Warning: unable to access index for repository https://Pandora-IsoMemo.github.io/drat/bin/macosx/contrib/4.2: cannot open URL 'https://Pandora-IsoMemo.github.io/drat/bin/macosx/contrib/4.2/PACKAGES' Package which is only available in source form, and may need compilation of C/C++/Fortran: 'mpiBpred' Do you want to attempt to install these from sources? (Yes/no/cancel) Yes installing the source package 'mpiBpred'

trying URL 'https://Pandora-IsoMemo.github.io/drat/src/contrib/mpiBpred_22.06.2.tar.gz' Content type 'application/gzip' length 13913047 bytes (13.3 MB)

downloaded 13.3 MB

During startup - Warning messages: 1: Setting LC_CTYPE failed, using "C" 2: Setting LC_TIME failed, using "C" 3: Setting LC_MESSAGES failed, using "C" 4: Setting LC_MONETARY failed, using "C" ERROR: dependencies 'Rcpp', 'coda', 'dplyr', 'ggplot2', 'htmltools', 'jsonlite', 'magrittr', 'modules', 'rsync', 'shiny', 'shinyMatrix', 'shinyWidgets', 'shinycssloaders', 'shinyjs', 'shinythemes', 'xlsx', 'zip' are not available for package 'mpiBpred'

• removing '/Library/Frameworks/R.framework/Versions/4.2/Resources/library/mpiBpred'

The downloaded source packages are in '/private/var/folders/t_/30fj3cg51xg5ypnn12m_209c0000gn/T/RtmpAL3S6H/downloaded_packages' Warning message: In install.packages("mpiBpred") : installation of package 'mpiBpred' had non-zero exit status

library(mpiBpred) Error in library(mpiBpred) : there is no package called 'mpiBpred' options(repos = c(getOption("repos"), PANDORA = "https://Pandora-IsoMemo.github.io/drat/"))

[jroachell15]

@isomemo

I am currently looking into this. BUT a very easy alternative to this installation problem is to let the user download Docker as we discussed before, all the system incompatibility goes away and the user just need to run 2 commands and everything works. Have you tried downloading Docker?

[isomemo]

@jroachell15 a quick message: no objections on using Docker provided that it works across platforms (Windows, Linux, and Mac). Please also send me a webpage and/or video link describing what Docker is as I am not familiar with it.

[jroachell15]

@isomemo Yes actually, because Docker is a simulated local machine, it has all the required packages installed and has a turnkey configuration set to optimize the usage of the Apps (if not, the user has to download all the separate missing packages by themselves, like we are doing now, which is stressful and time costly for the user). @wahani correctly me if I am wrong, but it's correct to say that Docker works across all the platform OS for the apps (windows, Mac, and linux) so then running the same line of code to start up the App is all we need to do?

The big difference is that, instead of using R to startup the app, we open up Docker to run a command that automatically configures the app for usage.

First download Docker

• windows: https://docs.docker.com/desktop/windows/install/
• Mac: https://docs.docker.com/desktop/mac/install/
• linux: https://docs.docker.com/desktop/linux/install/

see video instructions (https://www.youtube.com/watch?v=_9AWYlt86B8)

[isomemo]

@jroachell15

In principle, given that this is just an additional command line, I see no problem! Again, provided that this works for all platforms. However, in this case, we should adapt drat so that ALL apps use Docker moving forward (e.g. describing in detail how users using the different platforms install Docker. Again, this should be accessible to users not experienced in command line coding)

[jroachell15]

Okay, this will be an executive decision,? can I confirm with you that we will officially switch to the Docker version of Installation? plus if we use Docker, the user doesn't have to even mess with downloading R or Rstudio.

Benefits, no need to download any other software, just download Docker ONLY. These are the only commands you need to run:

• download and install app: docker pull ghcr.io/pandora-isomemo/bpred:main
• start app: docker run -p 3838:3838 ghcr.io/pandora-isomemo/bpred:main

then the user can use the app immediately.

@isomemo could you please try to download docker and I can walk you through this version of the installation process?

[isomemo]

@jian Yes, if:

1) this is done for all apps 2) works on all platforms 3) you provide detailed instructions for non-expert users to install and run apps (e.g., how do you call the command line in the different platforms? after the install, how do you run the app in future uses?)

[jroachell15]

How about this, let me know when you first test it out with after you download Docker https://github.com/Pandora-IsoMemo/drat/issues/17#issuecomment-1169950082 and run the codes from above as mentioned. Then we can make a more informed documentation change as necessary. @isomemo

[isomemo]

@jroachell15 I am not an expert on this and I would only be able to test it on my computer. I would need confirmation on the 3 points mentioned in my previous post.

[jroachell15]

@isomemo yep I can confirm on the 3 points mentioned. So now it's about testing it. So maybe also @CarloCocozza can help us test out this on his local machine?

[isomemo]

@jroachell15 point 2 is that it works on all platforms :-) so, if this is confirmed I will indeed do the install using Docker and ask others (including Carlo) to also do so.

[wahani]

Benefits, no need to download any other software, just download Docker ONLY. These are the only commands you need to run:

Now that I read it, I think we can even reduce this to one line:

docker run --pull always -p 3838:3838 ghcr.io/pandora-isomemo/bpred:main

and always start the latest version of the app. This approach works for all Shiny-Apps and this is (more or less) exactly the same way as they are run on https://isomemoapp.com/.

[isomemo]

@wahani that sounds good! But, this would have to be the same method for all apps (for consistency) and work for all platforms (Linux, Mac, Windows). Also, installation documentation would have to be updated (including explaining how to install Docker for those not familiar with command-line options and how to run an app after installation). If all this is ok, then please move ahead!

[wahani]

@isomemo I understand your concerns. Docker is a fairly complex project. It is also one of the most popular tools amongst developers and has very good support for all major platforms (mac, win, all major Linux distros). (Even) Microsoft has done a phenomenal job in integrating it into Windows. In fact, I believe we might just find a short YouTube tutorial, like the one @jroachell15 already posted, for all platforms.

this would have to be the same method for all apps (for consistency) and work for all platforms

Agreed, we need to confirm this, I am pretty sure it does.

Once you have tested this approach on your own machine, you can share your experience.

[jroachell15]

@Isomemo i just tested the docker method on my own Mac machine. It works perfectly. So now we know it works for Linux and Mac. Please tested it on your window machine. I've posted the links and video tutorials. Please use them and give us your feedback!

On Thu, Jun 30, 2022, 09:58 Sebastian Warnholz @.***> wrote:

@isomemo https://github.com/isomemo I understand your concerns. Docker is a fairly complex project. It is also one of the most popular tools amongst developers and has very good support for all major platforms (max, win, all major Linux distros). (Even) Microsoft has done a phenomenal job in integrating it into Windows. In fact, I believe we might just find a short YouTube tutorial, like the one @jroachell15 https://github.com/jroachell15 already posted, for all platforms.

this would have to be the same method for all apps (for consistency) and work for all platforms

Agreed, we need to confirm this, I am pretty sure it does.

Once you have tested this approach on your own machine, you can share your experience.

— Reply to this email directly, view it on GitHub https://github.com/Pandora-IsoMemo/drat/issues/17#issuecomment-1171253963, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARTW5QVP3NEVTBUUK2LUMOTVRWRWXANCNFSM5ZUMENLA . You are receiving this because you were mentioned.Message ID: @.***>

[isomemo]

@wahani @jroachell15 @Karamky

I tried this together with Karam from our IT (included in this message). We had the following issues:

1) first we could not install Docker on my computer. Notice, that this only works for Windows 10 and 11. I have Windows 10 but Karam noticed that something was missing. He can explain this in more detail but it is important to notice that we will certainly have many users with older Windows versions 2) Karam manage to install an older version of Docker but installing the apps did not work! Karam can comment more on this but it is possible that this is related with the fact that we are working in an enterprise environment and to something related to our anti-virus (Kaspersky)

@Karamky can you please comment on the above in more detail? any additional suggestions on how to pinpoint the problem or alternative solutions?

[isomemo]

@wahani @jroachell15 @Karamky

I tried Dropbox at using my Windows laptop and was informed of an issue with a non-trivial fix:

@Karamky: did you look into this further?

[isomemo]

@karamkky @wahani @jroachell15

I am sending this message again since I did not use Karam's most recent GitHub account!

I tried this together with Karam from our IT (included in this message). We had the following issues:

1. Initially, we could not install Docker on my office computer. Notice, that this only works for Windows 10 and 11. I have Windows 10 but Karam realized that something was missing. He can explain this in more detail but it is important that I mention that we will certainly have many users with older Windows versions

2. Karam manage to install an older version of Docker but installing the apps did not work! Karam can comment more on this but it is possible that this is related to the fact that we are working in an enterprise environment and to something related to our anti-virus (Kaspersky)

3. I tried installing Docker on a different computer with Windows installed and to this notice for a non-trivial fix:

@karamkky can you please comment on the above in more detail? any additional suggestions on how to pinpoint the problem or alternative solutions?

[jroachell15]

@isomemo Okay, I will remind @wahani to comment on this. Here is a video of how we use docker to run Bpred: https://drive.google.com/drive/folders/1d7vEg2KRkBMFUszADOpgpRjc-3oz_udA

[wahani]

@karamkky can you provide more information what the problems on Windows are and if you see a solution for them?

[karamkky]

@wahani

1. We have some machines that are still running Windows 10 LTSB which apparently Docker Desktop doesn't support anymore The OS build is 14393. This might be an issue if users are running Windows 10 1709 or older. The only way around it is to either update the system to a newer build or install an older version of Docker According to @isomemo some of the users might be running an outdated OS that might not support the latest docker desktop so it's something to keep in mind (In our situation most of our devices are running the Windows 10 LTSC version (build 1809) so it will be an issue down the road running the latest docker desktop).

2. After installing the docker community (2018 edition), whenever we tried to pull the container. the connection times out. I did disable the AV software but it didn't help. It could be our firewall blocking the connection (I've to check and run some more tests) I think a better and easier option down the road is to push it to Docker Hub and let the user pull it from there.

3. WSL2 doesn't have to be used you can uncheck the option from the Docker Desktop settings however, Hyper-V must be enabled) Windows subsystem for Linux is not enabled by default the user (or the IT department) has to enable it along with Hyper-V however, this might be an issue because we (and I think most enterprise and educational/research institutes) run Windows 10 LTSC. Ours is build 1809 which doesn't support WSL2, it only supports WSL1.

[wahani]

@karamkky thanks for the insights. That sounds discouraging. We are not up-to-date with developments in the Microsoft ecosystem, so I depend on your recommendations here.

Do I understand you correctly, that you would not recommend using Docker as a means to get these apps running on local machines? Or should we check if:

After installing the docker community (2018 edition), whenever we tried to pull the container. the connection times out. I did disable the AV software but it didn't help. It could be our firewall blocking the connection (I've to check and run some more tests) I think a better and easier option down the road is to push it to Docker Hub and let the user pull it from there.

pushing the images to Docker Hub is a solution? Or will you try to identify, if it is because of firewall settings? We use these repositories on the VMs and do not have any issues. These VMs are managed by MPI IT, so it is probably an issue on the local/personal machines. Can you pull images from Docker Hub? E.g.

docker pull inwt/r-base

[isomemo]

@wahani @karamkky

I am looking forward to Karam's feedback! But perhaps we could also arrange a meeting to discuss this?

In addition, I was suggested Electron as an alternative: https://www.electronjs.org/ https://www.youtube.com/watch?v=ARrbbviGvjc https://foretodata.com/how-to-make-a-standalone-desktop-application-with-shiny-and-electron-on-windows/

Are any of you familiar with this possibility?

[karamkky]

@wahani The problem is not exclusive to Microsoft Windows, users gonna have the same issue if they're still running macOS 10.14 or older. Generally installing Docker on Windows requires activating Hyper-V which by itself is sometimes troublesome especially if the machine is missing certain updates. I'm sorry I'm not trying to be discouraging here, I'm just giving you an insight. According to @isomemo the user base has a lot of users running different operating systems with different versions (some of them are outdated) and this fragmentation is going to be a nightmare, especially when you're relying on another software to run your software.

After installing the docker community (2018 edition), whenever we tried to pull the container. the connection times out. I did disable the AV software but it didn't help. It could be our firewall blocking the connection (I've to check and run some more tests) I think a better and easier option down the road is to push it to Docker Hub and let the user pull it from there.

Upon testing the issue only occurs on the docker community edition, our AV and Firewall are not blocking it. I tested multiple devices on multiple networks and got the same error, so it's probably an issue with the older versions of docker.

pushing the images to Docker Hub is a solution? Or will you try to identify, if it is because of firewall settings? We use these repositories on the VMs and do not have any issues. These VMs are managed by MPI IT, so it is probably an issue on the local/personal machines. Can you pull images from Docker Hub? E.g.

The reason why I suggested Docker Hub is that certain institutes or companies might block pull requests from an unknown source (on a firewall or/and AV level) which will leave the user unable to pull the image. It's just a safer option to make the image also available on Docker Hub.

In addition, I was suggested Electron as an alternative: https://www.electronjs.org/ https://www.youtube.com/watch?v=ARrbbviGvjc https://foretodata.com/how-to-make-a-standalone-desktop-application-with-shiny-and-electron-on-windows/

I actually wanted to suggest Electron when we were talking last week but I wasn't sure how far are you guys in terms of development and reliance on docker. One thing to keep in mind though is that Electron relies on NodeJS for the backend and it comes bundled with Chromium so the application size will be a bit big and it will be a bit of a memory hog. But it's a great framework. Another alternative you can check is Tauri, it uses Rust for the backend and Tao for the frontend so it's smaller and less resource-intensive. https://tauri.app/

[isomemo]

@karamkky no final decision has been made on using Docker and given its recent problems, it is necessary to explore alternatives. The important thing to remember is that we need a simple and fully self-contained installer (including R and dependencies) for our apps.

I have read more about this and found these pages and describe how to create a standalone app but only for Windows:

https://towardsdev.com/converting-a-shiny-app-into-a-standalone-desktop-app-for-windows-ca3656da8468 https://github.com/chasemc/electricShine

There is a stackoverlow discussion that mentions several possibilities (including others than Electron):

https://stackoverflow.com/questions/69515395/how-to-run-a-shiny-app-as-a-standalone-application

it would be great if you @wahani @karamkky could have a look at these so that we could figure out the best option.

[jroachell15]

We can't do anything about docker on Windows on our end. If users are unable to install it, we have to take a different approach. (and how sure are we that most of our users will have older versions of windows?) If we want to turn these Apps into desktop applications, that are portable across platforms, using phantom/Electron or any other approach into that direction, we have an open ended project with uncertain outcome and potentially several hundred thousand euros invest. we would rather suggest to provide more cloud resources, which has a determinable cost and time to implement. Sebastians is on vacation until Juli 28, then he will be back to comment on this issue. @karamkky @isomemo

[isomemo]

@jroachell15 Developing an installer is not the option that I was proposing but rather using a readily available tool. The options I saw for Electron only require a couple of lines of code to describe the R app and dependencies but only seem to be working currently for Windows. If you become aware of any other off-the-shelf option then let us know. From our side, I will keep consulting with our IT and others. If this does not work and no solution is found for Docker (given the major problems observed for this option this is actually an open ended project that we cannot pursue) then we return to the original specification of having the instructions to run apps locally without Docker. In the meantime we pause the work on the local installation as we explore alternatives. If we find these on our side then we can free up the programming time awarded to INWT for something else.

@wahani @karamkky did you have a look at the options that I sent you (and potentially others)? Could we pursue these?

[wahani]

I have reviewed the path through Electron (and comparable tools) and I see two options:

• There are great frameworks for writing desktop applications with JS. The problem, however, is well described by @karamkky: we use R as backend for the reason, that it has all tools for (Bayesian) statistics available, most other languages do not. The modelling part, as far as I understand the requirements, is what consumes too much hardware resources and is why we want to push all computations to the user. @jroachell15 describes the dangers of trying to implement this in a different language und I agree, that it is a daunting task with a high chance of failure. Basically we would need to rewrite all apps and a considerable part of R packages we rely on.
• Using a readily available tool would be great: @isomemo the blog post you are linking to, that describes how to bundle a Shiny app as Electron executable, does look very interesting. It is basically what Docker tries to do for us, just in a more ad-hoc way. The script and project behind it has not received any updates in the past 3 years, so we may have to do more research, why they stopped and if anybody continued that effort. It might be an option for some Windows versions, but I don't think we will cover all OSs with it. Still, might be worth a try.

If we go back and think about the reason, why users want to have these apps running locally, my understanding is, that the web service has been unreliable in the past, due to missing resources at times when several users access the service and share the hardware. Although off-topic, we may be much better off, if we just scale out the hardware. Technologies like Kubernetes, and other services come to mind here. At least from the development time, this would be the fastest and easiest way to move on.

In case we want to test the Electron app bundle, we still need to make sure that we can install these apps on all OSs, because Electron requires all R-Packages and system dependencies in a portable format. From my point of view this would be a first important step into that direction.

[isomemo]

@wahani @jroachell15 @arunge Thanks for the feedback! In my last meeting with Jian and Antonia (about a couple of weeks ago) I asked that INWT stopped all work on the installer. Our IT is looking into this and I will let you know if (or not) the work should be resumed on Docker. If we do not used the time for this task we will use it for something else.

[isomemo]

@wahani forgot to mention in previous post that it is still necessary to provide instructions for the regular R install. That is, instructions for the user to overcome typical installation problems. Please start with BMSC.

It will take our IT some time to look into the Electron/Proton options. In the meantime, I was also suggested Podman (https://podman.io/) as an alternative to Docker. It works on the same images but does not require root access. I do not want you to investigate this, just wondering if you may already know it and if you have any general opinion.

[wahani]

@isomemo I believe we will run into the same problems with podman as we had with docker. As per installation instructions, Windows computer need to install the Windows Subsystem for Linux (WSL2): https://podman.io/getting-started/installation. And I think this dependency was causing issues. If we can resolve that part, then I think both, docker and podman should work.

[isomemo]

@wahani could this dependency be somehow "attached" to Docker/Podman "installers"? Also, Docker seems to need root access but would this not be an issue in institutional settings (like my own at the MP)?

Found this (in case relevant): https://docs.docker.com/engine/security/rootless/

[wahani]

@isomemo for the installation of WSL2 you have to have root access. This would be something that needs support by the corporate IT. As far as I know, it is technically not possible to run/install WSL2 without root access to the Windows machine.

Also, Docker seems to need root access but would this not be an issue in institutional settings (like my own at the MP)?

Yes, docker requires root access. I believe this would be an advantage of podman.

[isomemo]

@wahani Ok! So, let's assume that we give instructions to install WSL2 (btw this is not necessary for Mac?) we would still need root access to run the Docker app OR this would only be required for the install but not afterwards?

[wahani]

btw this is not necessary for Mac?

WSL2 is Windows only. Mac OSs have already a Linux kernel running. However, see the system requirements in https://docs.docker.com/desktop/install/mac-install/, a recent release of MAC OS is required for Docker.

we would still need root access to run the Docker app OR this would only be required for the install but not afterwards?

This is a requirement for installation and for running the apps with docker. I would assume that podman might be a solution if that is not possible.

[isomemo]

@wahani is this now working using Docker in Mac? If so, we can close this issue.

[wahani]

@isomemo sorry, I forgot to hand over to @jan-abel-inwt.

Prerequisite is a working installation of docker. Then it is working on all OSs (Win, Mac, Linux).

@jan-abel-inwt can you check the install instructions for R for completeness? Then we can close this issue.

[jan-abel-inwt]

Under Ubunutu I can run the bred app in a container using docker perfectly. This should work for macOs too.

Also i updated the wiki with instruction on how to run the application in a docker container.