search

philipdelff / NMsim

Simulate Nonmem models from R
https://philipdelff.github.io/NMsim/
Other
5 stars 2 forks source link

could you provide an example of running nonmem from R (on windows)? #6

Closed nskoch closed 2 weeks ago

philipdelff commented 10 months ago

Unfortunately, Windows is still not supported. I assume you are running Nonmem on Windows and not only R (connecting to a linux/unix machine running Nonmem).

It is a small task implementing support for Windows - especially if using PsN to call Nonmem. However, I don't use windows myself, and I could use some help.

I hope someone who is familiar with running Nonmem in Windows can help implementing this. Essentially, we need a system() call that will call PsN's execute on a Nonmem control stream and then close (return to the R console). Anyone?

philipdelff commented 10 months ago

@nskoch Would you be able to test this from the git main branch? I think it should now work with PSN if you have that installed.

Install from github: library(remotes) install_github("philipdelff/NMsim")

You can use this script to test a basic simulation: https://github.com/philipdelff/NMsim/blob/main/inst/examples/R/testrun_NMsim.R

Currently, only the PSN execution method seems to work on Windows. This will allow you to do most with NMsim. If this works, next step will be to get the internal Nonmem execution method to work. That will unlock some more features to Windows users.

nskoch commented 10 months ago

Hi Philip,

Many thanks for your advice. Unfortunately I do not use PSN and not planned to do so. I may have found another way to run nonmem from R on Windows, but I need to check. I'll let you know if you're interested.

Best regards Nicolas

Professor of Medicine, MD PhD Addiction specialist Head of the Department of Clinical Pharmacology, Aix-Marseille University Head of the Poison Control Center Provence-Alpes-Côte d'Azur, Ste Marguerite Hospital Office: +33 49 106 5874 Mobile: +33 63 339 1765

Le mer. 18 oct. 2023 à 17:49, Philip Delff @.***> a écrit :

@nskoch https://github.com/nskoch Would you be able to test this from the git main branch? I think it should now work with PSN if you have that installed.

Install from github: library(remotes) install_github("philipdelff/NMsim")

You can use this script to test a basic simulation:

https://github.com/philipdelff/NMsim/blob/main/inst/examples/R/testrun_NMsim.R

Currently, only the PSN execution method seems to work on Windows. This will allow you to do most with NMsim. If this works, next step will be to get the internal Nonmem execution method to work. That will unlock some more features to Windows users.

— Reply to this email directly, view it on GitHub https://github.com/philipdelff/NMsim/issues/6#issuecomment-1768774009, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKBGW6APFUUVPL3O5LWWV7TX7727BAVCNFSM6AAAAAA57XZOMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRYG43TIMBQHE . You are receiving this because you were mentioned.Message ID: @.***>

philipdelff commented 10 months ago

Thank you Nicolas,

I would currently not recommend NMsim if the main purpose is running/calling Nonmem from R on a Windows system. NMsim does include the NMexec() function to call Nonmem but it's developed on linux and windows still needs work.

If you want to simulate Nonmem models from R, that is where NMsim becomes really helpful. Currently on Windows it has to be used in combination with PSN but that will hopefully soon be improved.

Best, Philip

philipdelff commented 1 month ago

Hi @nskoch ,

I think this works in recent NMsim versions, 0.1.1 and 0.1.2. You can try this to run Nonmem on a model:

library(NMdata)
library(NMsim)
## Path to Nonmem executable on your system. This is an example:
NMdataConf(path.nonmem ="C:/nm75g64/run/nmfe75.bat")

## run Nonmem on a control stream:
NMexec("path/to/myctl.mod",sge=FALSE)

If you try this, I would appreciate feedback.

By the way, simulation is supported in windows so you can use NMsim() to simulate with the model.

nskoch commented 1 month ago

Hi Philip

I tried the following:

NMdataConf(path.nonmem ="C:/nm75g64/run/nmfe75.bat") NMexec("C:/nm75g64/run/test/CONTROL6.ctl",sge=FALSE)

NMexec("C:/nm75g64/run/test/CONTROL6.ctl",sge=FALSE)Executing C:/nm75g64/run/test/CONTROL6.ctl Erreur in NMtransInp(data.input, file, translate = translate, recover.cols = recover.cols) : is.data.table(data) is not TRUE

Best regards Nicolas

Professor of Medicine, MD PhD Addiction specialist Head of the Department of Clinical Pharmacology, Aix-Marseille University Head of the Poison Control Center Provence-Alpes-Côte d'Azur, Ste Marguerite Hospital Office: +33 49 106 5874 Mobile: +33 63 339 1765

Le mer. 17 juil. 2024 à 20:17, Philip Delff @.***> a écrit :

Hi @nskoch https://github.com/nskoch ,

I think this works in recent NMsim versions, 0.1.1 and 0.1.2. You can try this to run Nonmem on a model:

library(NMdata) library(NMsim)

Path to Nonmem executable on your system. This is an example:

NMdataConf(path.nonmem ="C:/nm75g64/run/nmfe75.bat")

run Nonmem on a control stream:

NMexec("path/to/myctl.mod",sge=FALSE)

If you try this, I would appreciate feedback.

By the way, simulation is supported in windows so you can use NMsim() to simulate with the model.

— Reply to this email directly, view it on GitHub https://github.com/philipdelff/NMsim/issues/6#issuecomment-2233955633, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKBGW6FYLDUGXKQDUOV5UCDZM2YLJAVCNFSM6AAAAABLBEYNECVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZTHE2TKNRTGM . You are receiving this because you were mentioned.Message ID: @.***>

philipdelff commented 1 month ago

Thank you @nskoch !

That is not a very helpful error, but it looks like something we should be able to solve. It looks like there is no data available for some reason.

Before getting deeper into it, please make sure you have recent versions of NMdata and NMsim, so we are not looking into issues that have been solved already.

## NMdata should be 0.1.6
packageVersion("NMdata")
## NMsim should be 0.1.1 or 0.1.2
packageVersion("NMsim")

## If your package versions are older, you can upgrade this way (making sure we install from CRAN):
unloadNamespace("NMdata")
unloadNamespace("NMsim")
install.packages("NMdata",repos="https://cloud.r-project.org")
install.packages("NMsim",repos="https://cloud.r-project.org")

I don't have that example in my Nonmem installation, and I am unsure whether it is a complete example with data and correct paths. Can you try on one of your own models to see if you get the same issue? You can also try on a model provided with NMsim like:

NMexec(system.file("examples/nonmem/xgxr021.mod",package="NMsim"),sge=FALSE)

The drawback of using that example is that it will run it inside your NMsim installation folder. Not really a problem but not ideal either.

Best, Philip

nskoch commented 1 month ago

Hi Philip

I tried today after reloading the packages and renamed the dataset woth a csv extention. The result is below.

Execute is not recognized as an internal or external command. The command call c:/nm75g64/run/testNMsim_exec.bat failed with error code 1

NMexec("C:/nm75g64/run/test/CONTROL6.ctl",sge=FALSE)Executing C:/nm75g64/run/test/CONTROL6.ctl C:\Users\nicol\Dropbox\nm730\run\NSK_essai>cd "C:/nm75g64/run/test"

C:\nm75g64\run\test>"execute" -model_dir_name -nm_output=xml,ext,cov,cor,coi,phi,shk CONTROL6.ctl '"execute"' n'est pas reconnu en tant que commande interne ou externe, un programme ex�cutable ou un fichier de commandes.Message d'avis : Dans shell(shQuote(paste("call", path.script), type = "cmd")) : l'exécution de '"call C:/nm75g64/run/test/NMsim_exec.bat"' a échoué avec le code d'erreur 1

Many thanks for your work to launch nonmem from R.

Best regards Nicolas

Le jeu. 18 juil. 2024 à 12:27, Philip Delff @.***> a écrit :

Thank you @nskoch https://github.com/nskoch !

That is not a very helpful error, but it looks like something we should be able to solve. It looks like there is no data available for some reason.

Before getting deeper into it, please make sure you have recent versions of NMdata and NMsim, so we are not looking into issues that have been solved already.

NMdata should be 0.1.6

packageVersion("NMdata")

NMsim should be 0.1.1 or 0.1.2

packageVersion("NMsim")

If your package versions are older, you can upgrade this way (making sure we install from CRAN):

unloadNamespace("NMdata") unloadNamespace("NMsim") install.packages("NMdata",repos="https://cloud.r-project.org") install.packages("NMsim",repos="https://cloud.r-project.org")

I don't have that example in my Nonmem installation, and I am unsure whether it is a complete example with data and correct paths. Can you try on one of your own models to see if you get the same issue? You can also try on a model provided with NMsim like:

NMexec(system.file("examples/nonmem/xgxr021.mod",package="NMsim"),sge=FALSE)

The drawback of using that example is that it will run it inside your NMsim installation folder. Not really a problem but not ideal either.

Best, Philip

— Reply to this email directly, view it on GitHub https://github.com/philipdelff/NMsim/issues/6#issuecomment-2236164178, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKBGW6ENL6OKG77MBUJPKX3ZM6KB5AVCNFSM6AAAAABLBEYNECVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZWGE3DIMJXHA . You are receiving this because you were mentioned.Message ID: @.***>

philipdelff commented 1 month ago

Hi Simon @nskoch ,

NMsim is trying to run Nonmem through PSN's execute. It does so because it does not know where the Nonmem executable is located. Previously, you wrote this:

NMdataConf(path.nonmem ="C:/nm75g64/run/nmfe75.bat")

This is exactly how to tell NMsim where to find the Nonmem executable. Did you forget to execute that command this time? Note that NMdataConf() is a NMdata function so you need to load NMdata too.

nskoch commented 1 month ago

After loading the 2 packages, I execute the 2 following comands: NMdataConf(path.nonmem ="C:/nm75g64/run/nmfe75.bat") NMexec("C:/nm75g64/run/test/CONTROL6.ctl",sge=FALSE)

I have reinstalled the 2 packages to be sure of the version.

Would you like that I try something else?

Best regards Nicolas

philipdelff commented 1 month ago

Thank you for your patience Simon!

Yes, I think I know why it happens. Try this to explicitly have NMsim call nonmem directly instead of using PSN:

MdataConf(path.nonmem ="C:/nm75g64/run/nmfe75.bat")
NMexec("C:/nm75g64/run/test/CONTROL6.ctl",
method.execute="nmsim",sge=FALSE)

Best, Philip

nskoch commented 1 month ago

It WORKS.Nice jobI have a CONTROL6_dir0001 folder with files in it such as clt, coi, cor…Also the nonmem.exe and others

philipdelff commented 1 month ago

Execellent @nskoch ,

Let me recap. NMexec() does successfully execute Nonmem on a control stream on your Windows system. However, it was tricky to figure out how to make it do that in the absence of PSN. We saw that

Can you please make sure if you have NMsim >= 0.1.1 by running this?

packageVersion("NMsim")

It is worth noting that having NMsim use its internal execution method is preferable over having it use PSN. The extra flexibility of using the built-in method allows NMsim to perform additional types of simulations.

nskoch commented 1 month ago

Here it is

packageVersion("NMsim")[1] ‘0.1.2’

I had downloaded the last one.

Le lun. 22 juil. 2024 à 23:55, Philip Delff @.***> a écrit :

Execellent @nskoch https://github.com/nskoch ,

Let me recap. NMexec() does successfully execute Nonmem on a control stream on your Windows system. However, it was tricky to figure out how to make it do that in the absence of PSN. We saw that

-

NMsim needs the path to the Nonmem bat file. This can be provided as an argument to NMsim()/NMexec() using the path.nonmem argument or as you did using NMdata's configuration tool: NMdataConf(), again using the same path.nonmem argument. This is as expected because NMsim has no method to find this installation on its own.

On Windows, NMsim still relies on PSN even when path.nonmem is provided. You had to explicitly tell NMexec() to use the internal method by specifying method.execute="nmsim" In NMsim >= 0.1.1 this is not intended.

Can you please make sure if you have NMsim >= 0.1.1 by running this?

packageVersion("NMsim")

It is worth noting that having NMsim use its internal execution method is preferable over having it use PSN. The extra flexibility of using the built-in method allows NMsim to perform additional types of simulations.

— Reply to this email directly, view it on GitHub https://github.com/philipdelff/NMsim/issues/6#issuecomment-2243878782, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKBGW6AOIRMUNSSZZMC7JYTZNV5THAVCNFSM6AAAAABLBEYNECVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTHA3TQNZYGI . You are receiving this because you were mentioned.Message ID: @.***>

philipdelff commented 1 month ago

Thank you @nskoch

I consider this a bug. method.execute="nmsim" should be default when specifying path.nonmem.

It would also be useful to have a function available to the user to test what methods work on their system, given the installation paths they have provided. Something to work towards.

nskoch commented 2 weeks ago

Hi Philip,

I tried also with nmfe64 and I have the following error message:

NMdataConf(path.nonmem ="C:/nm74g64/run/nmfe74.bat")> NMexec("C:/nm74g64/run/test/CONTROL6.ctl",method.execute="nmsim",sge=FALSE)Executing C:/nm74g64/run/test/CONTROL6.ctlErreur dans NMtransInp(data.input, file, translate = translate, recover.cols = recover.cols) : is.data.table(data) n'est pas TRUE

However the following is always fine:

NMdataConf(path.nonmem ="C:/nm75g64/run/nmfe75.bat")> NMexec("C:/nm75g64/run/test/CONTROL6.ctl",method.execute="nmsim",sge=FALSE)Executing C:/nm75g64/run/test/CONTROL6.ctl C:\Users\nicol\Dropbox\nm730\run\NSK_essai>CD "C:/nm75g64/run/test/CONTROL6_dir0005"

C:\nm75g64\run\test\CONTROL6_dir0005>call "run_nonmem.bat"

C:\nm75g64\run\test\CONTROL6_dir0005>call C:/nm75g64/run/nmfe75.bat CONTROL6.ctl CONTROL6.lst Pathlist information for compiler and MPI systems are located in nmpathlist.txt

Starting NMTRAN

WARNINGS AND ERRORS (IF ANY) FOR PROBLEM 1

(WARNING 2) NM-TRAN INFERS THAT THE DATA ARE POPULATION.

(WARNING 43) THE $PK BLOCK REQUESTS "CALL ONCE PER INDIVIDUAL RECORD", BUT DATA ITEMS ARE USED IN THE $PK BLOCK. VALUES OF THESE DATA ITEMS SUBSEQUENT TO THOSE FROM THE FIRST EVENT RECORD WILL BE IGNORED. IF THIS IS NOT APPROPRIATE, THE CALL DATA ITEM CAN BE USED TO OBTAIN ADDITIONAL CALLS, OR $PK'S CALLING PROTOCOL SHOULD BE CHANGED. Recompiling certain components. starting wait ending wait Exiting lpreddo Compiling FSUBS Building NONMEM Executable Starting nonmem execution ... License Registered to: Medical School of Marseille Expiration Date: 14 FEB 2025 Current Date: 9 AUG 2024 Days until program expires : 190

First Order MONITORING OF SEARCH:

0ITERATION NO.: .../...

It seems that your function works fine with nmfe75 but not with nmfe74.

For information, I am looking for a way to send a list of models to be launched and I tried to use the repeat() function waiting for the end of the previous run before starting. I use the Sys.sleep() function.

Best regards Nicolas

Le mar. 23 juil. 2024 à 20:15, Philip Delff @.***> a écrit :

Thank you @nskoch https://github.com/nskoch

I consider this a bug. method.execute="nmsim" should be default when specifying path.nonmem.

It would also be useful to have a function available to the user to test what methods work on their system, given the installation paths they have provided. Something to work towards.

— Reply to this email directly, view it on GitHub https://github.com/philipdelff/NMsim/issues/6#issuecomment-2245936239, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKBGW6FU4Q6YZZRTFGEK7I3ZN2MTDAVCNFSM6AAAAABLBEYNECVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBVHEZTMMRTHE . You are receiving this because you were mentioned.Message ID: @.***>

philipdelff commented 2 weeks ago

Hi Nicolas,

It looks like you are missing to specify the path to the control stream you want to run. Like file.mod="path/to.mod".

Best, Philip

nskoch commented 2 weeks ago

Hi Philip,

In the line

NMexec("C:/nm74g64/run/test/CONTROL6.ctl",method.execute="nmsim",sge=FALSE)

CONTROL6.ctl is the control stream with its path.

Best regards Nicolas

philipdelff commented 2 weeks ago

My bad, I read that too quickly.

The two instances of NMexec() that you are showing are different in two ways. They use distinct Nonmem installations due to the different path.nonmem paths, and they run on distinct control streams due to the different file.mod values.

The error you are getting is not very intuitive. But it suggests that the input data used by C:/nm74g64/run/test/CONTROL6.ctl is not being read correctly. This happens before calling Nonmem, so it does not look to me like it has to do with differences between the Nonmem executables - it seems to be the two control streams and their input data sets.

Can you please check that the input data set listed in that control stream exists on your file system? You can try to read it using dat.inp <- NMdata::NMscanInput("C:/nm74g64/run/test/CONTROL6.ctl",applyFilters = FALSE) If the input data file exists, but NMscanInput() fails, it would be useful to see the error. In that case, I am also curious about your NMdata version. You can check that with packageVersion("NMdata").

Best, Philip

nskoch commented 2 weeks ago

Hi Philip,I tried NMscanInput() and had an error messageError in NMtransInp(data.input, file, translate = translate, recover.cols = recover.cols)  :is.data.table(data) is not true.The control stream and the THEOPP dataset are in the folder.My NMdata version is 0.1.6Sorry to bother you with this.Best regardsNicolasLe 9 août 2024 à 23:43, Philip Delff @.***> a écrit : My bad, I read that too quickly. The two instances of NMexec() that you are showing are different in two ways. They use distinct Nonmem installations due to the different path.nonmem paths, and they run on distinct control streams due to the different file.mod values. The error you are getting is not very intuitive. But it suggests that the input data used by C:/nm74g64/run/test/CONTROL6.ctl is not being read correctly. This happens before calling Nonmem, so it does not look to me like it has to do with differences between the Nonmem executables - it seems to be the two control streams and their input data sets. Can you please check that the input data set listed in that control stream exists on your file system? You can try to read it using dat.inp <- NMdata::NMscanInput("C:/nm74g64/run/test/CONTROL6.ctl",applyFilters = FALSE) If the input data file exists, but NMscanInput() fails, it would be useful to see the error. In that case, I am also curious about your NMdata version. You can check that with packageVersion("NMdata"). Best, Philip

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

philipdelff commented 2 weeks ago

Hi Nicolas,

Thank you for your patience and continued efforts to resolve this.

This seems to be an issue related to reading the input data file used by "C:/nm74g64/run/test/CONTROL6.ctl" Worth noting, the issue does not seem to affect reading the input data file from: "C:/nm75g64/run/test/CONTROL6.ctl"

I don't have these files in my Nonmem installations so I can't debug well without your help.

This is technically an NMdata issue but for practical purposes that doesn't matter because it prevents you from running NMexec() on the former of those two control streams. We see the error from "NMtransInp()" which is being used by NMscanInput() which NMexec() uses to copy the data set to a new location in which it will go on an run Nonmem. Two things have happened prior to where the error occurs, and the issue is likely in one of those steps. The steps are 1) Extracting the path to the input data set 2) Reading the input data set

I suspect step 1). I know the current CRAN NMdata version (0.1.6) does not support data files without file name extensions. So a path like my/datafile could be causing this while my/datafile.csv or whatever file name extension should work. That is one thing to check. For a more complete check of the first step, it would be informative for me to see: 1a) $INPUT and $DATA in the control stream. 1b) Results from NMdata::NMextractData("C:/nm74g64/run/test/CONTROL6.ctl")

For step 2, you can try to run NMreadCsv() on the path to the input data set. Just check that you get a reasonably looking data.frame. No need to check any details.

Best, Philip

nskoch commented 2 weeks ago

Hi Philip,In my control file, the dataset has no extension. It is the THEOPP dataset given with nonmem. If I save it as a csv file with an extension, it runs without error message.Best regardsNicolasLe 12 août 2024 à 14:09, Philip Delff @.***> a écrit : Hi Nicolas, Thank you for your patience and continued efforts to resolve this. This seems to be an issue related to reading the input data file used by "C:/nm74g64/run/test/CONTROL6.ctl" Worth noting, the issue does not seem to affect reading the input data file from: "C:/nm75g64/run/test/CONTROL6.ctl" I don't have these files in my Nonmem installations so I can't debug well without your help. This is technically an NMdata issue but for practical purposes that doesn't matter because it prevents you from running NMexec() on the former of those two control streams. We see the error from "NMtransInp()" which is being used by NMscanInput() which NMexec() uses to copy the data set to a new location in which it will go on an run Nonmem. Two things have happened prior to where the error occurs, and the issue is likely in one of those steps. The steps are

Extracting the path to the input data set Reading the input data set

I suspect step 1). I know the current CRAN NMdata version (0.1.6) does not support data files without file name extensions. So a path like my/datafile could be causing this while my/datafile.csv or whatever file name extension should work. That is one thing to check. For a more complete check of the first step, it would be informative for me to see: 1a) $INPUT and $DATA in the control stream. 1b) Results from NMdata::NMextractData("C:/nm74g64/run/test/CONTROL6.ctl") For step 2, you can try to run NMreadCsv() on the path to the input data set. Just check that you get a reasonably looking data.frame. No need to check any details. Best, Philip

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

philipdelff commented 2 weeks ago

OK, great. The next version of NMdata should resolve this. I am hoping to get NMdata 0.1.7 to CRAN within 2-3 weeks.

Best, Philip