Closed mnfienen closed 3 years ago
Here's the code for extracting time from run.sh (in NWT_SUBMIT/NWTOPT_FILES/objective.py, runModel()). timelim is what is used to timeout the modflow process. timelim needs to be in seconds given the function that is using it (subprocess.run). As you can see in the the try: except: the last line of run.sh is converted to seconds by multiplying by 60 and is set. It is then divided by 60 inline so that we can display the time out in minutes to the user (I've been assuming that setting timeout in minutes is much more convenient than setting in seconds). If the float() conversion fails, then there must be an errored input in the last line of run.sh, or it must be empty, resulting in timelim being set to none and the program informing the user of this. Let me know if you still think this is flawed, but after some tests of my own, it seems to be working (I put in a timeout of 12.5 minutes for a run and the upper bound of time elapsed for all the runs was 750 seconds, which checks out)
with open(os.path.join(cwd, 'run.sh')) as f:
for line in f:
if use_next:
run_command = line
use_next = False
elif line.startswith('# Run Command:'):
use_next = True
last_line = line
try:
timelim = float(last_line) * 60
print(f'[INFO] Timeout for model run is set to {timelim / 60} minutes')
except Exception as e:
timelim = None
use_timer = False
print('[INFO] No timeout set for model run')
print(f'[INFO] Using run command: {run_command.strip()}')
print(f'[INFO] Starting run out of {cwd}')
oh - that makes sense. Something else must be going on, then. Both @hwreeves-USGS and I were trying runs and the runs were writing a little output to the screen and then quitting. It seemed the amount of output they were writing was consistent with the timeout length if it was in seconds. So....maybe barking up the wrong tree :).
I tried a run with a much longer timeout and now no model output is being printed to the logs at all, but I'm not sure I gracefully killed the previous run.
hmmmm, do you have any of that output? are you running through condor? just a note, each machine should be printing out its output and all error messages it received while running, in that order. Also note, if a run is terminated by the timeout it should say "[WARNING] Time Limit reached, terminating run", and if it runs to completion it should say "[INFO] Successful termination of trial"
ha! actually, discovered that when I set the timeout in seconds, it was too big (saw that in the logfile). I just reset to something reasonable and off and running again
ahhh, makes sense. let me know if it works smoothly this time
hmmmm.......it's running now and screen output is like
Using NAME file: gv3.nam
Run start date and time (yyyy/mm/dd hh:mm:ss): 2021/09/21 20:28:14
Solving: Stress period: 1 Time step: 1 Groundwater-Flow Eqn.
[INFO] Successful termination of trial
this is a transient model, though, so maybe it's only working on the first stress period. Is there something special to inform the code that my model is transient?
Also just for reference: This is what the errors log looks like after a successful (timeout or no timeout) run:
INFO:hyperopt.mongoexp:PROTOCOL mongo
INFO:hyperopt.mongoexp:USERNAME None
INFO:hyperopt.mongoexp:HOSTNAME 130.11.177.99
INFO:hyperopt.mongoexp:PORT 27017
INFO:hyperopt.mongoexp:PATH /db/jobs
INFO:hyperopt.mongoexp:AUTH DB None
INFO:hyperopt.mongoexp:DB db
INFO:hyperopt.mongoexp:COLLECTION jobs
INFO:numexpr.utils:Note: NumExpr detected 20 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
INFO:hyperopt.mongoexp:job finished: 614956d15ce150f0e5244685
And this is what a successful run no timeout should look like in logs/outputs:
[INFO] Timeout for model run is set to 12.5 minutes
[INFO] Using run command: wine MODFLOW-NWT_64.exe tmr_pr_11_13_calib_PM_flooded_scen3.nam
[INFO] Starting run out of /var/lib/condor/execute/dir_4126325/NWT_SUBMIT/PROJECT_FILES
MODFLOW-NWT-SWR1
U.S. GEOLOGICAL SURVEY MODULAR FINITE-DIFFERENCE GROUNDWATER-FLOW MODEL
WITH NEWTON FORMULATION
Version 1.1.4 4/01/2018
BASED ON MODFLOW-2005 Version 1.12.0 02/03/2017
SWR1 Version 1.04.0 09/15/2016
Using NAME file: tmr_pr_11_13_calib_PM_flooded_scen3.nam
Run start date and time (yyyy/mm/dd hh:mm:ss): 2021/09/20 23:01:23
Solving: Stress period: 1 Time step: 1 Groundwater-Flow Eqn.
Run end date and time (yyyy/mm/dd hh:mm:ss): 2021/09/20 23:08:07
Elapsed run time: 6 Minutes, 43.742 Seconds
Normal termination of simulation
[INFO] Successful termination of trial
[MASS BALANCE]: 0.01
[SECONDS]: 403.742
[TOTAL ITERATIONS]: 40.0
shoot - I had an error mkdir: cannot create directory 'NWT_SUBMIT/PROJECT_FILES/nwts': File exists
I guess I need to fix that....
that's totally normal, i'm assuming there is just a nwts directory in your project_files already. either way its a nwtopt.sh error and shouldn't impact your run
nwtopt.sh is the script sent and ran by condor to start each worker
Sorry I just saw that you edited your previous comment, did the transient model issue work itself out or no?
No - transient models are still stopping after a single stress period but also don't seem to complete the sequence (see below)
I'm getting this for error file which is consistent with what you posted
INFO:hyperopt.mongoexp:USERNAME None
INFO:hyperopt.mongoexp:HOSTNAME 130.11.177.99
INFO:hyperopt.mongoexp:PORT 2742
INFO:hyperopt.mongoexp:PATH /db/jobs
INFO:hyperopt.mongoexp:AUTH DB None
INFO:hyperopt.mongoexp:DB db
INFO:hyperopt.mongoexp:COLLECTION jobs
INFO:numexpr.utils:Note: NumExpr detected 40 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
INFO:hyperopt.mongoexp:job finished: 614a889bae0fcc8c8df31d5c```
but the output file doesn't seem to be completing and is somehow terminating after one stress period.
var [INFO] Timeout for model run is set to 350.0 minutes [INFO] Using run command: ./mfnwt gv3.nam [INFO] Starting run out of /var/lib/condor/execute/dir_40943/NWT_SUBMIT/PROJECT_FILES
MODFLOW-NWT-SWR1
U.S. GEOLOGICAL SURVEY MODULAR FINITE-DIFFERENCE GROUNDWATER-FLOW MODEL
WITH NEWTON FORMULATION
Version 1.1.4 4/01/2018
BASED ON MODFLOW-2005 Version 1.12.0 02/03/2017
SWR1 Version 1.04.0 09/15/2016
Using NAME file: gv3.nam Run start date and time (yyyy/mm/dd hh:mm:ss): 2021/09/21 20:36:28
Solving: Stress period: 1 Time step: 1 Groundwater-Flow Eqn.
[INFO] Successful termination of trial
Hmmmm, if you run the pull_nwts.py script and pull the results, look in the nwt_performance.csv file and see if all the runs are erroring out, timing out, or if they are reporting back normal time elapsed and mbs
You can run pull_nwts concurrently with the normal process btw. Also if you want it to continuously pull, use argument —loop True
yeah - not looking good.....
NWT Number,Loss,Mass Balance,Seconds Elapased,# of Iterations,Min Loss
5,999999999999.0,999999,999999,-1,9999
7,999999999999.0,999999,999999,-1,9999
20,999999999999.0,999999,999999,-1,9999
21,999999999999.0,999999,999999,-1,9999
22,999999999999.0,999999,999999,-1,9999
23,999999999999.0,999999,999999,-1,9999
24,999999999999.0,999999,999999,-1,9999
25,999999999999.0,999999,999999,-1,9999
26,999999999999.0,999999,999999,-1,9999
I would say wait for other trials to come in. Pls note that the initial runs are all random
right on - thanks for looking into all this! Imma let it run to completion and check it out later
Of course! Let me know how it goes!
This does bring up an issue of run documentation, I'll add something to print when the run successfully completes but errors out due to bad solver settings
Make sure you use screen so the process doesn't terminate prematurely - even if it does you can resume by using the same key
To add some more information to what Mike sent last night - here is an outfile and errors file - MODFLOW seems to be stopping right after it starts, these runs should take a couple of hours. I asked for 300 trials and 300 cores - in 5 minutes or so 298 of the cores finished errors_1710_103.txt outfile_1710_103.txt .
I am seeing this near the end of the errors file:
Traceback (most recent call last):
File "/var/lib/condor/execute/dir_4162136/nwtenv/bin/hyperopt-mongo-worker", line 10, in
Ahhh, so the reserveTimeout()'s are the condor machine trying to reach the database and ask for a job, but they don't get any response. There a two likely scenarios: We had two jobs running at the same time, and combined they were using more than 375 workers, or the master script wasn't ran with "screen" and ended up closing after some time. The early error runs shouldn't be an issue as they are NWTOPT trying to grab some information from the search space
Were both of you using NWTOPT at the same time? What ip, port combo did you use for the database (ik you used 130.11.177.99:27020, but what about mike)? How many machines did both of you start up? And did you both use screen to run all of the main processes?
In my experimentation each database can have at most ~750 connections. Each worker takes up two connections, the nwtopt master script takes two, and pull nwts takes two, but disconnects and reconnects frequently
I used screen for both the mongodb and optimize_NWT commands (2 shells, each with screens).
I ran on port 27020 and asked for 300 nodes - which seemed to work OK looking at condor_q
HWR
Howard W. Reeves Acting Chief Science Officer, Groundwater and Watershed Modeling Team Lead USGS Upper Midwest Water Science Center 5840 Enterprise Drive Lansing, Michigan 48911 517-887-8914
From: Max Newcomer @.> Sent: Wednesday, September 22, 2021 8:58 AM To: maxwnewcomer/NWTOPT @.> Cc: Reeves, Howard W @.>; Mention @.> Subject: [EXTERNAL] Re: [maxwnewcomer/NWTOPT] timeout units (#15)
This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.
Were both of you using NWTOPT at the same time? What ip, port combo did you use for the database? How many machines did both of you start up? And did you both use screen to run all of the main processes?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmaxwnewcomer%2FNWTOPT%2Fissues%2F15%23issuecomment-924904707&data=04%7C01%7Chwreeves%40usgs.gov%7Cc933a51719ec4c4ad96708d97dc8a686%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679123031073545%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=9UKe9LVg%2BYeq3wTe30bpwIB2yShHjdQ3ZQmnzVk1gJE%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABMHEL5AM7S42KQYZHMYNPDUDHHFTANCNFSM5EQBBRDQ&data=04%7C01%7Chwreeves%40usgs.gov%7Cc933a51719ec4c4ad96708d97dc8a686%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679123031083503%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8or3kHR3tDAKvH7THdhCJfwooCEhP%2FpybXNOEV15qjM%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Chwreeves%40usgs.gov%7Cc933a51719ec4c4ad96708d97dc8a686%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679123031083503%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=KtisIpC7eJVpu1NwTkOH%2Be%2Fyu9%2FB09beMwKNsk6ApNw%3D&reserved=0 or Androidhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Chwreeves%40usgs.gov%7Cc933a51719ec4c4ad96708d97dc8a686%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679123031083503%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=bZIpphzI3Tvget0VUWm0n0bw1kqrFLy%2B9fpVFoecOjs%3D&reserved=0.
Also could've been that you set your trials too low and nwtopt actually finished running and failed to populate the database with more trials. Could you send your nwt_performance.csv?
Was mike running at the same time?
What happens if you try to start 50 workers with the same key and an extra 100 trials?
yes- HWR
Howard W. Reeves Acting Chief Science Officer, Groundwater and Watershed Modeling Team Lead USGS Upper Midwest Water Science Center 5840 Enterprise Drive Lansing, Michigan 48911 517-887-8914
From: Max Newcomer @.> Sent: Wednesday, September 22, 2021 9:02 AM To: maxwnewcomer/NWTOPT @.> Cc: Reeves, Howard W @.>; Mention @.> Subject: [EXTERNAL] Re: [maxwnewcomer/NWTOPT] timeout units (#15)
This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.
Was mike running at the same time?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmaxwnewcomer%2FNWTOPT%2Fissues%2F15%23issuecomment-924908403&data=04%7C01%7Chwreeves%40usgs.gov%7Ccd8b0daafb19411ca46308d97dc95a73%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679126025632789%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3vkc03wyhjqFn2ugQi0yfOd37RVLUfQoF%2Fc5JPAnHqY%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABMHEL3I3ZWSKG3SRRFBZPTUDHHXXANCNFSM5EQBBRDQ&data=04%7C01%7Chwreeves%40usgs.gov%7Ccd8b0daafb19411ca46308d97dc95a73%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679126025642745%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JFtQENV7lIyIzxw%2BTfuSptf8Xci2jswxeCuhDX8uiGA%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Chwreeves%40usgs.gov%7Ccd8b0daafb19411ca46308d97dc95a73%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679126025642745%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8ZoQID4vXeVeXL0Ri4dbthBL2S4h2h67j%2Bho3sQIh7A%3D&reserved=0 or Androidhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Chwreeves%40usgs.gov%7Ccd8b0daafb19411ca46308d97dc95a73%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679126025642745%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Bc7vSUtCpmTwJ6468GXU22Squ6gLKgJdIuxUObHKGVs%3D&reserved=0.
I haven't really tested concurrent users running multiple runs on the same machine, but likely there were too many overall connections between the both of you that clogged down the database. Unfortunately, when this happens, I have found that mongodb acts like Pythagoras's cup and purges all runs. Just to reiterate, reserveTimeouts() are all mongodb and nwtwork related issues.
I can find the nwt_performance.csv -- where would that be?
Howard W. Reeves Acting Chief Science Officer, Groundwater and Watershed Modeling Team Lead USGS Upper Midwest Water Science Center 5840 Enterprise Drive Lansing, Michigan 48911 517-887-8914
From: Max Newcomer @.> Sent: Wednesday, September 22, 2021 9:02 AM To: maxwnewcomer/NWTOPT @.> Cc: Reeves, Howard W @.>; Mention @.> Subject: [EXTERNAL] Re: [maxwnewcomer/NWTOPT] timeout units (#15)
This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.
Also could've been that you set your trials too low and nwtopt actually finished running and failed to populate the database with more trials. Could you send your nwt_performance.csv?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmaxwnewcomer%2FNWTOPT%2Fissues%2F15%23issuecomment-924908128&data=04%7C01%7Chwreeves%40usgs.gov%7C0909dfae1f47438d628508d97dc94b16%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679125781047266%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MYOQbFFSzILeCxEBE%2FhhzoFMHnplbuqIt3egfL5iJyI%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABMHEL7DRPSFO5ZIQR4ARTLUDHHWRANCNFSM5EQBBRDQ&data=04%7C01%7Chwreeves%40usgs.gov%7C0909dfae1f47438d628508d97dc94b16%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679125781057221%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1edwF04jm8lXWQ8X4GcqDTEVCR3C3rCGnS0xgNPEPOU%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Chwreeves%40usgs.gov%7C0909dfae1f47438d628508d97dc94b16%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679125781057221%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1SagqvtcbXW9LmFfL1LHVKejivQgnBTw11mI%2BMxVJ18%3D&reserved=0 or Androidhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Chwreeves%40usgs.gov%7C0909dfae1f47438d628508d97dc94b16%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679125781067179%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=T4z7Xty9eHACSc69WwLCgZzZN9k462E35Olip1lfoao%3D&reserved=0.
You should just run python pull_nwts.py --ip 130.11.177.99 --port 27020 --key your_key
(and if you were still running i would recommend --loop True)
Also, if @mnfienen could provide an update as to what his run ended up looking like, that would be great.
I personally really like running pull_nwts during runs because it really helps you see deep down what each machine is returning
is there a way to list valid keys?
Howard W. Reeves Acting Chief Science Officer, Groundwater and Watershed Modeling Team Lead USGS Upper Midwest Water Science Center 5840 Enterprise Drive Lansing, Michigan 48911 517-887-8914
From: Max Newcomer @.> Sent: Wednesday, September 22, 2021 9:12 AM To: maxwnewcomer/NWTOPT @.> Cc: Reeves, Howard W @.>; Mention @.> Subject: [EXTERNAL] Re: [maxwnewcomer/NWTOPT] timeout units (#15)
This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.
You should just run python pull_nwts.py --ip 130.11.177.99 --port 27020 --key your_key (and if you were still running i would recommend --loop True)
Also, if @mnfienenhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmnfienen&data=04%7C01%7Chwreeves%40usgs.gov%7C6740781c6ced4b8c1ee408d97dcaabf9%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679131711552601%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XBBJGQ7qB63JYXygZaTIbQF0z9Ic5bp4pX1TN5R6wUQ%3D&reserved=0 could provide an update as to what his run ended up looking like, that would be great.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmaxwnewcomer%2FNWTOPT%2Fissues%2F15%23issuecomment-924917158&data=04%7C01%7Chwreeves%40usgs.gov%7C6740781c6ced4b8c1ee408d97dcaabf9%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679131711552601%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=FaaGVaFHwVVaubvmZnwcHRRqzcWkS1s%2FzFm4bwnVHQ0%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABMHELZNE4SMKGADAWEP3VLUDHI35ANCNFSM5EQBBRDQ&data=04%7C01%7Chwreeves%40usgs.gov%7C6740781c6ced4b8c1ee408d97dcaabf9%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679131711562557%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zUwQ5zd7lPh0P0LtLpYQDbsuC2FoOdcQMAp%2BtRvUdvg%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Chwreeves%40usgs.gov%7C6740781c6ced4b8c1ee408d97dcaabf9%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679131711562557%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NqEn955i0lpnVoCqYcko3g4yvjPuaZEfV3gCgEv45Fo%3D&reserved=0 or Androidhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Chwreeves%40usgs.gov%7C6740781c6ced4b8c1ee408d97dcaabf9%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679131711562557%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=2mw%2F3VcdDBP3oBKKAAEZjJ7sTHxOtbHM9Ys6iK376LE%3D&reserved=0.
There currently isn't, but sometimes using the up arrow in the terminal will get you to the command you used to start up NWTOPT which should have the key. That is a good idea for a feature though, I'll add an issue to have that setup
in order to run pull_nwts - mongodb needs to be running?
Howard W. Reeves Acting Chief Science Officer, Groundwater and Watershed Modeling Team Lead USGS Upper Midwest Water Science Center 5840 Enterprise Drive Lansing, Michigan 48911 517-887-8914
From: Max Newcomer @.> Sent: Wednesday, September 22, 2021 9:20 AM To: maxwnewcomer/NWTOPT @.> Cc: Reeves, Howard W @.>; Mention @.> Subject: [EXTERNAL] Re: [maxwnewcomer/NWTOPT] timeout units (#15)
This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.
I personally really like running pull_nwts during runs because it really helps you see deep down what each machine is returning
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmaxwnewcomer%2FNWTOPT%2Fissues%2F15%23issuecomment-924924655&data=04%7C01%7Chwreeves%40usgs.gov%7Ce1335b2a6eaf4a098dec08d97dcbc382%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679136392508155%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rlGgcKUnmNDDcUadvKyU4Q5%2BdBd5XWDWruOa7Sn4V1s%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABMHEL6L7BXUVX3PMQQVYNDUDHJZLANCNFSM5EQBBRDQ&data=04%7C01%7Chwreeves%40usgs.gov%7Ce1335b2a6eaf4a098dec08d97dcbc382%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679136392518110%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zw%2F%2Bwr9x568OaMUcQ50mTLyA4djg9JpYPhVv8LPOFjA%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Chwreeves%40usgs.gov%7Ce1335b2a6eaf4a098dec08d97dcbc382%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679136392518110%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zv0Z%2BvEL5W26gsBoYkKau8DEQ2evFgCRzznROAxVFHM%3D&reserved=0 or Androidhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Chwreeves%40usgs.gov%7Ce1335b2a6eaf4a098dec08d97dcbc382%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679136392518110%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=2vpVhmbaL1BWkntqY%2BptHWcRh9ZDF0bLqQj%2BZ3gvrtA%3D&reserved=0.
Yep, because it's pulling directly from the database and needs to make a connection to request the information
here is the performance.csv file --- all 9999s
Howard W. Reeves Acting Chief Science Officer, Groundwater and Watershed Modeling Team Lead USGS Upper Midwest Water Science Center 5840 Enterprise Drive Lansing, Michigan 48911 517-887-8914
From: Max Newcomer @.> Sent: Wednesday, September 22, 2021 10:13 AM To: maxwnewcomer/NWTOPT @.> Cc: Reeves, Howard W @.>; Mention @.> Subject: [EXTERNAL] Re: [maxwnewcomer/NWTOPT] timeout units (#15)
This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.
Yep, because it's pulling directly from the database and needs to make a connection to request the information
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmaxwnewcomer%2FNWTOPT%2Fissues%2F15%23issuecomment-924972094&data=04%7C01%7Chwreeves%40usgs.gov%7C43b19b85eea649ac45ef08d97dd33623%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679168371783430%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=V4uDm%2FLrQLTmwYmqJMDHwe9CRNGXEjQU1fZ%2FhpoNz5k%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABMHEL7JWZG45BMSR2ABTKTUDHQA7ANCNFSM5EQBBRDQ&data=04%7C01%7Chwreeves%40usgs.gov%7C43b19b85eea649ac45ef08d97dd33623%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679168371793390%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RsnVblxZwOFYennrijlrM7Cz0vLJF5GoVy7r1xRjIas%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Chwreeves%40usgs.gov%7C43b19b85eea649ac45ef08d97dd33623%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679168371793390%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=qtLQlpAt%2Fng0l4WhnYlq4eUgmhC80HQamzd8%2Bo%2BGbcw%3D&reserved=0 or Androidhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Chwreeves%40usgs.gov%7C43b19b85eea649ac45ef08d97dd33623%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679168371793390%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=oQh67YszxFfrobN9YREsPnfkFbd6j%2FybLusTaByO5T0%3D&reserved=0.
How many trials?
Are there gaps in the NWTNUMs? like 2,3,5,8,12?
I submitted 300; but stopped the run after a little while when I saw that the list was all 999s
I could try something like 20 trails with 10 workers if that will help.
Howard W. Reeves Acting Chief Science Officer, Groundwater and Watershed Modeling Team Lead USGS Upper Midwest Water Science Center 5840 Enterprise Drive Lansing, Michigan 48911 517-887-8914
From: Max Newcomer @.> Sent: Wednesday, September 22, 2021 10:28 AM To: maxwnewcomer/NWTOPT @.> Cc: Reeves, Howard W @.>; Mention @.> Subject: [EXTERNAL] Re: [maxwnewcomer/NWTOPT] timeout units (#15)
This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.
How many trials?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmaxwnewcomer%2FNWTOPT%2Fissues%2F15%23issuecomment-924985735&data=04%7C01%7Chwreeves%40usgs.gov%7C1925e9021df04849962f08d97dd550d7%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679177412562748%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=IrkzVdCqJolhbZiBOHcN%2FDUCj%2FYOUvrAUd7l%2FuUv3eU%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABMHELYSW2ADNSMHDIHOQYLUDHRZXANCNFSM5EQBBRDQ&data=04%7C01%7Chwreeves%40usgs.gov%7C1925e9021df04849962f08d97dd550d7%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679177412562748%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PJaoY%2BavInwk7jkTA3R3hRYcQZB8Y0Pr7HP1Pd9JUaw%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Chwreeves%40usgs.gov%7C1925e9021df04849962f08d97dd550d7%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679177412562748%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=H2B6uNdGcJ9GUOkEouoS4bjN9S5Sp%2Flx%2BXfYIgUVXaM%3D&reserved=0 or Androidhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Chwreeves%40usgs.gov%7C1925e9021df04849962f08d97dd550d7%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679177412572703%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Nq2eyf0IzELfOyWXY%2B7y0PGPRLWIl%2FLl4wWVZPB%2BcEY%3D&reserved=0.
I don't see any gaps in the numbering of the nwt files
Howard W. Reeves Acting Chief Science Officer, Groundwater and Watershed Modeling Team Lead USGS Upper Midwest Water Science Center 5840 Enterprise Drive Lansing, Michigan 48911 517-887-8914
From: Max Newcomer @.> Sent: Wednesday, September 22, 2021 10:29 AM To: maxwnewcomer/NWTOPT @.> Cc: Reeves, Howard W @.>; Mention @.> Subject: [EXTERNAL] Re: [maxwnewcomer/NWTOPT] timeout units (#15)
This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.
Are there gaps in the NWTNUMs? like 2,3,5,8,12?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmaxwnewcomer%2FNWTOPT%2Fissues%2F15%23issuecomment-924986094&data=04%7C01%7Chwreeves%40usgs.gov%7C6104d1c8c13447e49ba008d97dd562e2%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679177715082721%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BoaFkgGSp%2F1SOSTjW2N%2BSoIY9Eb4INltuCL5Qrd1KjM%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABMHELZAYCTB3QLHK2Z4SLLUDHR3HANCNFSM5EQBBRDQ&data=04%7C01%7Chwreeves%40usgs.gov%7C6104d1c8c13447e49ba008d97dd562e2%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679177715092672%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=D9dX49gnm2eFXiloI9ewwnus0XevUvRRxlK11rKbg%2BM%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Chwreeves%40usgs.gov%7C6104d1c8c13447e49ba008d97dd562e2%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679177715092672%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UXzOtriMvnIKOXl95ioXrl5qQP3py%2Fv8x02%2FZzQHMX0%3D&reserved=0 or Androidhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Chwreeves%40usgs.gov%7C6104d1c8c13447e49ba008d97dd562e2%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679177715102627%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=QrhWbnmRnriew0UaPoX%2FwRmoydySBn34HI%2BYjtCGS2o%3D&reserved=0.
shoot, sorry for so many short emails.
One thing I forgot to mention is that Mike and I are running the linux compiled version of mfnwt --> not using wine and the windows version.
I wonder if that is why mfnwt is starting OK but then nwtopt says it is finished so quickly??
HWR
Howard W. Reeves Acting Chief Science Officer, Groundwater and Watershed Modeling Team Lead USGS Upper Midwest Water Science Center 5840 Enterprise Drive Lansing, Michigan 48911 517-887-8914
From: Reeves, Howard W @.> Sent: Wednesday, September 22, 2021 10:40 AM To: maxwnewcomer/NWTOPT @.>; maxwnewcomer/NWTOPT @.> Cc: Mention @.> Subject: Re: [EXTERNAL] Re: [maxwnewcomer/NWTOPT] timeout units (#15)
I don't see any gaps in the numbering of the nwt files
Howard W. Reeves Acting Chief Science Officer, Groundwater and Watershed Modeling Team Lead USGS Upper Midwest Water Science Center 5840 Enterprise Drive Lansing, Michigan 48911 517-887-8914
From: Max Newcomer @.> Sent: Wednesday, September 22, 2021 10:29 AM To: maxwnewcomer/NWTOPT @.> Cc: Reeves, Howard W @.>; Mention @.> Subject: [EXTERNAL] Re: [maxwnewcomer/NWTOPT] timeout units (#15)
This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.
Are there gaps in the NWTNUMs? like 2,3,5,8,12?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmaxwnewcomer%2FNWTOPT%2Fissues%2F15%23issuecomment-924986094&data=04%7C01%7Chwreeves%40usgs.gov%7C6104d1c8c13447e49ba008d97dd562e2%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679177715082721%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BoaFkgGSp%2F1SOSTjW2N%2BSoIY9Eb4INltuCL5Qrd1KjM%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABMHELZAYCTB3QLHK2Z4SLLUDHR3HANCNFSM5EQBBRDQ&data=04%7C01%7Chwreeves%40usgs.gov%7C6104d1c8c13447e49ba008d97dd562e2%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679177715092672%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=D9dX49gnm2eFXiloI9ewwnus0XevUvRRxlK11rKbg%2BM%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Chwreeves%40usgs.gov%7C6104d1c8c13447e49ba008d97dd562e2%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679177715092672%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UXzOtriMvnIKOXl95ioXrl5qQP3py%2Fv8x02%2FZzQHMX0%3D&reserved=0 or Androidhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Chwreeves%40usgs.gov%7C6104d1c8c13447e49ba008d97dd562e2%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679177715102627%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=QrhWbnmRnriew0UaPoX%2FwRmoydySBn34HI%2BYjtCGS2o%3D&reserved=0.
Would you mind pointing me to where your model is and I'll try running it on my own?
it's on M000
hwreeves/NWTOPT has the run.sh and nwtopt.sub; I put port = 27020 in there and used it in my run lines but I guess you can use your sub file and make the port match when you call mongodb and optimize_NWT
Project files in
hwreeves/NWTOPT/NWT_SUBMIT/PROJECT_FILES
The executable is 'mfnwt' and it is in the PROJECT_FILES directory.
Howard W. Reeves Acting Chief Science Officer, Groundwater and Watershed Modeling Team Lead USGS Upper Midwest Water Science Center 5840 Enterprise Drive Lansing, Michigan 48911 517-887-8914
From: Max Newcomer @.> Sent: Wednesday, September 22, 2021 2:16 PM To: maxwnewcomer/NWTOPT @.> Cc: Reeves, Howard W @.>; Mention @.> Subject: [EXTERNAL] Re: [maxwnewcomer/NWTOPT] timeout units (#15)
This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.
Would you mind pointing me to where your model is and I'll try running it on my own?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmaxwnewcomer%2FNWTOPT%2Fissues%2F15%23issuecomment-925169338&data=04%7C01%7Chwreeves%40usgs.gov%7C7452c06dcd0e43c6e60008d97df525f9%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679314133021717%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ivGNDkr7fzcegdi9Fu4Xdq1YV5%2F170meUgnvQc2UAK0%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABMHEL6LYTCQSNDXBYPOA3TUDIMPNANCNFSM5EQBBRDQ&data=04%7C01%7Chwreeves%40usgs.gov%7C7452c06dcd0e43c6e60008d97df525f9%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679314133021717%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Nrr1xTfOBg7hKbiijmlfgHXaeSEaXxBPEy%2BZr2dGnkU%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Chwreeves%40usgs.gov%7C7452c06dcd0e43c6e60008d97df525f9%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679314133031672%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=IIY3opfHqe3FL2kklKailCxFNSH%2BPHa9jObZSsGEh7k%3D&reserved=0 or Androidhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Chwreeves%40usgs.gov%7C7452c06dcd0e43c6e60008d97df525f9%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637679314133031672%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=75K3KzPV%2FFsEjuvkP3ssjDkMKgF7RCxf1zaBZNx3C28%3D&reserved=0.
Great thanks, I'll give it a try tonight. If you and @mnfienen could just make sure that you don't have your own mongodb process running by tonight that would be great
I'm going to close this issue since it seems that everything including the timeout is working correctly and that Howard's run seemed to be using a model that has some issues. Constant ReserveTimeout and SeverSelectionErrors are able to be fixed by changing database port. If anyone disagrees, I'll be happen to reopen it.
Looks like the example
run.sh
script says to enter timeout value in minutes but the code is converting to seconds.....just need to choose :)