natcap / invest

InVEST®: models that map and value the goods and services from nature that sustain and fulfill human life.
Apache License 2.0
168 stars 68 forks source link

Workbench needs a more reliable way to be informed of the invest logfile location #900

Open davemfish opened 2 years ago

davemfish commented 2 years ago

A running invest process announces the location of the logfile in a message from utils.prepare_workspace.

The workbench stores the logfile path so that the log of a previously completed run can be displayed. Storing the invest workspace path is not sufficient because multiple runs can use the same workspace.

The workbench also uses the announcement from utils.prepare_workspace to know that the invest subprocess has actually started successfully.

This is all a bit fragile and relies on string matching. And right now there are rare cases where the Workbench's stdout listener receives more data than expected in one chunk, which breaks the string matching:

const stdOutCallback = async (data) => {
    if (!investStarted) {
        if (`${data}`.match('Writing log messages to')) {
           investStarted = true;
           runningJobs[jobID] = investRun.pid;
           const investLogfile = `${data}`.split(' ').pop().trim();
...

We can workaround that last problem of unreliable string matching by adding a little more structure to the message from invest, like brackets or something around the filepath: Writing log messages to [/path/to/my workspace/yeah] and other text here

But maybe there's a better to way to handle this communication in general.

davemfish commented 2 years ago

Option for communication from python process to javascript:

invest could stream to a file with a known name & location (like PID) known to both processes. And the file could include whatever data we want to pass.

davemfish commented 2 years ago

Removed the milestone because we have a working solution now. But could still improve on it in the future