In order to make a GUI, a redone API and greater distinction between the UI and y-cruncher is needed

[fffffgggg54]

A discussion on how to separate the GUI and calculation code used in y-cruncher, how to interface the two, and how the new y-cruncher will work.

[fffffgggg54]

I was thinking that both UI's would be included, but only one is launched. The UI would then start y-cruncher in "slave mode" as mentioned in #4 and #3, communicate with y-cruncher using some form of connection, and allow the user to do what they would like from there. @Mysticial how do you plan to do that? Would there be certain messages that would be sent and either data would be sent or received or some other method?

[Mysticial]

My plan right now is to use JSON over TCP. But I'm open to other ideas.

Plain-text JSON over TCP has more overhead than an optimized IPC+protobuf mechanism. But for this purpose, I'm thinking JSON is more straight-forward.

There will be two main components that will require communication:

• Interactions with the menus.
• Status updates during computations.

What I have in mind for how it will work is this. (and I'm open to other ideas)

Interactions with the Menus:

For the menus, y-cruncher is stateless. The GUI retains all state and user-input.

When the user opens up a new menu, the GUI sends a request to y-cruncher for that menu. y-cruncher will then respond with:

• The complete list of state variables as an object tree set to their default values.
• Metadata about each state variable and what they can be set to. (i.e. A list of constants/algorithms so that the GUI can render a drop-down menu for them.)
• Any annotations. (warnings, errors, etc...)

The GUI can then render this however it wants. To get an idea of what the state object tree for a particular menu looks like, the "Save to Configuration File" option will dump it out. I've added an example here.

When the user changes a variable. The GUI updates the state tree and sends to y-cruncher. y-cruncher will send back the same information above, but instead of the default state values, it sends back the same state tree with any adjustments made. The GUI can then re-render with this updated state tree.

Once the user is done selecting the settings and hits "start", the GUI sends the state tree to y-cruncher along with a command to run it. This starts the routine (computation, stress-test, benchmark, etc...).

When the user inputs an invalid parameter, y-cruncher will either respond to the GUI with an error message or will silently accept it and make any adjustments to make it valid as described above.

Status Updates During the Computation:

I haven't thought about this part yet.

Right now the console UI simply overwrites the same line over and over again. This happens every time the computation reaches a "status point". These prints are throttled to 1/second to avoid spamming the console.

In the future, this will change. There will be multiple lines each updating independently from each other. And instead of the computation thread "pushing out" updates (and getting throttled to 1/sec), there will be a main thread that periodically polls the computation threads for their status and prints it out.

I currently don't have a timeframe for when this change will happen. While it's conceptually trivial to make a GUI interact with either of these models, it is a moving target right now as far as implementation goes.

[fffffgggg54]

Is there anything else that can be done for the GUI?

[Mysticial]

I think the answer is no at this point. Even for the menus, there's more internal work that needs to be done. I'm coming to the realization that in addition to sending the parameters object model, we're also going to need to send a renderer object model.

The parameters object model will look like this. This is the important one since it is the core functionality of y-cruncher.

{
    "Action": "CustomCompute",
    "CustomCompute": {
        "Constant": {
            "Constant": "pi",
            "Algorithm": "chudnovsky"
        },
        "ComputeSize": {
            "DecimalDigits": 50000000,
            "EnableHexDigits": "true"
        },
        "Output": {
            "Path": "",
            "DigitsPerFile": 0
        },
        "Mode": "ram",
        "Parallelism": {
            "TaskDecomposition": 8,
            "Framework": "pushpool",
            "WorkerThreads": 0,
            "Randomization": "true"
        },
        "Allocator": {
            "Allocator": "mmap",
            "LargePages": "attempt",
            "LockedPages": "attempt"
        }
    }
}

The renderer object model will probably look something like this: (This is just an example I threw together in a few minutes. The final layout may be different.)

{
    "CustomCompute-Renderer": {
        "Constant": {
            "Constant": "Pi",
            "Algorithm": "Chudnovsky Formula"
        },
        "ComputeSize": {
            "DecimalDigits": "50,000,000",
            "HexadecimalDigits": "41,524,102"
        },
        "Output": {
            "Path": "Default Path",
            "Compress Output": "None - Write digits to a text file."
        },
        "Mode": "Ram Only",
        "Parallelism": {
            "Summary": "Push Pool  ->  8 / ?  (randomization on)",
            "TaskDecomposition": "8",
            "Framework": "Push Pool",
            "WorkerThreads": "Unlimited",
            "Randomization": "Enabled"
        },
        "Memory": {
            "Ram": "330 MiB",
            "Disk": "0 bytes",
            "Output": "87.3 MiB"
        },
        "Allocator": {
            "Allocator": "WinAPI VirtualAlloc()",
            "LargePages": "Attempt",
            "LockedPages": "Attempt"
        }
    }
}

Likewise, there will also need to be some sort of sub-options list to render any drop-down menus.

{
    "CustomCompute-SubOptions": {
        "Constant": [
            {"Tag": "sqrt", "DisplayName": "Sqrt(n)", ***TBA for the argument***},
            {"Tag": "goldenratio", "DisplayName": "Golden Ratio"},
            {"Tag": "e", "DisplayName": "e"},
            {"Tag": "pi", "DisplayName": "Pi"},
            ...
        ],
        "Parallelism": {
            "Framework": [
                {"Tag": "pushpool", "DisplayName": "Push Pool"},
                {"Tag": "cilk", "DisplayName": "Cilk Plus Work-Stealing"}
                ...
            ]
        }
        ...
    }
}

Right now, y-cruncher already has code to generate and parse the parameters object model in JSON exactly as the first example above shows. But it doesn't have anything for the renderers or the sub-options. So that's a to-do on y-cruncher's side. (in addition to all the networking stuff)

Depending on how much work the GUI wants to do, it doesn't have to obey everything in the renderers or sub-options models. So if someone wants to do a localized version, they won't be using the English strings that y-cruncher will return. Instead they'll need a custom tag->string mapping to render most of the GUI directly from the parameters object model. (i.e. the "gamma" tag maps to the string, "Euler-Mascheroni Constant")

I currently don't have a complete list of tags and their English strings since they're scattered all over the code. Some are also context-dependent. (such as the algorithm display name for Log(n) and the Euler-Mascheroni Constant) This will probably be one of the many things that'll need to be cleaned up.

[fffffgggg54]

Okay. Please post updates on the program. Maybe a library could be made and open-sourced. y-cruncher->json or other->proposed library->interface This may allow for easier and more open development for interfaces.

[Mysticial]

Will do.

Here's a rough to-do list for the internal code that will change/expand later as progress is made.

General:

• Implement a JSON parser and generator for the current object model library. (Done 12/26/2017)
• Implement a network server. (Done 3/3/2018)
• Implement the "slave mode". (Done 3/3/2018)
• Define and implement general communication protocols with the GUI. (Done 3/10/2018)

Menu Interactions:

• Define and implement an object model for the menu parameters. (Already existed since v0.7.3.)
• Define and implement an object model for rendering the menus. (~January 2018)
  • Done and implemented for Custom Compute, I/O Benchmark, and Stress Test.
  • But the contents are subject to change since there no consumers of the model yet.
• Define and implement an object model for listing menu options.
• Define a spec for all communication between GUI and y-cruncher with respect to the menus. (Done 3/10/2018)
• Implement generators for these object models for all the menus.
• Implement all the RPCs needed:
  • Custom Compute: Parameter Query (Done 3/3/2018)
  • Custom Compute: Run Computation (Done 3/10/2018)
  • Stress Tester: Parameter Query (Done 3/3/2018)
  • Stress Tester: Run Stress Test (Done 3/10/2018)

Status Printing:

• Define a spec for how status printing is supposed to work with a GUI. (Done 12/26/2017 - likely to change later)
• Refactor y-cruncher to place an abstraction layer between console printing and computational code. (Done 12/29/2017)
• Implement a new front-end that pushes status updates to the GUI. (Done 3/10/2018)

Post-Computation:

• Custom Compute: Send result/summary back to GUI. (Done 3/10/2018)
• TBD

Unexpected Loose-Ends that are not Trivial to Fix:

• Checkpoint Restart conflicts with Slave Mode.
• Clean shutdown when requested by GUI.
• Exception Handling: Passing fatal exceptions back to the GUI before terminating y-cruncher.
• Exception side-effects. (i.e. computation error will terminate the computation and cause y-cruncher to exit the "Computation" state and return to the "Idle" state . But the GUI won't know this.)

---

I'll be uploading the various specs above as they are completed or near completion. Since the spec for menu parameters has already existed for almost a year, that can go up now. I just need to translate it from source code to English.

[fffffgggg54]

Okay.

[Mysticial]

As an update. So far I've spent about 3 weekends working on this for real. And I think I already have enough to begin testing. I'll try to slip* it into y-cruncher v0.7.6 as an undocumented feature. But there's no ETA for that yet. So I might upload some one-off binaries instead.

It's still far from being production-ready as there's still a lot of bugs and missing features. But the core functionality is there for Custom Compute and the Stress Tester.

I'm very far behind on documenting all the protocols. This is somewhat intentional since they keep changing. But it's starting to stabilize a bit at this point.

*My worry at this point is the introduction of sockets into the application. The program already has issues with AV false positives. And I'm worried that linking in sockets will exacerbate the problem. Especially if Slave Mode remains hidden and undocumented for the next few releases while it's being worked on.