how to break 1000 insts limit

[k88936]

i ve encountered a really large Project where one single function cost me 300+instruction usages.

Multiprocessing support - some kind of library or framework (probably) that could be used to provide high-level support for inter-processor communication using either memory cells/banks, or even unit flags.

will it be that answer ? to separate functions in different processors and find a framework to invoke that. i hope Compiler can detect it and solve it smarty and automatically.

[cardillan]

will it be that answer ?

I'm afraid not.

Mindustry Logic has very very limited support for communication between processors. They can only communicate through memory banks/unit flags, and only numeric values can the passed through these channels. On top of that, there are no synchronization mechanisms whatsoever. Automatically calling functions across processors using just these tools just might be possible, if you wanted to pass only numeric values along. I can imagine making such call automatically and waiting for an answer, but the inability to pass through strings, items or bound units would make the thing not very useful. (If this would be useful to you after all, let me know - it does sound like a fun feature to implement.)

We might also raise the idea of better IPC communication with Anuken, although I'm afraid that he intentionally keeps the processor abilities very limited and might find this feature too powerful.

What I was envisioning was providing tools to the programmer to split the program among several processors and for some very basic communication, such as generating/consuming events (still based solely on numbers, though). The use-case I imagined was, for example, scanning the entire map for resources or buildings. This task could be easily split among processors that would search different parts of the map in parallel. As another example, in one of my schematics I have a processor that measures power balance (which fluctuates quite wildly) and provides an average to another processor which then controls impact reactors using these data.

I'm currently concentrating on further optimizations of the generated code, so this idea is quite remote at the moment. For people that would want to manually split their programs among several processors, the Schematic builder could be already quite useful, and if someone was truly interested, I'd gladly add support for more languages through external compilers (in the command line version, provided the compiler has a command line interface as well).

[k88936]

im working on splitting code into different processors,and writing a framework https://github.com/k88936/Splitter ,But i did not find a class similar to function (maybe mindcode dosen`t work on that) i d like to get some guide from you to find a proper one or make one,,,,,,,,,and im wondering which is better? to genegrate schemacode source code or to directly genegrate .msch using your code inschemacode

[cardillan]

That sounds great! I'm looking forward to it.

Regarding the function class - what do you need it for? Do you want to manipulate Mindcode source code or compiled code? I might be able to point you to some specific class or method if I understand better what you intend to do (and if such class actually does exist here).

Regarding the final representation, it again depends on your design. If the splitter is meant to split Mindcode source, generating Schemacode would allow to embed the Mindcode source in it, which would seem preferable to me. If the splitter would work on compiled code, I wouldn't see a difference - it is already possible to compile Schemacode to .msch and back.

[k88936]

Regarding the function class - what do you need it for?

i need to know

1. what other function is called(just the name in the source code is enough )
2. basic info of the function :name ,num of args ,if it has pointer(of a shared memory) as args ,nums of instructions it will produce(no need precise)
3. of course some mechanic such as Global Variable, non-Number variable which can't be passed through memory can be discussed later

it could

1. independently be compiled to a String

Regarding the final representation

i ll work on compiled code because i intend to make this splitter work on other language too,and to build a frame ,i need to add some logic code directly(also may be compiled by mindcode :p )

[cardillan]

You might want to look at the OptimizationContext class. It contains these fields:

• program: list of logic instructions (may contain virtual instructions - these are resolved as one of the final steps in compilation)
• callGraph: instance keeping track of all functions and calls between functions.
• rootContext: the top AstContext node. AstContext nodes mirror the AST structure obtained from parser. Each instruction keeps a reference to an AstContext instance it belongs to. The therefore maps instructions to AST nodes they were created from. This could be used to locate the functions in compiled code and/or obtain the size of the function.

You'll probably want to inspect this structure after both compilation and optimization is finished. There are caveats with functions though:

• Functions can get inlined. Functions that are called just once are inlined by the compiler itself, other ones may be inlined by the optimizer. A single function can therefore appear in the source code multiple times.
• Functions can be completely removed by Unreachable Code Elimination. Such functions don't get removed from the CallGraph structure, however.
• At this moment, Mindcode variables aren't typed and type inference isn't performed (yet). Therefore, there's no information on types and values passed as function arguments.
• I hope to implement function pointers in the future. This will probably complicate matters substantially.

[k88936]

public CompilerOutput<String> compile(String sourceCode) {
        String instructions = "";

        try {
            long parseStart = System.nanoTime();
            final Seq program = parse(sourceCode);
            if (messages.stream().anyMatch(CompilerMessage::isError)) {
                return new CompilerOutput<>("", messages);
            }
            printParseTree(program);
            long parseTime = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - parseStart);

            long compileStart = System.nanoTime();
            DirectiveProcessor.processDirectives(program, profile, messages::add);
            instructionProcessor = InstructionProcessorFactory.getInstructionProcessor(messages::add, profile);
            GeneratorOutput generated = generateCode(program);
            long compileTime = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - compileStart);

            long optimizeStart = System.nanoTime();
            List<LogicInstruction> result;
            if (profile.optimizationsActive() && generated.instructions().size() > 1) {
                result = optimize(generated);
            } else {
                result = generated.instructions();
            }
            long optimizeTime = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - optimizeStart);

            if (profile.getFinalCodeOutput() != null) {
                debug("\nFinal code before resolving virtual instructions:\n");
                String output = switch (profile.getFinalCodeOutput()) {
                    case PLAIN      -> LogicInstructionPrinter.toString(instructionProcessor, result);
                    case FLAT_AST   -> LogicInstructionPrinter.toStringWithContextsShort(instructionProcessor, result);
                    case DEEP_AST   -> LogicInstructionPrinter.toStringWithContextsFull(instructionProcessor, result);
                    case SOURCE     -> LogicInstructionPrinter.toStringWithSourceCode(instructionProcessor, result, sourceCode);
                };
                debug(output);
            }

            info("Performance: parsed in %,d ms, compiled in %,d ms, optimized in %,d ms.".formatted(parseTime, compileTime, optimizeTime));

           /*
              I  plan to do split at this phase, and  remove LogicInstructionLabelResolver.resolve 
              because Splitter is based on LabeledJump
              and take over the rest work to generate .msch or Schematics 
              will that be OK?

           */
           // result = LogicInstructionLabelResolver.resolve(instructionProcessor, result); 

            instructions = LogicInstructionPrinter.toString(instructionProcessor, result);
        } catch (Exception e) {
            if (profile.isPrintStackTrace()) {
                e.printStackTrace();
            }
            switch (e) {
                case MindcodeException ex -> messages.add(MindcodeMessage.error("Error while compiling source code: " + e.getMessage()));
                case MindcodeInternalError ex -> messages.add(MindcodeMessage.error("Internal error: " + e.getMessage()));
                default -> messages.add(MindcodeMessage.error("Error while compiling source code."));
            }
        }

        return new CompilerOutput<>(instructions, messages);
    }

[k88936]

and shamefully i still can't understand mechnics of mindcode in detail,maybe i need your help to realize the function call,return ,recursion,memory model in Splitter. interface with docs on writing : https://github.com/k88936/Splitter/blob/master/src/main/java/org/kvto/Generator.java https://github.com/k88936/Splitter/blob/master/src/main/java/org/kvto/Module.java https://github.com/k88936/Splitter/blob/master/src/main/java/org/kvto/SplittableUnit.java

[cardillan]

Honestly, this part of the code needs redesign quite badly. No wonder you don't understand it well. I planned to improve it someday, but I'm quite busy right now. The top-level compilation mechanism should definitely be better structured and might support some kind of module or plugin mechanism to add or replace compiler functionality. For now, yes, adding the code at the place you indicated might work. We'll need some mechanism - such as a compiler directive and command-line switch - to activate the Splitter functionality.

I might be able to help somewhat if you described what you intend to do in greater detail. It looks like you're aiming for a deeper integration with Mindcode (nothing wrong with that), but you also want to keep your tool language independent, and I don't know how do you plan to design that.

[k88936]

I might be able to help somewhat if you described what you intend to do in greater detail.

https://github.com/k88936/Splitter/blob/master/README.md

Generates some logic code for efficient implementation of function calls and return mechanisms.

• Module.java

  Abstract class that needs to be implemented Represents the entire program and defines the framework (code other than functions) and parameters for main logic blocks and sub logic blocks. Also includes methods for parsing parameters. Please refer to the source code comments for more details.

• Segment.java

  Collection of functions stored in a single processor.

• Splittable.java

  Parent class of Splittable and SplittableGroup.

• SplittableGroup.java

  Abstracts recursive functions into one function for easier program analysis.

• SplittableUnit.java

  Abstract class that needs to be implemented Represents a function in a program. It needs to indicate whether it is a pure algorithm and define the method body generation.

• Splitter.java

  Main program.

Instructions

1. It is recommended to integrate Splitter into the compiler (to collect necessary information).you need to Implement abstract method in module and splittablUnit , generate splittableUnit for each function ,add all of them into module ,at last ,use splitter,get the final result.
2. Splitter is a framework, and to make it compatible with your compiler, you often need to implement how your compiler handles functions. For example, if you want to call a recursive function in shared memory, you will often need to pass the stack pointer in the implementation of the recursive function call in Module using the methods from Generator. When parsing parameters for subroutine in Module, remember to parse the stack pointer.

Represents the entire program and defines the framework (code other than functions) and parameters for main logic blocks and sub logic blocks. Also includes methods for parsing parameters. Please refer to the source code comments for more details.