Extend the single file import to multiple files (DMN) on KIE Sandbox

[dmitry-buykin]

I'm using sandbox image to illustrate a specific file, by attaching my local folder to the sandbox container:

  kie_sandbox:
    container_name: kie_sandbox
    volumes:
      - ${PROJECT_HOME}:/var/www/html/samples

http://localhost:9090/#/import?url=http://localhost:9090/samples/dmns/primary.dmn

Unfortunately, I cannot import multiple files, which can be great for complex use cases. I don't want to use github to host these examples, because I need it for a local development environment.

Is it possible to extend /#/import endpoint to support upload of multiple pre-defined files (or all files in the directory URL).

[tiagobento]

That's indeed an interesting use case... Extending the import route to allow multiple files might be a little cumbersome, but I guess we can extend it to support .zip files? I'm not sure how we would be able to tell that a URL represents a directory, actually. Another possibility is to run a separate container containing a git server with repos that could be cloned...

Let me know what you think. In any case, I think having a mechanism for more complex samples (i.e. multi-file samples) is a must have.

[jomarko]

The Sandbox has this option to upload whole folder, highlighted with orange in the screenshot. I wonder if that do not solve your use case. Because even using import via url will not sync the file edited in Sandbox with the local volume.

[dmitry-buykin]

@tiagobento In my case I work with a local folder of dmn files for development - it's not a whole repository, but a sub-folder in the sandbox for multiple projects. As I could see in online-editor/importFromUrl, it's possible to retrieve data from github or another git repo link, but not from an arbitrary folder of the web server.

I've prepared script which is starting docker-compose with mapping project to

  kie_sandbox:
    container_name: kie_sandbox
    volumes:
      - ${PROJECT_HOME}:/var/www/html/samples

and then starting this project and opening link in the browser:

start_project() {
  local project_home=${1:-$(pwd)}
  local dmn=$2
  local kie_port
  kie_port=$(get_kie_sandbox_port "$ENV_PATH")
  if [[ -z $kie_port ]]; then
    echo "Error: Could not find KIE_SANDBOX_DISTRIBUTION__kieSandboxExposedPort in $ENV_PATH"
    exit 1
  fi
  # Start Docker Compose with the project home mapped to the container
  PROJECT_HOME="$project_home" docker-compose -f "$DOCKER_COMPOSE_PATH" --env-file "$ENV_PATH" up -d
  # Wait for the container to start
  wait_for_container kie_sandbox
  # Wait for the port to be exposed
  wait_for_port "$kie_port"
  # Generate the HTTP link
  local link="http://localhost:$kie_port/#/import?url=http://localhost:$kie_port/samples/$dmn"
  echo "Opening $link in the default browser..."
  sleep 2
  # Open the link in the default browser
  if command -v xdg-open > /dev/null; then
    xdg-open "$link"
  elif command -v open > /dev/null; then
    open "$link"
  else
    echo "Please open $link in your browser."
  fi
}

It is really useful when you work with many small projects, so you can switch between projects. My focus here is not on opening samples in browser, but rather how to optimize my daily work by improving environment for many small projects - with ability to switch between many small projects, usually 3-10 dmn files (up to 150 in extreme cases, but for it I would rather use VSCode extension).

So I'm asking about a practical tool for local development when switching between contexts. I don't want to use a dedicated github repo with proxy, because 1) I have multiple small projects in the single monorepo. 2) I need a tool for local debugging and testing - it's usually 1-2 weeks of development and debugging without publishing.

I think the main issue with Kie toolbox - it's not designed for TDD and code-first development, but rather for UI editing. For complex logic with many variations, I need to maintain a large set of test cases which I want to run regular, plus ability to quickly spin-up project, check how it works with some corner cases.

[dmitry-buykin]

@jomarko thank you, I'm aware about this option. I've requested another option to load the current dmn context to the kie-sandbox by using command-line script. I'm working with many small projects and need to switch between them, so it's much more convenient just to load the project using #/import url, like I've shared in the previous post. It is similar to using git or gh from command-line.

I'm using also some additional scripting to call extended-service to validate dmn model against predefined inputs, then compare with expected outputs, to use it with TDD. Also I have bunch of java code to integrate each dmn project with other services. So switching from code editor to Kogito UI would be much easier with such option I'm requesting.

Unfortunately Kogito was not designed with focus on many small projects to support easy integration and switching context between them, but for me it's a best way to use DMN in the real world use-cases.

[jomarko]

Thank you @dmitry-buykin for clarification of your use case. To me it sounds, until the feature you require is implemented, the VS Code extension is the most suitable for you. It will have soon also the validation mechanism available - https://github.com/apache/incubator-kie-tools/pull/2165

[dmitry-buykin]

Thanks @jomarko , what is ETA for this new feature in VSCode plugin? As I could see, it's not yet merged into main.

[jomarko]

Really can not say any specific ETA. Just can say these days the review comments are being addressed. So my guess is rather weeks than months, but it is only a guess.

[tiagobento]

@dmitry-buykin You're right that doing proper TDD on KIE Sandbox right now is not a great experience, as we lack support for .scesim files (unit tests for DMN models).

KIE Sandbox was also not designed to connect with your local file system, but rather to integrate well with Git repositories.

In fact, KIE Sandbox was created with the "small repo" approach in mind, although it would absolutely still work for a fairly large monorepo.

I'm really interested in your use-case @dmitry-buykin, as there might be more people going through what you are right now. If I may, would you mind answering these questions?

1. What is it that makes you prefer using KIE Sandbox rather than the VS Code Extension?
2. Why not import your monorepo directly on KIE Sandbox?
3. Would .scesim support on KIE Sandbox help you?
4. What is this tool you need for local debugging? What exactly are you debugging?
5. How do you run your application locally? mvn quarkus:dev? Or other?

Thanks a lot in advance!

[dmitry-buykin]

@dmitry-buykin You're right that doing proper TDD on KIE Sandbox right now is not a great experience, as we lack support for .scesim files (unit tests for DMN models).

KIE Sandbox was also not designed to connect with your local file system, but rather to integrate well with Git repositories.

In fact, KIE Sandbox was created with the "small repo" approach in mind, although it would absolutely still work for a fairly large monorepo.

I'm really interested in your use-case @dmitry-buykin, as there might be more people going through what you are right now. If I may, would you mind answering these questions?

Sure @tiagobento , here are our points.

1. What is it that makes you prefer using KIE Sandbox rather than the VS Code Extension?

   • Our entry-technical business users want to use Web UI to quickly adjust DRG/expression, and who will not use VS Code
   • Testing expressions is not possible in VS Code, so Web UI much easier to use for learning how DMN works (focus on these entry-level technical business users).

2. Why not import your monorepo directly on KIE Sandbox?

   • Some of projects contains 150 dmn files (one per country, total 25K rules), KIE Sandbox cannot render it. I found that we can use up to 7 DMN files loaded into KIE Sandbox
   • But majority of other projects are smaller - usually request/response converter, types and 1-2 DMNs with decisions.

3. Would .scesim support on KIE Sandbox help you?

   • Yes, for validation
   • No, for interactive debugging and development by entry-technical business users

4. What is this tool you need for local debugging? What exactly are you debugging?

   • Mostly routing logic between multiple DMN files - for standard cases like matching request to rule we generate DMN files, then need to check integration
   • Types validation, testing processing all elements in array then combining into output, some priority conditions
   • Exploring how to evaluate expressions like discounts/pricing, date-time conversions etc. (by entry-technical business users)

5. How do you run your application locally? mvn quarkus:dev? Or other?

   • For initial development using KIE Sandbox, it's nice to check steps interactively
   • For advanced-technical business users, mvn quarkus:dev, but it's just our team - too complex for other users
   • For more complex setup, merged with generated DMNs - deploying to our TEST environment, then manual testing

Thanks a lot in advance! You're welcome!

P.S. entry-technical business users - these are usually offshore teams (several hundreds users) who maintain business departments, for them using Web UI is much easier than setting up a local DEV environments with VS Code.