Open dchwong opened 1 week ago
We can do this! Just for better understanding, do you plan migrating from one Spark version to another or from Spark to something else?
We need to migrate a bunch of Spark jobs from one data source to another. Does it sound like a big effort?
It sounds like a perfect use case for Hide! Currently, we're working on adding support for Java #147. As soon as it's ready we'll start working on Scala. Meanwhile, let us know if there's anything else we can do to help you succeed with your migration project.
@artmoskvin thanks! I can't wait to try it out. I'd like to use an ollama model since we need to run the model on a mac.
@dchwong nice! we're big fans of ollama here. have you already implemented the agent part or do you plan to use some library for that?
not yet, i want to start from few-shot prompt engineering
not yet, i want to start from few-shot prompt engineering
sounds reasonable. let us know if you need any help.
I stumbled upon a Scala project today and started looking at Metals, LSP for Scala. They have pretty good documentation for integrating new editors https://scalameta.org/metals/docs/integrations/new-editor
Based on a quick glimpse of the docs, to run Metals we'll need to:
Next, the project needs to be installed (or imported) using a build tool. For example, sbt uses Bloop, a build server for Scala, and requires two steps:
sbt-bloop
sbt bloopInstall
sbt build.connect
I assume these steps should be performed as part of the devcontainer lifecycle commands (docs https://containers.dev/implementors/json_reference/#lifecycle-scripts).
As any LSP, Metals have different configuration parameters. While we ignored this before, I think it's time to add proper support for LSP configuration by adding a corresponding field(s) in the CreateProjectRequest.
I'm gonna leave it here for now in case someone wants to pick it up and implement. Otherwise, me or @aleh-null will get back to it later.
I'd like to try using this coding agent to do some code migration for our Spark jobs.