Running swift-openapi-generator via CLI should not need a openapi-generator-config.yaml file

[jbehrens94]

Motivation

I tried to run the CLI tool via swift run only to find out I still need a openapi-generator-config.yaml file. I had not expected that, because what is the added value of passing it via the CLI if I need the config anyway? I would love to use the CLI without config file, so that I can generate the things I want/need in (for example) a Makefile.

Proposal

Separate the CLI from the openapi-generator-config.yaml file, or at least let it be another option that doesn't mutually include each other.

[czechboy0]

Hi @jbehrens94, you're right - it shouldn't require a config file.

Are you talking about the error Error: Must either provide a config file or specify --mode.?

You can provide --mode types --mode client, for example, if generating a client, and the CLI invocation will not require you to have a config file.

Does that fix your issue?

[jbehrens94]

Hi @czechboy0, thanks for your blazing fast response! Let me elaborate about the options I've tried:

• Using the swift run swift-openapi-generator command without a configuration file present on disk. swift run swift-openapi-generator generate openapi.yaml --mode types --mode client --output-directory ./. This does not work
• Adding the configuration yaml file on disk with generate client and types, filter tag basic. swift run swift-openapi-generator openapi.yaml generate openapi.yaml --mode types --mode client --output-directory ./ works, but outputs everything and doesn't filter.
• Using the swift run swift-openapi-generator filter command like described in the documentation doesn't seem to work, it's not available in swift run swift-openapi-generator -help command as a subcommand.

I'll explain my specific use case, maybe you've got an idea on what could work. I've got a huge OpenAPI yaml file, with about 12 tags. That would create huge PRs if I regenerate a part. Therefore, I would like to split up the generated code per tag. That way, I could even create a local package per tag and build more modular packages.

[czechboy0]

Thanks for the extra context. Okay, a few things here. (cc @simonjbeaumont as this is related to filtering)

Using the swift run swift-openapi-generator command without a configuration file present on disk. swift run swift-openapi-generator generate openapi.yaml --mode types --mode client --output-directory ./. This does not work

That definitely should work, I just tried it on the 1.0.0-alpha.1 and it worked for me. Which tag are you using?

Adding the configuration yaml file on disk with generate client and types, filter tag basic. swift run swift-openapi-generator openapi.yaml generate openapi.yaml --mode types --mode client --output-directory ./ works, but outputs everything and doesn't filter.

Here it seems you have the openapi.yaml argument before the generate command, that won't work.

Now, what you actually care about is this: filtering requires using the config file. Sorry, I didn't realize you wanted to use filtering before; we currently do not have CLI options for filters. Is that something you'd like to see? (If so, maybe we can repurpose this issue to track that feature request.)

Using the swift run swift-openapi-generator filter command like described in the documentation doesn't seem to work, it's not available in swift run swift-openapi-generator -help command as a subcommand.

I do see it there, maybe you're using an older version?

swift run swift-openapi-generator -help 
Building for debugging...
Build complete! (0.12s)
OVERVIEW: Generate Swift client and server code from an OpenAPI document

USAGE: swift-openapi-generator <subcommand>

OPTIONS:
  -h, --help              Show help information.

SUBCOMMANDS:
  filter                  Filter an OpenAPI document
  generate                Generate Swift files from an OpenAPI document

  See 'swift-openapi-generator help <subcommand>' for detailed help.

---

Having said all this, my read is that the thing you care about is doing filtering from the CLI, without a config file, is that accurate?

[jbehrens94]

I had not updated to 1.0.0-alpha.1, I just tried that.

Given this folder (/Users/jbehrens/Developer/<project>/API/Sources/API):

ls -la
total 3944
drwxr-xr-x  6 jbehrens  staff      192 29 nov 22:42 .
drwxr-xr-x  3 jbehrens  staff       96 28 nov 20:56 ..
-rw-r--r--@ 1 jbehrens  staff      288 29 nov 21:32 API.swift
-rw-r--r--@ 1 jbehrens  staff   258522 29 nov 22:00 Client.swift
-rw-r--r--@ 1 jbehrens  staff  1531323 29 nov 22:42 Types.swift
-rw-r--r--  1 jbehrens  staff   220642 28 nov 20:56 openapi.yaml

The output on 1.0.0-alpha.1 is this:

swift run swift-openapi-generator generate openapi.yaml --mode types --mode client --output-directory ./
warning: 'api': found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /Users/jbehrens/Developer/<project>/API/Sources/API/openapi.yaml
Building for debugging...
Build complete! (0.38s)
error: Issues with required files: No config file found in the target named 'API'. Add a file called 'openapi-generator-config.yaml' or 'openapi-generator-config.yml' to the target's source directory. See documentation for details..
error: build stopped due to build-tool plugin failures
warning: 'api': found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /Users/jbehrens/Developer/<project>/API/Sources/API/openapi.yaml
Building for debugging...
Build complete! (0.38s)
error: Issues with required files: No config file found in the target named 'API'. Add a file called 'openapi-generator-config.yaml' or 'openapi-generator-config.yml' to the target's source directory. See documentation for details..
error: build stopped due to build-tool plugin failures

I do indeed care about filtering from the CLI, so I can check in separate files or split them into several packages. That's the goal. :)

[jbehrens94]

Also, running swift run swift-openapi-generator --help results in:

swift run swift-openapi-generator --help
warning: 'api': found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /Users/jbehrens/Developer/<project>/API/Sources/API/openapi.yaml
Building for debugging...
Build complete! (0.38s)
error: Issues with required files: No config file found in the target named 'API'. Add a file called 'openapi-generator-config.yaml' or 'openapi-generator-config.yml' to the target's source directory. See documentation for details..
error: build stopped due to build-tool plugin failures
warning: 'api': found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /Users/jbehrens/Developer/<project>/API/Sources/API/openapi.yaml
Building for debugging...
Build complete! (0.39s)
error: Issues with required files: No config file found in the target named 'API'. Add a file called 'openapi-generator-config.yaml' or 'openapi-generator-config.yml' to the target's source directory. See documentation for details..
error: build stopped due to build-tool plugin failures

[czechboy0]

Oh this is interesting, I'm now unsure which adoption method you're trying to use:

Option 1: Build plugin generates code at build time in the build folder, generated files are not checked into git. You configure this in your Package.swift. Option 2: Manually invoking the CLI to generate files that are checked into git, no plugin mentioned in your Package.swift. Option 3: Command plugin, files are generated manually by you invoking the command, files are generated into the git repo, you need to depend on the generator in your Package.swift.

We generally recommend (1) unless you need to check in your generated files into git, in which case use (2) or (3).

Which are you trying to use here?

[jbehrens94]

Alright, I forgot to remove the plugin from Package.swift for option 2 and/or 3. One of method 2 or 3 would work for me, I'm not really hung on any of the two.

From what I understand from the CLI help, the filter command will output the correct YAML configuration for the filters. That's not what I am looking for, I want the filter to be applied on the output of the CLI tool. So, if I want only the operations etc. for tag basic, I don't want the other stuff there.

I'm not sure if I am explaining my needs well enough, if so, I can give another example.

[czechboy0]

I think I understand. The filter command is only used as a way to test out your filter part of the configuration file. Once you have the right config file, e.g.

generate:
  - types
  - client
filter:
  tags:
    - foo
    - bar

you would use the plugin or the CLI to generate the code. The generator will first filter down to only your requested content, and then generated the code for it. I think that's what you want?

Check out the docs on filtering here: https://swiftpackageindex.com/apple/swift-openapi-generator/0.3.5/documentation/swift-openapi-generator/configuring-the-generator#Document-filtering

[simonjbeaumont]

OK, there are two issues here, which I summarise as (details to follow):

1. Running swift run swift-openapi-generator from an adopter package isn't something we've expected (it is being interpreted as a plugin invocation).
2. Filtering requires a config file.

---

Details on running swift-openapi-generator "from the CLI".

Right, I think the core difference here is "running swift-openapi-generator from the CLI" can mean one of three things:

1. Running swift-openapi-generator, the binary, directly, from some precompiled version.
2. Running swift run swift-openapi-generator from a checkout of swift-openapi-generator.
3. Running swift package generate-code-from-openapi using the command plugin from your package.

(1) is equivalent to (2) and both of these should work just fine without a config file.

(3) is invoking the generator as a Swift package command plugin.

What you're doing is something I didn't think was even possible:

4. Running swift run swift-openapi-generator from your own package, that depends on swift-openapi-generator.

I just tried it and was surprised/confused as I thought that swift run would only allow you to run executable targets in your own package. However, I can confirm I tried it and swift run swift-openapi-generator does indeed try and run the generator and I can confirm that it has the limitations you see.

Specifically, if I take the IntegrationTest directory, which is a package that depends on swift-openapi-generator like an adopter package would... In this environment, I can run the following command with no problems:

❯ swift run swift-openapi-generator --help
...
Build complete! (1.65s)
OVERVIEW: Generate Swift client and server code from an OpenAPI document
...

Additionally I can use the generate subcommand:

❯ swift run swift-openapi-generator generate --mode types Sources/Types/openapi.yaml
Building for debugging...
Build complete! (0.12s)
Swift OpenAPI Generator is running with the following configuration:
- OpenAPI document path: /Users/Si/work/code/swift-openapi-workspace/packages/IntegrationTest/Sources/Types/openapi.yaml
- Configuration path: <none>
- Generator modes: types
- Feature flags: <none>
- Output file names: Types.swift
- Output directory: /Users/Si/work/code/swift-openapi-workspace/packages/IntegrationTest
- Diagnostics output path: <none - logs to stderr>
- Current directory: /Users/Si/work/code/swift-openapi-workspace/packages/IntegrationTest
- Plugin source: <none>
- Is dry run: false
- Additional imports: <none>
Writing data to file Types.swift...

This is a strange hybrid mode... From the logs, we can see that it has written Types.swift to the current working directory, but also made use of the config file that was next to the openapi.yaml.

However, I can confirm, that if we remove the config file thats sitting next to the openapi.yaml passed, I can reproduce the behaviour that it cannot be run without the config file...

❯ rm Sources/Types/openapi-generator-config.yaml

❯ swift run swift-openapi-generator generate --mode types Sources/Types/openapi.yaml
...
error: Issues with required files: No config file found in the target named 'Types'. Add a file called 'openapi-generator-config.yaml' or 'openapi-generator-config.yml' to the target's source directory. See documentation for details..
error: build stopped due to build-tool plugin failures

I think what's happening here is that the tool thinks it's being invoked as a plugin. When it's invoked as a plugin, it must have a config file.

I think the confusion here is that running swift run swift-openapi-generator form an package that depends on swift-openapi-generator (cf. swift-openapi-generator repo itself) results in SwiftPM executing this in a way that we (inside swift-openapi-generator) perceive to be a plugin invocation.

This is something we can investigate.

---

Running the swift-openapi-generator as a CLI (the way we support today)

You will need to run it in either way (1) or (2) above. Here's an example showing that it's possible to run the generator without a config file just fine with the generate subcommand. Note that it uses <none> config file and still writes the file.

❯ swift run --package-path /path/to/swift-openapi-generator-checkout swift-openapi-generator generate --mode types Sources/Types/openapi.yaml
...
Swift OpenAPI Generator is running with the following configuration:
- OpenAPI document path: /Users/Si/work/code/swift-openapi-workspace/packages/IntegrationTest/Sources/Types/openapi.yaml
- Configuration path: <none>
- Generator modes: types
- Feature flags: <none>
- Output file names: Types.swift
- Output directory: /Users/Si/work/code/swift-openapi-workspace/packages/IntegrationTest
- Diagnostics output path: <none - logs to stderr>
- Current directory: /Users/Si/work/code/swift-openapi-workspace/packages/IntegrationTest
- Plugin source: <none>
- Is dry run: false
- Additional imports: <none>
Writing data to file Types.swift...

Running swift-openapi-generator filter

You should be able to perform filtering in a similar manner. However, as you noted, this does require a config file to express the filters you want to apply.

❯ cat filter-config.yaml
generate: []
filter:
  schemas:
    - Greeting

❯ swift run --package-path /path/to/swift-openapi-generator-checkout swift-openapi-generator filter --config filter-config.yaml Sources/Types/openapi.yaml
Parsing document...
Parsing document complete! (0.00s)
Filtering document...
Filtering document complete! (0.00s)
openapi: 3.1.0
info:
  title: GreetingService
  version: 1.0.0
servers:
- url: https://example.com/api
  description: Example
components:
  schemas:
    Greeting:
      type: object
      properties:
        message:
          type: string
      required:
      - message

Does this help?

[simonjbeaumont]

Heh, in the time it took me to compose the above, I see there were more updates. 😅

As @czechboy0 said, you can use filter: in the config file when using generate subcommand or when using the plugin workflow. But you cannot provide filtering options on the command line when using the generate subcommand.

The filter subcommand is designed as OpenAPI document in, OpenAPI document out. For both debugging, and preprocessing.

[jbehrens94]

@czechboy0 @simonjbeaumont Thanks for your really elaborate explanations and the time and effort!

I think I'm understanding it now, from an adopter package I would generate the configuration file with the filter command and then use that to generate a filtered output. In that case I could probably script something. Create filter, write to disk, use in generate command, move files and rinse and repeat for all tags.

[czechboy0]

Yup that's one way. Another is to use the same OpenAPI document as a source of truth in the repo (even at the root), and only create symlinks in the Sources directory for each module that point to the original. Then, each module would have a different config file that filters a different tag.

That way, you only have a single OpenAPI document with everything, and the config is used automatically by the generator to only generate the bits you selected. No need to have N copies of the document in your repo, each filtered differently.

But if you prefer that, of course that works too.

[jbehrens94]

I tried the symlinking before, but that causes Xcode to completely freeze on my end.

[simonjbeaumont]

I tried the symlinking before, but that causes Xcode to completely freeze on my end.

That should not be the case. I'm curious about this, though; could you try cloning this repo and opening the IntegrationTest/ directory in Xcode (do not open the root package). It makes use of symlinks for theopenapi.yaml all over.

I am able to open it fine on the most recent Xcode (15.0.1).

[jbehrens94]

I don't know what it is with me, but in the past it didn't work and now it does. I used the symlinking approach to generate the GeneratedSources folder per package.

Like this:

for folder in *API; do cd "$folder" && swift package plugin generate-code-from-openapi --target "$folder" && cd ..; done

However, I seem to get errors because the generator plugin also generates the code under the hood of course. Am I using this as intended?

[czechboy0]

You should use either the command plugin (swift package plugin generate-code-from-openapi --target ... which emits the code into your git repo, and you check that in, in which case you only include the generator as a package dependency, but you do not add it into the plugins section for your target), or as the build plugin (recommended, you add the generator as a package dependency and add it to the plugins section for your target, this way the code is generated during the build and is not checked into your git repo).

Which one would you like to use?

[jbehrens94]

Sorry for my late response!

It was the first, but I didn't realise I could keep the dependency and remove the plugin, so I think I got there! Thank you both so much for your help, much appreciated.