Config generation fails for document service, as schema url is missing

[danceb]

I am trying to implement the document service. While generating the configs with the config-generator, I am getting the error

ERROR: Schema URL for service 'document' may not be blank

Maybe this occurs, because the schema url of the document service is missing here: https://github.com/qwc-services/qwc-config-generator/blob/master/src/schema-versions.json

So I added the url there manually, but then I got this error

[2024-05-02 05:44:58.173872] INFO: <b>Generating 'document' service config</b>
[2024-05-02 05:44:58.174008] WARNING: Could not load JSON schema from /srv/qwc_service/schemas/qwc-document-service.json:
[Errno 2] No such file or directory: '/srv/qwc_service/schemas/qwc-document-service.json'
[2024-05-02 05:44:58.174031] INFO: Downloading JSON schema from https://github.com/qwc-services/qwc-document-service/raw/master/schemas/qwc-document-service.json
[2024-05-02 05:44:58.467688] ERROR: Validation error: 'resources' is a required property
[2024-05-02 05:44:58.467789] WARNING: Location: 
[2024-05-02 05:44:58.467938] WARNING: Value: {
  "$schema": "https://github.com/qwc-services/qwc-document-service/raw/master/schemas/qwc-document-service.json",
  "service": "document",
  "config": {
    "jasper_service_url": "..."
  }
}

May you have a look in the schema validation of the document service? Didn't get exactly, what is missing here. Thanks!

[danceb]

Uh, I get the second part, I have to provide the resources within the service config in the tenantConfig.json. I have tried this and still get the same error, but this should came from misconfiguration I have to figure out.

[manisandro]

I've added the schema in https://github.com/qwc-services/qwc-config-generator/commit/5843c7c04be4641dcd7cc1ca6dc1206ddf7acdbf

[danceb]

Within the running docker container of the generator there are already schema files for each service in /srv/qwc_service/schemas/. I have wondered where they come from? Because the URLs within schema-version.json are meant as fallback, if the schema isn't already there, right? So maybe you could provide the schema directly there aswell?

As I am getting still errors, I have tried to mount the schema file of the document service there within the docker-compose.yml. So now I have a file at /srv/qwc_service/schemas/qwc-document-service.json, but I am still getting this message:

[2024-05-02 20:28:05.458167] INFO: <b>Generating 'document' service config</b>
[2024-05-02 20:28:05.458287] WARNING: Could not load JSON schema from /srv/qwc_service/schemas/:
[Errno 21] Is a directory: '/srv/qwc_service/schemas/'

I don't know, why the file could not be found, but this should be gone, if you provide it directly in the image.

I am sorry, but maybe you have some hints for the other part, too. Does the generator checks already, if the document_templates are available via the jasper reporting service or does it just write the configuration to the config files?

I have added this part to the tenantConfig.json:

    {
      "name": "document",
      "config": {
        "jasper_service_url": "http://jasper-reporting-service/reports/",
        "jasper_timeout": 60
      },
      "resources": {
        "document_templates": [
          {
            "template": "test",
            "report_filename": "Test Report"
          }
        ]
      }
    }

I have tried this with "jasper_service_url": "http://localhost:8002/reports/" aswell. So the resources property is provided there. But I am still getting this error message:

[2024-05-02 19:29:01.748578] ERROR: Validation error: 'resources' is a required property
[2024-05-02 19:29:01.748678] WARNING: Location: 
[2024-05-02 19:29:01.748816] WARNING: Value: {
  "$schema": "https://github.com/qwc-services/qwc-document-service/raw/master/schemas/qwc-document-service.json",
  "service": "document",
  "config": {
    "jasper_service_url": "..."
  }
}
[2024-05-02 19:29:01.756591] ERROR: 'document' service config failed schema validation

[manisandro]

Within the running docker container of the generator there are already schema files for each service in /srv/qwc_service/schemas/. I have wondered where they come from? Because the URLs within schema-version.json are meant as fallback, if the schema isn't already there, right? So maybe you could provide the schema directly there aswell?

They are downloaded at image-build-time, see https://github.com/qwc-services/qwc-config-generator/blob/master/Dockerfile#L18

As I am getting still errors, I have tried to mount the schema file of the document service there within the docker-compose.yml. So now I have a file at /srv/qwc_service/schemas/qwc-document-service.json, but I am still getting this message:

[2024-05-02 20:28:05.458167] INFO: <b>Generating 'document' service config</b>
[2024-05-02 20:28:05.458287] WARNING: Could not load JSON schema from /srv/qwc_service/schemas/:
[Errno 21] Is a directory: '/srv/qwc_service/schemas/'

This looks looks like a bad configuration, in that the schema filename cannot be resolved from the schema URL (see https://github.com/qwc-services/qwc-config-generator/blob/master/src/config_generator/config_generator.py#L544)

I am sorry, but maybe you have some hints for the other part, too. Does the generator checks already, if the document_templates are available via the jasper reporting service or does it just write the configuration to the config files?

It just writes the configuration.

I have added this part to the tenantConfig.json:

    {
      "name": "document",
      "config": {
        "jasper_service_url": "http://jasper-reporting-service/reports/",
        "jasper_timeout": 60
      },
      "resources": {
        "document_templates": [
          {
            "template": "test",
            "report_filename": "Test Report"
          }
        ]
      }
    }

I have tried this with "jasper_service_url": "http://localhost:8002/reports/" aswell. So the resources property is provided there. But I am still getting this error message:

[2024-05-02 19:29:01.748578] ERROR: Validation error: 'resources' is a required property
[2024-05-02 19:29:01.748678] WARNING: Location: 
[2024-05-02 19:29:01.748816] WARNING: Value: {
  "$schema": "https://github.com/qwc-services/qwc-document-service/raw/master/schemas/qwc-document-service.json",
  "service": "document",
  "config": {
    "jasper_service_url": "..."
  }
}
[2024-05-02 19:29:01.756591] ERROR: 'document' service config failed schema validation

This looks like an incorrect input configuration is read, as it does not include "resources" even though the snippet you show above does include it?

[danceb]

Thanks for your reply! I will get through the several steps. May you update the docker image (https://hub.docker.com/r/sourcepole/qwc-config-generator) with the last commit (https://github.com/qwc-services/qwc-config-generator/commit/5843c7c04be4641dcd7cc1ca6dc1206ddf7acdbf)? Thanks!

[manisandro]

I've tagged v2024.05.06

[danceb]

This looks like an incorrect input configuration is read, as it does not include "resources" even though the snippet you show above does include it?

Yes, is included and I don't really understand, what is happening here ... The resources property is part of the configuration in the tenantConfig.json, as I used the example config from https://github.com/qwc-services/qwc-document-service/blob/master/README.md?plain=1#L27 as template.

But for some reason the schema validation does not found this property and so the generating of the document service config fails.

[danceb]

I gave it another try and looked into the config generation for different services. I came across this part: https://github.com/qwc-services/qwc-config-generator/blob/master/src/config_generator/config_generator.py#L294. Here services with resources are dealed different than "config-only services".

If the wording is correct, the document service IS a service with resources and should be handled the same way like for instance the search service (https://github.com/qwc-services/qwc-config-generator/blob/master/src/config_generator/search_service_config.py#L32)?

[danceb]

I came across this part: https://github.com/qwc-services/qwc-config-generator/blob/master/src/config_generator/config_generator.py#L294. Here services with resources are dealed different than "config-only services".

If the wording is correct, the document service IS a service with resources and should be handled the same way like for instance the search service

It is exactly as I thought here. As the document service has "resources" it should be treated as the other services with resources. With these changes the config generation for the document service works as expected. I added a PR to solve this.