garden-of-delete / airbyte-topiary

Airbyte deployment and configuration management tool
12 stars 4 forks source link

Destination cannot be modified #33

Open marsh-harv opened 2 years ago

marsh-harv commented 2 years ago

I have tried various different destinations to configure and every time I get the same error.

I want to set up a LocalJSON destination. My config is as below:

destinations:
  - name: demo-dest 
    destinationName: LocalJSON
    connectionConfiguration:
      destination_path: '/json_output'

error recieved:

API connection is healthy: True
Retrieving source and destination definitions from: http://localhost:8000/
Retrieving Airbyte configuration from: http://localhost:8000/
Applying changes to deployment: http://localhost:8000/
Error: unable to modify destination: LocalJSON
Response code: 400 Error: Unrecognized response code

Is there a catalog of config setups being worked on to help users or can you spot something I have missed?

garden-of-delete commented 2 years ago

@marsh-harv. Thanks for the issue. I'll take a look to see what's going on here.

In general I do need a lot more config.yml examples in the /examples directory that show many more sources and destination types than I have tested so far. Might be some easy contributions to make and/or potential bugs like this to find?

garden-of-delete commented 2 years ago

@marsh-harv Try Local JSON instead of LocalJSON.

This highlights that I should add the full list of sources and destinations from the Airbyte API to the documentation so the canonical name of every source and destination type is clear.

harveymarshall commented 2 years ago

hi, @garden-of-delete i am looking to add a folder for examples of config like a registry - do you have any idea how to determine the format for sources and dests that you have not yet tried. I am trying to update the big query dest to show config when you want to use the GCS loading method. Any help would be good while i try create a solid contribution

garden-of-delete commented 2 years ago

@devhmars I'm going to circle back and do a quality pass / update on the repo this weekend. I'll try to get a good answer to your question by the time I'm finished.