keu
commented
2 years ago @marcosmarxm from the logs I can see that error is thrown in the Core, not in the connector's code. The spec looks correct to me. Maybe UI issue as well.
Closed marcosmarxm closed 1 year ago
keu
commented
2 years ago @marcosmarxm from the logs I can see that error is thrown in the Core, not in the connector's code. The spec looks correct to me. Maybe UI issue as well.
sherifnada
commented
2 years ago it does seem like a UI issue where the UI shold exclude an empty field from being sent to the backend
@jrhizor assigning to you for triage
jamakase
commented
2 years ago @sherifnada UI shouldn't exclude empty fields to be sent to the backend, should it? What if we entered some input that is a string but then cleared it - I believe we should send an empty string for such case. The same is here.
I believe It is a backend problem, where an empty string is assumed as a not correct value. If we start to exclude empty strings - it may break some other connectors.
I would even say that we probably should send empty values always instead of setting them to undefined, null for strings or totally omitting them - it will be easier to notice any errors when fields was somehow omitted
jamakase
commented
2 years ago Actually it also looks like there is some issue on backend here: I tried to call it with the same schema and it looks like it passes validation step as with end_date: "" I get check_connection failure with message: "Check Connection Failed!"
2021-11-30 10:01:19 INFO TemporalAttemptExecution(get):116 - Executing worker wrapper. Airbyte version: 0.32.8-alpha-cloud
jamakase
commented
2 years ago Implements better error catching in #8029.
sherifnada
commented
2 years ago @jrhizor @Jamakase should this be closed since #8029 has been merged?
jrhizor
commented
2 years ago @marcosmarxm can you verify that this is working as expected now?
marcosmarxm
commented
2 years ago @jrhizor I was able to update the value, but not to remove the field value.
jrhizor
commented
2 years ago @sherifnada is it safe to assume that we never have empty strings (or if there are then we can send a null)? This would make it much easier for the frontend.
I think it is true since it'd be marked as required. I don't think we should have a required field that permits empty strings.
sherifnada
commented
2 years ago @jrhizor in this case the field is optional right? so if it's an optional field, we can assume empty string is safely replaced with null. The same is probably true for a required field? i'm less certain on the 2nd one
alafanechere
commented
2 years ago This error also happens for the Google search console connector, which exposes an end-date parameter too.
davydov-d
commented
2 years ago From what I see this issue has almost nothing to do with the connector.
First, there is inconsistency in the payload sent to /api/v1/scheduler/sources/check_connection.
When omitting the end_date field in the UI, it is also missing in the payload:
When filling in the end_date param and then returning back to an empty string, the key end_date is present in the payload:
and this leads to an error:
Request to /api/v1/sources/check_connection_for_update is also sent with an empty string value for the end_date key:
This seems like an issue with editing the field in the UI.
Second, config validation made in java seems to be different from the one made in python, because python is ok with an empty string against {'format': 'date-time'} field.
I suggest sending null instead of empty string as @jrhizor and @sherifnada mentioned earlier would fix the problem.
The only thing I would change in the connector's specs is the validation pattern, which is obviously not the best choice.
davydov-d
commented
2 years ago Hi @marcosmarxm could you please have a look ta my comment above? I guess the fix should be done on the platform UI side, not the connector
misteryeo
commented
2 years ago Resurfacing this for the frontend team based on the latest investigation above @timroes
timroes
commented
2 years ago That's an interesting problem, and I am not entirely sure what's the desired fix should be here. Is an empty input box a null (or not present in the object) or an empty string? In general I'd suggest empty input fields should be empty strings (since otherwise you'd also not have any way of specifying empty string).
For the specific format: date-time I agree that if left empty we might want to rather treat it as null (and later on anyway have a proper datepicker for them). Since we're already working on improving the connector components (and datepicker would be part of this) and this issue is already open since quiet some time: @misteryeo what is your priority and urgency for this? Should we try to quick fix this still before the datepicker, or does this have another month to wait for a proper datepicker?
sherifnada
commented
2 years ago I'm in favor of treating empty optional string fields as null - i can't think of a case where a connector would specifically want empty string but break on null
ghost
commented
2 years ago Issue is still present for Facebook Marketing
sherifnada
commented
2 years ago @timroes @edmundito did the frontend team have a level of effort on this already? trying to do some guesstimates for Q4 prios
grubberr
commented
2 years ago @sherifnada @edgao
This is a real problem for a lot of connectors which has optional date parameter(s):
source-facebook-marketing, source-google-ads, source-mixpanel, etc
For example we are struggling on this oncall https://github.com/airbytehq/oncall/issues/503 which still does not work
My proposal:
if we have such non-required field
{
"start_date": {
"type": "string",
"format": "date-time"
}
}and customer entered empty string ""
UI has to send config.json to server WITHOUT key "start_date" at all
PSEUDO-CODE:
if config["start_date"] == "" and config["start_date"] is not required and config["start_date"] has format:
config.pop("start_date")@lshrinivas that should be rather low level of effort. The largest part of effort would be testing that this change doesn't break anything else.
jgrinaveckis
commented
1 year ago Any news in regards to this issue?
Current Behavior
After creating a Facebook Marketing source I'm not able to remove the
end_dateparams. If I try the UI return me to the source page, give me the impression it worked... but when I return to the connector setting page the end date param still there.Expected Behavior
Tell us what should happen.
Logs
Response from check_connection_for_update endpoint:
{ "message": "The provided configuration does not fulfill the specification. Errors: json schema validation failed when comparing the data to the json schema. \nErrors: $.end_date: does not match the regex pattern ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$, $.end_date: is an invalid date-time \nSchema: \n{\n \"type\" : \"object\",\n \"title\" : \"Source Facebook Marketing\",\n \"required\" : [ \"account_id\", \"access_token\", \"start_date\" ],\n \"properties\" : {\n \"end_date\" : {\n \"type\" : \"string\",\n \"title\" : \"End Date\",\n \"format\" : \"date-time\",\n \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",\n \"examples\" : [ \"2017-01-26T00:00:00Z\" ],\n \"description\" : \"The date until which you'd like to replicate data for AdCreatives and AdInsights APIs, in the format YYYY-MM-DDT00:00:00Z. All data generated between start_date and this date will be replicated. Not setting this option will result in always syncing the latest data.\"\n },\n \"account_id\" : {\n \"type\" : \"string\",\n \"title\" : \"Account Id\",\n \"description\" : \"The Facebook Ad account ID to use when pulling data from the Facebook Marketing API.\"\n },\n \"start_date\" : {\n \"type\" : \"string\",\n \"title\" : \"Start Date\",\n \"format\" : \"date-time\",\n \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",\n \"examples\" : [ \"2017-01-25T00:00:00Z\" ],\n \"description\" : \"The date from which you'd like to replicate data for AdCreatives and AdInsights APIs, in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated.\"\n },\n \"access_token\" : {\n \"type\" : \"string\",\n \"title\" : \"Access Token\",\n \"description\" : \"The value of the access token generated. See the docs for more information\",\n \"airbyte_secret\" : true\n },\n \"custom_insights\" : {\n \"type\" : \"array\",\n \"items\" : {\n \"type\" : \"object\",\n \"title\" : \"InsightConfig\",\n \"required\" : [ \"name\" ],\n \"properties\" : {\n \"name\" : {\n \"type\" : \"string\",\n \"title\" : \"Name\",\n \"description\" : \"The name value of insight\"\n },\n \"fields\" : {\n \"type\" : \"array\",\n \"items\" : {\n \"type\" : \"string\"\n },\n \"title\" : \"Fields\",\n \"default\" : [ ],\n \"description\" : \"A list of chosen fields for fields parameter\"\n },\n \"breakdowns\" : {\n \"type\" : \"array\",\n \"items\" : {\n \"type\" : \"string\"\n },\n \"title\" : \"Breakdowns\",\n \"default\" : [ ],\n \"description\" : \"A list of chosen breakdowns for breakdowns\"\n },\n \"action_breakdowns\" : {\n \"type\" : \"array\",\n \"items\" : {\n \"type\" : \"string\"\n },\n \"title\" : \"Action Breakdowns\",\n \"default\" : [ ],\n \"description\" : \"A list of chosen action_breakdowns for action_breakdowns\"\n }\n }\n },\n \"title\" : \"Custom Insights\",\n \"description\" : \"A list wich contains insights entries, each entry must have a name and can contains fields, breakdowns or action_breakdowns)\"\n },\n \"include_deleted\" : {\n \"type\" : \"boolean\",\n \"title\" : \"Include Deleted\",\n \"default\" : false,\n \"description\" : \"Include data from deleted campaigns, ads, and adsets.\"\n },\n \"insights_days_per_job\" : {\n \"type\" : \"integer\",\n \"title\" : \"Insights Days Per Job\",\n \"default\" : 7,\n \"maximum\" : 30,\n \"minimum\" : 1,\n \"description\" : \"Number of days to sync in one job. The more data you have - the smaller you want this parameter to be.\"\n },\n \"insights_lookback_window\" : {\n \"type\" : \"integer\",\n \"title\" : \"Insights Lookback Window\",\n \"default\" : 28,\n \"maximum\" : 28,\n \"minimum\" : 0,\n \"description\" : \"The attribution window for the actions\"\n }\n },\n \"definitions\" : {\n \"InsightConfig\" : {\n \"type\" : \"object\",\n \"title\" : \"InsightConfig\",\n \"required\" : [ \"name\" ],\n \"properties\" : {\n \"name\" : {\n \"type\" : \"string\",\n \"title\" : \"Name\",\n \"description\" : \"The name value of insight\"\n },\n \"fields\" : {\n \"type\" : \"array\",\n \"items\" : {\n \"type\" : \"string\"\n },\n \"title\" : \"Fields\",\n \"default\" : [ ],\n \"description\" : \"A list of chosen fields for fields parameter\"\n },\n \"breakdowns\" : {\n \"type\" : \"array\",\n \"items\" : {\n \"type\" : \"string\"\n },\n \"title\" : \"Breakdowns\",\n \"default\" : [ ],\n \"description\" : \"A list of chosen breakdowns for breakdowns\"\n },\n \"action_breakdowns\" : {\n \"type\" : \"array\",\n \"items\" : {\n \"type\" : \"string\"\n },\n \"title\" : \"Action Breakdowns\",\n \"default\" : [ ],\n \"description\" : \"A list of chosen action_breakdowns for action_breakdowns\"\n }\n }\n }\n }\n}", "exceptionClassName": "io.airbyte.server.errors.BadObjectSchemaKnownException", "exceptionStack": [ "io.airbyte.server.errors.BadObjectSchemaKnownException: The provided configuration does not fulfill the specification. Errors: json schema validation failed when comparing the data to the json schema. ", "Errors: $.end_date: does not match the regex pattern ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$, $.end_date: is an invalid date-time ", "Schema: ", "{", " \"type\" : \"object\",", " \"title\" : \"Source Facebook Marketing\",", " \"required\" : [ \"account_id\", \"access_token\", \"start_date\" ],", " \"properties\" : {", " \"end_date\" : {", " \"type\" : \"string\",", " \"title\" : \"End Date\",", " \"format\" : \"date-time\",", " \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",", " \"examples\" : [ \"2017-01-26T00:00:00Z\" ],", " \"description\" : \"The date until which you'd like to replicate data for AdCreatives and AdInsights APIs, in the format YYYY-MM-DDT00:00:00Z. All data generated between start_date and this date will be replicated. Not setting this option will result in always syncing the latest data.\"", " },", " \"account_id\" : {", " \"type\" : \"string\",", " \"title\" : \"Account Id\",", " \"description\" : \"The Facebook Ad account ID to use when pulling data from the Facebook Marketing API.\"", " },", " \"start_date\" : {", " \"type\" : \"string\",", " \"title\" : \"Start Date\",", " \"format\" : \"date-time\",", " \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",", " \"examples\" : [ \"2017-01-25T00:00:00Z\" ],", " \"description\" : \"The date from which you'd like to replicate data for AdCreatives and AdInsights APIs, in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated.\"", " },", " \"access_token\" : {", " \"type\" : \"string\",", " \"title\" : \"Access Token\",", " \"description\" : \"The value of the access token generated. See the docs for more information\",", " \"airbyte_secret\" : true", " },", " \"custom_insights\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"object\",", " \"title\" : \"InsightConfig\",", " \"required\" : [ \"name\" ],", " \"properties\" : {", " \"name\" : {", " \"type\" : \"string\",", " \"title\" : \"Name\",", " \"description\" : \"The name value of insight\"", " },", " \"fields\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Fields\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen fields for fields parameter\"", " },", " \"breakdowns\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Breakdowns\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen breakdowns for breakdowns\"", " },", " \"action_breakdowns\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Action Breakdowns\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen action_breakdowns for action_breakdowns\"", " }", " }", " },", " \"title\" : \"Custom Insights\",", " \"description\" : \"A list wich contains insights entries, each entry must have a name and can contains fields, breakdowns or action_breakdowns)\"", " },", " \"include_deleted\" : {", " \"type\" : \"boolean\",", " \"title\" : \"Include Deleted\",", " \"default\" : false,", " \"description\" : \"Include data from deleted campaigns, ads, and adsets.\"", " },", " \"insights_days_per_job\" : {", " \"type\" : \"integer\",", " \"title\" : \"Insights Days Per Job\",", " \"default\" : 7,", " \"maximum\" : 30,", " \"minimum\" : 1,", " \"description\" : \"Number of days to sync in one job. The more data you have - the smaller you want this parameter to be.\"", " },", " \"insights_lookback_window\" : {", " \"type\" : \"integer\",", " \"title\" : \"Insights Lookback Window\",", " \"default\" : 28,", " \"maximum\" : 28,", " \"minimum\" : 0,", " \"description\" : \"The attribution window for the actions\"", " }", " },", " \"definitions\" : {", " \"InsightConfig\" : {", " \"type\" : \"object\",", " \"title\" : \"InsightConfig\",", " \"required\" : [ \"name\" ],", " \"properties\" : {", " \"name\" : {", " \"type\" : \"string\",", " \"title\" : \"Name\",", " \"description\" : \"The name value of insight\"", " },", " \"fields\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Fields\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen fields for fields parameter\"", " },", " \"breakdowns\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Breakdowns\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen breakdowns for breakdowns\"", " },", " \"action_breakdowns\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Action Breakdowns\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen action_breakdowns for action_breakdowns\"", " }", " }", " }", " }", "}", "\tat io.airbyte.server.apis.ConfigurationApi.execute(ConfigurationApi.java:685)", "\tat io.airbyte.server.apis.ConfigurationApi.checkConnectionToSourceForUpdate(ConfigurationApi.java:375)", "\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)", "\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)", "\tat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)", "\tat java.base/java.lang.reflect.Method.invoke(Method.java:564)", "\tat org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)", "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)", "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)", "\tat org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)", "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)", "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469)", "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391)", "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80)", "\tat org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253)", "\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)", "\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)", "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:292)", "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:274)", "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:244)", "\tat org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)", "\tat org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)", "\tat org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)", "\tat org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)", "\tat org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)", "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)", "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)", "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)", "\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)", "\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:569)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)", "\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)", "\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)", "\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)", "\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)", "\tat org.eclipse.jetty.server.Server.handle(Server.java:501)", "\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)", "\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)", "\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)", "\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)", "\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)", "\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)", "\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)", "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)", "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)", "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)", "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)", "\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)", "\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)", "\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)", "\tat java.base/java.lang.Thread.run(Thread.java:832)", "Caused by: io.airbyte.validation.json.JsonValidationException: json schema validation failed when comparing the data to the json schema. ", "Errors: $.end_date: does not match the regex pattern ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$, $.end_date: is an invalid date-time ", "Schema: ", "{", " \"type\" : \"object\",", " \"title\" : \"Source Facebook Marketing\",", " \"required\" : [ \"account_id\", \"access_token\", \"start_date\" ],", " \"properties\" : {", " \"end_date\" : {", " \"type\" : \"string\",", " \"title\" : \"End Date\",", " \"format\" : \"date-time\",", " \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",", " \"examples\" : [ \"2017-01-26T00:00:00Z\" ],", " \"description\" : \"The date until which you'd like to replicate data for AdCreatives and AdInsights APIs, in the format YYYY-MM-DDT00:00:00Z. All data generated between start_date and this date will be replicated. Not setting this option will result in always syncing the latest data.\"", " },", " \"account_id\" : {", " \"type\" : \"string\",", " \"title\" : \"Account Id\",", " \"description\" : \"The Facebook Ad account ID to use when pulling data from the Facebook Marketing API.\"", " },", " \"start_date\" : {", " \"type\" : \"string\",", " \"title\" : \"Start Date\",", " \"format\" : \"date-time\",", " \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",", " \"examples\" : [ \"2017-01-25T00:00:00Z\" ],", " \"description\" : \"The date from which you'd like to replicate data for AdCreatives and AdInsights APIs, in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated.\"", " },", " \"access_token\" : {", " \"type\" : \"string\",", " \"title\" : \"Access Token\",", " \"description\" : \"The value of the access token generated. See the docs for more information\",", " \"airbyte_secret\" : true", " },", " \"custom_insights\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"object\",", " \"title\" : \"InsightConfig\",", " \"required\" : [ \"name\" ],", " \"properties\" : {", " \"name\" : {", " \"type\" : \"string\",", " \"title\" : \"Name\",", " \"description\" : \"The name value of insight\"", " },", " \"fields\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Fields\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen fields for fields parameter\"", " },", " \"breakdowns\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Breakdowns\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen breakdowns for breakdowns\"", " },", " \"action_breakdowns\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Action Breakdowns\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen action_breakdowns for action_breakdowns\"", " }", " }", " },", " \"title\" : \"Custom Insights\",", " \"description\" : \"A list wich contains insights entries, each entry must have a name and can contains fields, breakdowns or action_breakdowns)\"", " },", " \"include_deleted\" : {", " \"type\" : \"boolean\",", " \"title\" : \"Include Deleted\",", " \"default\" : false,", " \"description\" : \"Include data from deleted campaigns, ads, and adsets.\"", " },", " \"insights_days_per_job\" : {", " \"type\" : \"integer\",", " \"title\" : \"Insights Days Per Job\",", " \"default\" : 7,", " \"maximum\" : 30,", " \"minimum\" : 1,", " \"description\" : \"Number of days to sync in one job. The more data you have - the smaller you want this parameter to be.\"", " },", " \"insights_lookback_window\" : {", " \"type\" : \"integer\",", " \"title\" : \"Insights Lookback Window\",", " \"default\" : 28,", " \"maximum\" : 28,", " \"minimum\" : 0,", " \"description\" : \"The attribution window for the actions\"", " }", " },", " \"definitions\" : {", " \"InsightConfig\" : {", " \"type\" : \"object\",", " \"title\" : \"InsightConfig\",", " \"required\" : [ \"name\" ],", " \"properties\" : {", " \"name\" : {", " \"type\" : \"string\",", " \"title\" : \"Name\",", " \"description\" : \"The name value of insight\"", " },", " \"fields\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Fields\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen fields for fields parameter\"", " },", " \"breakdowns\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Breakdowns\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen breakdowns for breakdowns\"", " },", " \"action_breakdowns\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Action Breakdowns\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen action_breakdowns for action_breakdowns\"", " }", " }", " }", " }", "}", "\tat io.airbyte.validation.json.JsonSchemaValidator.ensure(JsonSchemaValidator.java:68)", "\tat io.airbyte.server.handlers.SchedulerHandler.checkSourceConnectionFromSourceIdForUpdate(SchedulerHandler.java:172)", "\tat io.airbyte.server.apis.ConfigurationApi.lambda$checkConnectionToSourceForUpdate$23(ConfigurationApi.java:375)", "\tat io.airbyte.server.apis.ConfigurationApi.execute(ConfigurationApi.java:679)", "\t... 53 more" ], "rootCauseExceptionClassName": "java.lang.Class", "rootCauseExceptionStack": [ "io.airbyte.validation.json.JsonValidationException: json schema validation failed when comparing the data to the json schema. ", "Errors: $.end_date: does not match the regex pattern ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$, $.end_date: is an invalid date-time ", "Schema: ", "{", " \"type\" : \"object\",", " \"title\" : \"Source Facebook Marketing\",", " \"required\" : [ \"account_id\", \"access_token\", \"start_date\" ],", " \"properties\" : {", " \"end_date\" : {", " \"type\" : \"string\",", " \"title\" : \"End Date\",", " \"format\" : \"date-time\",", " \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",", " \"examples\" : [ \"2017-01-26T00:00:00Z\" ],", " \"description\" : \"The date until which you'd like to replicate data for AdCreatives and AdInsights APIs, in the format YYYY-MM-DDT00:00:00Z. All data generated between start_date and this date will be replicated. Not setting this option will result in always syncing the latest data.\"", " },", " \"account_id\" : {", " \"type\" : \"string\",", " \"title\" : \"Account Id\",", " \"description\" : \"The Facebook Ad account ID to use when pulling data from the Facebook Marketing API.\"", " },", " \"start_date\" : {", " \"type\" : \"string\",", " \"title\" : \"Start Date\",", " \"format\" : \"date-time\",", " \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",", " \"examples\" : [ \"2017-01-25T00:00:00Z\" ],", " \"description\" : \"The date from which you'd like to replicate data for AdCreatives and AdInsights APIs, in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated.\"", " },", " \"access_token\" : {", " \"type\" : \"string\",", " \"title\" : \"Access Token\",", " \"description\" : \"The value of the access token generated. See the docs for more information\",", " \"airbyte_secret\" : true", " },", " \"custom_insights\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"object\",", " \"title\" : \"InsightConfig\",", " \"required\" : [ \"name\" ],", " \"properties\" : {", " \"name\" : {", " \"type\" : \"string\",", " \"title\" : \"Name\",", " \"description\" : \"The name value of insight\"", " },", " \"fields\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Fields\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen fields for fields parameter\"", " },", " \"breakdowns\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Breakdowns\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen breakdowns for breakdowns\"", " },", " \"action_breakdowns\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Action Breakdowns\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen action_breakdowns for action_breakdowns\"", " }", " }", " },", " \"title\" : \"Custom Insights\",", " \"description\" : \"A list wich contains insights entries, each entry must have a name and can contains fields, breakdowns or action_breakdowns)\"", " },", " \"include_deleted\" : {", " \"type\" : \"boolean\",", " \"title\" : \"Include Deleted\",", " \"default\" : false,", " \"description\" : \"Include data from deleted campaigns, ads, and adsets.\"", " },", " \"insights_days_per_job\" : {", " \"type\" : \"integer\",", " \"title\" : \"Insights Days Per Job\",", " \"default\" : 7,", " \"maximum\" : 30,", " \"minimum\" : 1,", " \"description\" : \"Number of days to sync in one job. The more data you have - the smaller you want this parameter to be.\"", " },", " \"insights_lookback_window\" : {", " \"type\" : \"integer\",", " \"title\" : \"Insights Lookback Window\",", " \"default\" : 28,", " \"maximum\" : 28,", " \"minimum\" : 0,", " \"description\" : \"The attribution window for the actions\"", " }", " },", " \"definitions\" : {", " \"InsightConfig\" : {", " \"type\" : \"object\",", " \"title\" : \"InsightConfig\",", " \"required\" : [ \"name\" ],", " \"properties\" : {", " \"name\" : {", " \"type\" : \"string\",", " \"title\" : \"Name\",", " \"description\" : \"The name value of insight\"", " },", " \"fields\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Fields\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen fields for fields parameter\"", " },", " \"breakdowns\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Breakdowns\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen breakdowns for breakdowns\"", " },", " \"action_breakdowns\" : {", " \"type\" : \"array\",", " \"items\" : {", " \"type\" : \"string\"", " },", " \"title\" : \"Action Breakdowns\",", " \"default\" : [ ],", " \"description\" : \"A list of chosen action_breakdowns for action_breakdowns\"", " }", " }", " }", " }", "}", "\tat io.airbyte.validation.json.JsonSchemaValidator.ensure(JsonSchemaValidator.java:68)", "\tat io.airbyte.server.handlers.SchedulerHandler.checkSourceConnectionFromSourceIdForUpdate(SchedulerHandler.java:172)", "\tat io.airbyte.server.apis.ConfigurationApi.lambda$checkConnectionToSourceForUpdate$23(ConfigurationApi.java:375)", "\tat io.airbyte.server.apis.ConfigurationApi.execute(ConfigurationApi.java:679)", "\tat io.airbyte.server.apis.ConfigurationApi.checkConnectionToSourceForUpdate(ConfigurationApi.java:375)", "\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)", "\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)", "\tat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)", "\tat java.base/java.lang.reflect.Method.invoke(Method.java:564)", "\tat org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)", "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)", "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)", "\tat org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)", "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)", "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469)", "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391)", "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80)", "\tat org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253)", "\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)", "\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)", "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:292)", "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:274)", "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:244)", "\tat org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)", "\tat org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)", "\tat org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)", "\tat org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)", "\tat org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)", "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)", "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)", "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)", "\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)", "\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:569)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)", "\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)", "\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)", "\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)", "\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)", "\tat org.eclipse.jetty.server.Server.handle(Server.java:501)", "\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)", "\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)", "\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)", "\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)", "\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)", "\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)", "\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)", "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)", "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)", "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)", "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)", "\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)", "\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)", "\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)", "\tat java.base/java.lang.Thread.run(Thread.java:832)" ] }Steps to Reproduce
Create a Facebook Marketing source (using integration test creds)
After try to remove the end date params
Are you willing to submit a PR?
Remove this with your answer.