Newline characters in description

[mrVanDalo]

I tested the app with a dummy account on my taskserver, all good, but when I synced with my real account I got the error :

FormatException

FormatException: Taskwarrior documentation on JSON format indicates your task description should not contain newline characters. (at character 63)
Ticket text ticket text ticket text ticket text ticket text.
                                                              ^

#0      validateTaskDescription (package:taskw/src/validate.dart:15)
#1      new Task._ (package:taskj/src/json/task.dart:40)
#2      new _$Task._ (package:taskj/src/json/task.g.dart:362)
#3      TaskBuilder.build (package:taskj/src/json/task.g.dart:624)
#4      _$TaskSerializer.deserialize (package:taskj/src/json/task.g.dart:283)
#5      BuiltJsonSerializers._deserialize (package:built_value/src/built_json_serializers.dart:175)
#6      BuiltJsonSerializers.deserialize (package:built_value/src/built_json_serializers.dart:124)
#7      BuiltJsonSerializers.deserializeWith (package:built_value/src/built_json_serializers.dart:42)
#8      Task.fromJson (package:taskj/src/json/task.dart:57)
#9      Data.mergeSynchronizeResponse (package:taskc/src/home/impl/data.dart:187)
#10     Home.synchronize (package:taskc/src/home/home.dart:44)
<asynchronous suspension>
#11     _StorageWidgetState.synchronize (package:task/src/widgets/storage_widget.dart:246)
<asynchronous suspension>

I replaced the description text, because it is kinda private.

I have a lot of tickets using multiple lines. I hope this is easy to fix ( I'm not fit in dart, otherwise this would be a pull request )

[bradyt]

That error is alluding to the following:

Attribute: description

When a task is created, it MUST have a "description" field value, which contains UTF8 characters. A "description" field may not contain newline characters, but may contain other characters, properly escaped. See https://json.org/ for details.

-- https://taskwarrior.org/docs/design/task.html#attr_description

I'm not considering PRs at this time. There's a note on that still buried in the wiki. Anyways the logic is deliberate, at https://github.com/bradyt/taskw-dart/blob/v0.2.6/taskw/lib/src/validate.dart, so to simply remove it, would be trivial.

  } else if (description.contains('\n')) {
    throw FormatException(
      'Taskwarrior documentation on JSON format indicates your task '
      'description should not contain newline characters.',
      description,
      description.indexOf('\n'),
    );
  }

Before Taskwarrior 2.6.2 was released a month ago, I would have advised to use 2.5.3, as there was an outstanding issue with Taskserver and dependencies. And I don't think Taskwarrior 2.5.3 allowed newlines in description. I've only recently bumped CI tests from 2.5.3.

Does Taskwarrior now officially support newlines in descriptions? Do you think they would consider updating tw.org repo to reflect that?

[mrVanDalo]

Hmm I'm using taskwarrior since 2017 and I've used newlines in descriptions all the time (for example to add addresse, phone numbers, links or mail content), but maybe my memory tricks me here. Taskwarrior also documents how to create these multi-line tasks. Reading the https://json.org/ it is documented that "\n" is a properly escaped "linefeed" (in the string section). I haven't checked the responses from taskserver, but task export shows properly escaped "\n" (jq does not complain :D)

From how I understood it, this is invalid json

{"a":"this
is
not valid"}

and this should be valid

{"a":"this\nis\nvalid"}

Hope I could convince you :D

[bradyt]

And I don't think Taskwarrior 2.5.3 allowed newlines in description.

Hmm I'm using taskwarrior since 2017 and I've used newlines in descriptions all the time

I think I made a mistake there, disregard my report.

Hope I could convince you :D

I am fairly convinced now, thank you for the clarification. I will probably remove that snippet from validate.dart, but I'd like to make time to just look at the context a little more before I move forward on it.

[bradyt]

I expect this to be fixed in version 0.2.7, which is in process of being released for all three stores (as listed on README), and should be available within a few days. I will close now, in hopes that the release fixes this.

I wasn't sure what would be good unit tests or integration tests to make sure we avoid any bugs in the project surrounding control characters. It's interesting that Taskwarrior can import json with invalid newlines characters like in your example above. However, Taskserver will error if an app sends these, I thinks Taskserver might be going line by line, so assumes one of the lines is a sync key, and errors about not finding the sync key.

In other words, I'm not sure in what ways this app can manifest an error relating to the Taskwarrior project's specification. If anyone has a good understanding of strings, json, escapes, etc, and can suggest how a bug can manifest, please let us know what to look out for.

If you can find a bug in usage of this app, involving control characters, I would be very interested.