"\n" escape sequences instead of new lines in copied Keep notes

matthewlmcclure commented 2 years ago

The commands documented at https://github.com/taers232c/GAMADV-XTD3/wiki/Users-Keep#copy-notes-and-permissions-from-one-user-to-another result in notes with "\n" escape sequences instead of new lines in Keep.

gam redirect csv ./notes.csv user user1@domain.com print notes formatjson quotechar "'"
gam csv ./notes.csv quotechar "'" gam user user2@domain.com create note json "~JSON" copyacls

taers232c commented 2 years ago

Matt,

Send me a Meet/Zoom invitation

Ross

taers232c commented 2 years ago

Matt,

This should be fixed in 6.16.10.

I figured out the query.

gam user user@domain.com show notes filter "create_time > \"2022-03-09T15:00:00-08:00\""

matthewlmcclure commented 2 years ago

Hm. The command with filter didn't work for me.

$ gam user user@example.com show notes filter "create_time > "2022-03-09T15:00:00-08:00""
User: user@example.com, Note Show Failed: Request contains an invalid argument.

I also tried spelling the createTime parameter with CamelCase instead of snake_case. Same result.

$ gam user user@example.com show notes filter "createTime > "2022-03-09T15:00:00-08:00""
User: user@example.com, Note Show Failed: Request contains an invalid argument.

$ gam version
GAMADV-XTD3 6.16.10 - https://github.com/taers232c/GAMADV-XTD3 - pyinstaller
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.10.2 64-bit final
MacOS Big Sur 11.6 x86_64
...

taers232c commented 2 years ago

gam user user@example.com show notes filter "create_time > \"2022-03-09T15:00:00-08:00\""

You need the backslashes

Send me a Meet invitation

taers232c commented 2 years ago

gam user user@example.com show notes filter "create_time > \"2022-03-09T15:00:00-08:00\""

GitHub is tossing the backslashes

matthewlmcclure commented 2 years ago

That works. Crazy that Google's API uses snake_case when the parameter is documented using camelCase.

taers232c commented 2 years ago

That and almost every other query wants strings in single quotes not double quotes.

This should let your toss the badly imported notes and reimport

matthewlmcclure commented 2 years ago

I think something is still not quite right in versions 6.6.10 and 6.6.11 with respect to escaping characters in note text.

I have some Keep notes that contain single quotes and double quotes, so I tried using tab as the value of quotechar as follows.

$ gam redirect csv ./notes.csv user user@example.com print notes formatjson quotechar '       ' # I inserted a literal tab character in the real command using Control-V, tab. When I paste it into GitHub's comment editor, it pastes 7 spaces instead.

$ gam csv ../notes.csv quotechar "       " gam user user@example.com create note json "~JSON" copyacls
2022-03-10T03:31:43.497+00:00,0/1,Using 1 process...
Command: gam user user@example.com create note json >>>"{"body": {"text": {"text": "https://github.com/taers232c/GAMADV-XTD3/wiki/Users-Keep\\n\\nCopy notes and permissions from one user to another.\\ngam redirect csv ./notes.csv user user1@domain.com print notes formatjson quotechar \\"'\\"\\ngam csv ./notes.csv quotechar \\"'\\" gam user user2@domain.com create note json \\"~JSON\\" copyacls\\n\\nGoogle Keep\\nBackup Restore\\nImport Export\\nMigrate\\nTransfer\\nGAM\\nGAMADV-XTD3"}}, "createTime": "2022-03-04T13:14:59Z", "name": "notes/1X6-eSgnbTmVlhqthCR_977N5QjU9jMxAEu-fusNAn4lQLulZOBg1wFA3_aHUC0ZUDTmH", "ownedByMe": true, "owner": "user@example.com", "permissions": [{"email": "user@example.com", "name": "notes/1X6-eSgnbTmVlhqthCR_977N5QjU9jMxAEu-fusNAn4lQLulZOBg1wFA3_aHUC0ZUDTmH/permissions/AfoJJ7Zibw0YeZH75trnRApOnHvsMfeqEZ-BwqzH", "role": "OWNER", "user": {"email": "user@example.com"}}], "trashTime": "Never", "updateTime": "2022-03-04T13:15:37Z"}"<<< copyacls

ERROR: Expecting ',' delimiter: line 1 column 233 (char 232): {"body": {"text": {"text": "https://github.com/taers232c/GAMADV-XTD3/wiki/Users-Keep\\n\\nCopy notes and permissions from one user to another.\\ngam redirect csv ./notes.csv user user1@domain.com print notes formatjson quotechar \\"'\\"\\ngam csv ./notes.csv quotechar \\"'\\" gam user user2@domain.com create note json \\"~JSON\\" copyacls\\n\\nGoogle Keep\\nBackup Restore\\nImport Export\\nMigrate\\nTransfer\\nGAM\\nGAMADV-XTD3"}}, "createTime": "2022-03-04T13:14:59Z", "name": "notes/1X6-eSgnbTmVlhqthCR_977N5QjU9jMxAEu-fusNAn4lQLulZOBg1wFA3_aHUC0ZUDTmH", "ownedByMe": true, "owner": "user@example.com", "permissions": [{"email": "user@example.com", "name": "notes/1X6-eSgnbTmVlhqthCR_977N5QjU9jMxAEu-fusNAn4lQLulZOBg1wFA3_aHUC0ZUDTmH/permissions/AfoJJ7Zibw0YeZH75trnRApOnHvsMfeqEZ-BwqzH", "role": "OWNER", "user": {"email": "user@example.com"}}], "trashTime": "Never", "updateTime": "2022-03-04T13:15:37Z"}
Help: Syntax in file /Users/matthewlmcclure/code/gamadv-xtd3/gamadv-xtd3/GamCommands.txt
Help: Documentation is at https://github.com/taers232c/GAMADV-XTD3/wiki

Notice above that the JSON contains sequences like \\n and \\". I believe those should be \n and \" respectively.

I edited notes.csv accordingly. I replaced \\n with \n and I replace \\" with \" throughout the JSON body. Then I re-ran the above command, and it worked as expected.

$ gam csv ../gam-continuousdelta.com/notes-4-head.csv quotechar "       " gam user user@example.com create note json "~JSON" copyacls
2022-03-10T03:46:57.891+00:00,0/1,Using 1 process...
Note: notes/1ijfsWK9MTQE_-6j-fP_dmuzo_sByN16N4FJPQdl6UBhIy0F90vgTwDx9t31duA
  title:
  createTime: 2022-03-10T03:46:58Z
  updateTime: 2022-03-10T03:46:58Z
  trashTime: Never
  owner: user@example.com
  ownedByMe: True
  permissions:
    name: notes/1ijfsWK9MTQE_-6j-fP_dmuzo_sByN16N4FJPQdl6UBhIy0F90vgTwDx9t31duA/permissions/AfoJJ7ZHFg2WqB-REaP4RbU-ma6WieNFDjFPPkR5
      role: OWNER
      user: user@example.com
  text:
    https://github.com/taers232c/GAMADV-XTD3/wiki/Users-Keep

    Copy notes and permissions from one user to another.
    gam redirect csv ./notes.csv user user1@domain.com print notes formatjson quotechar "'"
    gam csv ./notes.csv quotechar "'" gam user user2@domain.com create note json "~JSON" copyacls

    Google Keep
    Backup Restore
    Import Export
    Migrate
    Transfer
    GAM
    GAMADV-XTD3

taers232c commented 2 years ago

Matt,

If you're available send me a Meet./Zoom invitation.

The \n will be converted to \n.

The single and double quotes in thet text shold be handled properly.

Ross

@.***

On Mar 9, 2022, at 8:06 PM, Matt McClure @.***> wrote:

I think something is still not quite right in versions 6.6.10 and 6.6.11 with respect to escaping characters in note text.

I have some Keep notes that contain single quotes and double quotes, so I tried using tab as the value of quotechar as follows.

$ gam redirect csv ./notes.csv user @. print notes formatjson quotechar ' ' # I inserted a literal tab character in the real command using Control-V, tab. When I paste it into GitHub's comment editor, it pastes 7 spaces instead. $ gam csv ../notes.csv quotechar " " gam user @. create note json "~JSON" copyacls 2022-03-10T03:31:43.497+00:00,0/1,Using 1 process... Command: gam user @. create note json >>>"{"body": {"text": {"text": "https://github.com/taers232c/GAMADV-XTD3/wiki/Users-Keep\\n\\nCopy notes and permissions from one user to another.\ngam redirect csv ./notes.csv user @. print notes formatjson quotechar \"'\"\ngam csv ./notes.csv quotechar \"'\" gam user @. create note json \"~JSON\" copyacls\n\nGoogle Keep\nBackup Restore\nImport Export\nMigrate\nTransfer\nGAM\nGAMADV-XTD3"}}, "createTime": "2022-03-04T13:14:59Z", "name": "notes/1X6-eSgnbTmVlhqthCR_977N5QjU9jMxAEu-fusNAn4lQLulZOBg1wFA3_aHUC0ZUDTmH", "ownedByMe": true, "owner": @.", "permissions": [{"email": @.", "name": "notes/1X6-eSgnbTmVlhqthCR_977N5QjU9jMxAEu-fusNAn4lQLulZOBg1wFA3_aHUC0ZUDTmH/permissions/AfoJJ7Zibw0YeZH75trnRApOnHvsMfeqEZ-BwqzH", "role": "OWNER", "user": {"email": @."}}], "trashTime": "Never", "updateTime": "2022-03-04T13:15:37Z"}"<<< copyacls

ERROR: Expecting ',' delimiter: line 1 column 233 (char 232): {"body": {"text": {"text": "https://github.com/taers232c/GAMADV-XTD3/wiki/Users-Keep\\n\\nCopy notes and permissions from one user to another.\ngam redirect csv ./notes.csv user @. print notes formatjson quotechar \"'\"\ngam csv ./notes.csv quotechar \"'\" gam user @. create note json \"~JSON\" copyacls\n\nGoogle Keep\nBackup Restore\nImport Export\nMigrate\nTransfer\nGAM\nGAMADV-XTD3"}}, "createTime": "2022-03-04T13:14:59Z", "name": "notes/1X6-eSgnbTmVlhqthCR_977N5QjU9jMxAEu-fusNAn4lQLulZOBg1wFA3_aHUC0ZUDTmH", "ownedByMe": true, "owner": @.", "permissions": [{"email": @.", "name": "notes/1X6-eSgnbTmVlhqthCR_977N5QjU9jMxAEu-fusNAn4lQLulZOBg1wFA3_aHUC0ZUDTmH/permissions/AfoJJ7Zibw0YeZH75trnRApOnHvsMfeqEZ-BwqzH", "role": "OWNER", "user": {"email": @.***"}}], "trashTime": "Never", "updateTime": "2022-03-04T13:15:37Z"} Help: Syntax in file /Users/matthewlmcclure/code/gamadv-xtd3/gamadv-xtd3/GamCommands.txt Help: Documentation is at https://github.com/taers232c/GAMADV-XTD3/wiki Notice above that the JSON contains sequences like \n and \". I believe those should be \n and \" respectively.

I edited notes.csv accordingly. I replaced \n with \n and I replace \" with \" throughout the JSON body. Then I re-ran the above command, and it worked as expected.

$ gam csv ../gam-continuousdelta.com/notes-4-head.csv quotechar " " gam user @. create note json "~JSON" copyacls 2022-03-10T03:46:57.891+00:00,0/1,Using 1 process... Note: notes/1ijfsWK9MTQE_-6j-fP_dmuzo_sByN16N4FJPQdl6UBhIy0F90vgTwDx9t31duA title: createTime: 2022-03-10T03:46:58Z updateTime: 2022-03-10T03:46:58Z trashTime: Never owner: @. ownedByMe: True permissions: name: notes/1ijfsWK9MTQE_-6j-fP_dmuzo_sByN16N4FJPQdl6UBhIy0F90vgTwDx9t31duA/permissions/AfoJJ7ZHFg2WqB-REaP4RbU-ma6WieNFDjFPPkR5 role: OWNER user: @.*** text: https://github.com/taers232c/GAMADV-XTD3/wiki/Users-Keep
Copy notes and permissions from one user to another.
gam redirect csv ./notes.csv user ***@***.*** print notes formatjson quotechar "'"
gam csv ./notes.csv quotechar "'" gam user ***@***.*** create note json "~JSON" copyacls

Google Keep
Backup Restore
Import Export
Migrate
Transfer
GAM
GAMADV-XTD3
— Reply to this email directly, view it on GitHub https://github.com/taers232c/GAMADV-XTD3/issues/251#issuecomment-1063642769, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCTYL6SWEG3GLWOHAEVWJDU7FYLVANCNFSM5QJ5B5FQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you commented.

taers232c commented 2 years ago

Matt,

json.dumps is causing trouble

Ross

@.***

On Mar 9, 2022, at 8:06 PM, Matt McClure @.***> wrote:

I think something is still not quite right in versions 6.6.10 and 6.6.11 with respect to escaping characters in note text.

I have some Keep notes that contain single quotes and double quotes, so I tried using tab as the value of quotechar as follows.

$ gam redirect csv ./notes.csv user @. print notes formatjson quotechar ' ' # I inserted a literal tab character in the real command using Control-V, tab. When I paste it into GitHub's comment editor, it pastes 7 spaces instead. $ gam csv ../notes.csv quotechar " " gam user @. create note json "~JSON" copyacls 2022-03-10T03:31:43.497+00:00,0/1,Using 1 process... Command: gam user @. create note json >>>"{"body": {"text": {"text": "https://github.com/taers232c/GAMADV-XTD3/wiki/Users-Keep\\n\\nCopy notes and permissions from one user to another.\ngam redirect csv ./notes.csv user @. print notes formatjson quotechar \"'\"\ngam csv ./notes.csv quotechar \"'\" gam user @. create note json \"~JSON\" copyacls\n\nGoogle Keep\nBackup Restore\nImport Export\nMigrate\nTransfer\nGAM\nGAMADV-XTD3"}}, "createTime": "2022-03-04T13:14:59Z", "name": "notes/1X6-eSgnbTmVlhqthCR_977N5QjU9jMxAEu-fusNAn4lQLulZOBg1wFA3_aHUC0ZUDTmH", "ownedByMe": true, "owner": @.", "permissions": [{"email": @.", "name": "notes/1X6-eSgnbTmVlhqthCR_977N5QjU9jMxAEu-fusNAn4lQLulZOBg1wFA3_aHUC0ZUDTmH/permissions/AfoJJ7Zibw0YeZH75trnRApOnHvsMfeqEZ-BwqzH", "role": "OWNER", "user": {"email": @."}}], "trashTime": "Never", "updateTime": "2022-03-04T13:15:37Z"}"<<< copyacls

ERROR: Expecting ',' delimiter: line 1 column 233 (char 232): {"body": {"text": {"text": "https://github.com/taers232c/GAMADV-XTD3/wiki/Users-Keep\\n\\nCopy notes and permissions from one user to another.\ngam redirect csv ./notes.csv user @. print notes formatjson quotechar \"'\"\ngam csv ./notes.csv quotechar \"'\" gam user @. create note json \"~JSON\" copyacls\n\nGoogle Keep\nBackup Restore\nImport Export\nMigrate\nTransfer\nGAM\nGAMADV-XTD3"}}, "createTime": "2022-03-04T13:14:59Z", "name": "notes/1X6-eSgnbTmVlhqthCR_977N5QjU9jMxAEu-fusNAn4lQLulZOBg1wFA3_aHUC0ZUDTmH", "ownedByMe": true, "owner": @.", "permissions": [{"email": @.", "name": "notes/1X6-eSgnbTmVlhqthCR_977N5QjU9jMxAEu-fusNAn4lQLulZOBg1wFA3_aHUC0ZUDTmH/permissions/AfoJJ7Zibw0YeZH75trnRApOnHvsMfeqEZ-BwqzH", "role": "OWNER", "user": {"email": @.***"}}], "trashTime": "Never", "updateTime": "2022-03-04T13:15:37Z"} Help: Syntax in file /Users/matthewlmcclure/code/gamadv-xtd3/gamadv-xtd3/GamCommands.txt Help: Documentation is at https://github.com/taers232c/GAMADV-XTD3/wiki Notice above that the JSON contains sequences like \n and \". I believe those should be \n and \" respectively.

I edited notes.csv accordingly. I replaced \n with \n and I replace \" with \" throughout the JSON body. Then I re-ran the above command, and it worked as expected.

$ gam csv ../gam-continuousdelta.com/notes-4-head.csv quotechar " " gam user @. create note json "~JSON" copyacls 2022-03-10T03:46:57.891+00:00,0/1,Using 1 process... Note: notes/1ijfsWK9MTQE_-6j-fP_dmuzo_sByN16N4FJPQdl6UBhIy0F90vgTwDx9t31duA title: createTime: 2022-03-10T03:46:58Z updateTime: 2022-03-10T03:46:58Z trashTime: Never owner: @. ownedByMe: True permissions: name: notes/1ijfsWK9MTQE_-6j-fP_dmuzo_sByN16N4FJPQdl6UBhIy0F90vgTwDx9t31duA/permissions/AfoJJ7ZHFg2WqB-REaP4RbU-ma6WieNFDjFPPkR5 role: OWNER user: @.*** text: https://github.com/taers232c/GAMADV-XTD3/wiki/Users-Keep
Copy notes and permissions from one user to another.
gam redirect csv ./notes.csv user ***@***.*** print notes formatjson quotechar "'"
gam csv ./notes.csv quotechar "'" gam user ***@***.*** create note json "~JSON" copyacls

Google Keep
Backup Restore
Import Export
Migrate
Transfer
GAM
GAMADV-XTD3
— Reply to this email directly, view it on GitHub https://github.com/taers232c/GAMADV-XTD3/issues/251#issuecomment-1063642769, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCTYL6SWEG3GLWOHAEVWJDU7FYLVANCNFSM5QJ5B5FQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you commented.

matthewlmcclure commented 2 years ago

"\\"" is an invalid JSON string. The \\ is an escaped \, and the immediately following " closes the string.

A one-character string consisting of " should be JSON-encoded and quoted as "\"".

Here's a more minimalist test case: https://gist.github.com/matthewlmcclure/ffbd7deae00215c954bee77bf3e71788

taers232c commented 2 years ago

test.py

#!/usr/bin/env python3                                                                                                                                                                                                                                                        

import json

foo = {"test": "abc \"def\" 'ghi' jkl\n"}
print("Orig:", foo)
row = {}
row["TEST"] = json.dumps(foo, ensure_ascii=False, sort_keys=True)
print("row['TEST]': ", row['TEST'])
print("row:", row)

$ ./test.py
Orig: {'test': 'abc "def" \'ghi\' jkl\n'}
row['TEST]':  {"test": "abc \"def\" 'ghi' jkl\n"}
row: {'TEST': '{"test": "abc \\"def\\" \'ghi\' jkl\\n"}'}```

taers232c commented 2 years ago

Matt,

A fix for Python 3.10.0 breaks things in 3.10.2

Try 6.16.12

You can say quotechar "\t" to set the quote character to tab

matthewlmcclure commented 2 years ago

I don't see a 6.6.12 release at https://github.com/taers232c/GAMADV-XTD3/releases

taers232c commented 2 years ago

6.16.13 available

@.***

On Mar 10, 2022, at 10:40 AM, Matt McClure @.***> wrote:

I don't see a 6.6.12 release at https://github.com/taers232c/GAMADV-XTD3/releases https://github.com/taers232c/GAMADV-XTD3/releases — Reply to this email directly, view it on GitHub https://github.com/taers232c/GAMADV-XTD3/issues/251#issuecomment-1064375461, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCTYL2Q4OMIQNMJRKN76Q3U7I625ANCNFSM5QJ5B5FQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you commented.

matt-continuousdelta commented 2 years ago

6.6.13 seems to have addressed this issue. Thanks!

taers232c commented 1 year ago

I tossed it, the fix to the fix was broken, it'll be back shortly

@.***

On Mar 10, 2022, at 10:40 AM, Matt McClure @.***> wrote:

I don't see a 6.6.12 release at https://github.com/taers232c/GAMADV-XTD3/releases https://github.com/taers232c/GAMADV-XTD3/releases — Reply to this email directly, view it on GitHub https://github.com/taers232c/GAMADV-XTD3/issues/251#issuecomment-1064375461, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCTYL2Q4OMIQNMJRKN76Q3U7I625ANCNFSM5QJ5B5FQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you commented.

taers232c / GAMADV-XTD3