Migration to Simperium API needed by October 2018

casutherland commented 6 years ago

The deprecated API issue (#63) is resolved, for now, so we should be able to sync again temporarily.

@roundhill said:

That said, we will be deprecating the old API in October so a migration to the Simperium API is still needed here.

The API this is currently using hasn't been supported for many years. We can provide an API key that will allow access to Simplenote.

Simperium HTTP - API reference

Simperium Python - Getting Started

samuelallan72 commented 6 years ago

I'm currently in discussion with Automattic with the aim of obtaining an API key and find out anything else required to begin porting sncli to the new api.

samuelallan72 commented 6 years ago

Update: Simplenote has provided me with details required to use sncli with the new api. I'll begin working on porting sncli to Simperium within the next couple of weeks (as soon as I get some free time).

samuelallan72 commented 6 years ago

The upstream python library for Simperium (which would be nice to use instead of rolling our own) doesn't currently support python3. Issue reported at https://github.com/Simperium/simperium-python/issues/10 with my thoughts.

samuelallan72 commented 6 years ago

Update: I have successfully ported the simperium python client to python 3. It can be found at https://github.com/swalladge/simperium-python3

I have done some tinkering with sncli to use the new library - auth is working fine, but getting the notes sync will be interesting due to changes to the note representation from the server. For example, we can no longer rely on syncnum (gone), version is no longer part of the note object, but returned with the note index function, etc.

Help wanted to test the python3 port and with porting sncli to the new api! :smiley:

samuelallan72 commented 6 years ago

Please see pull request. ^

N0ury commented 6 years ago

Hi swalladge I have installed successfuly simperium-python (python2). Everything works fine. I have tried to installed yours but install fails. I explain:

my default is python2. I dont't want to change this
I have used sudo -H pip install git+https://github.com/swalladge/simperium-python3.git to install your python3 simperium (Simperium3)
I have the error: Simperium3 requires Python '>=3' but the running Python is 2.7.15 Maybe this means you use python and not python3 in some scripts. Do you have any idea to sole this?

samuelallan72 commented 6 years ago

@nbenm sncli only supports python3, and therefore requires the python3 port of the simperium python client.

(You may need to use pip3 or python3 in your commands, or use a virtual environment, so that python 3 is used.)

N0ury commented 6 years ago

I Know. Both (python2 and python3) are installed on my Mac. I use Macports and youtube-dl package uses python3 with no problem. For all other packages, and for my personal apps, I use python2. What I would like to avoid is to have python3 as default I want python2 remains my default.

For the moment, I can’t install Simperium3, it fails. It wants to use my default python2.

Le 11 sept. 2018 à 09:11:54, Samuel Walladge notifications@github.com a écrit :

@nbenm sncli only supports python3, and therefore requires the python3 port of the simperium python client.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

samuelallan72 commented 6 years ago

I don't understand. Did you try installing the simperium3 package either in a python3 virtual environment or with pip3?

N0ury commented 6 years ago

Sorry, it’s my fault. I was using pip which defaults to pip version 2. Using « pip-3.7 » solved the problem.

I’ve tried successfully the Simperium3 example. Only last instruction fails:

api.todo.index()['count'] Traceback (most recent call last): File "", line 1, in KeyError: ‘count'

But I have the same with Simperium version python2

I’ll try later today the new sncli. I’ll do a feedback.

Le 11 sept. 2018 à 09:37:03, Samuel Walladge notifications@github.com a écrit :

I don't understand. Did you try installing the simperium3 package either in a python3 virtual environment or with pip3?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

N0ury commented 6 years ago

This is my first try. I have created a new Simplenote account. I have then created notes from web and from sncli. Common actions work well.

I have tried to import my 229 old notes to the new account

for file in ../Simplenote/*.json
do
    cat $file|sncli3 -c ~/.snclirc3 import -
    sleep 1
done

Only 166/229 have been imported. There are many errors in sncli.log. Here are some:

2018-09-11 15:41:48,719 [DEBUG] Starting new HTTPS connection (1): simple-note.appspot.com:443
2018-09-11 15:41:49,087 [DEBUG] https://simple-note.appspot.com:443 "POST /api/login HTTP/1.1" 500 0
2018-09-11 15:41:49,090 [DEBUG] AUTHENTICATE: login failed with status 500, check credentials
2018-09-11 15:41:49,093 [DEBUG] Starting new HTTPS connection (1): simple-note.appspot.com:443
2018-09-11 15:41:49,474 [DEBUG] https://simple-note.appspot.com:443 "POST /api2/data?email=toto%2540dagami.org HTTP/1.1" 500 0
2018-09-11 15:41:49,477 [DEBUG] RESPONSE ERROR: 500 Server Error: Internal Server Error for url: https://simple-note.appspot.com/api2/data?email=toto%2540dagami.org
2018-09-11 15:41:49,477 [DEBUG] ERROR: Failed to sync note to server (key=eb71fab4612643fe81fce5ff3e7b2959)
2018-09-11 15:41:49,480 [DEBUG] Starting new HTTPS connection (1): simple-note.appspot.com:443
2018-09-11 15:41:49,838 [DEBUG] https://simple-note.appspot.com:443 "POST /api/login HTTP/1.1" 500 0
2018-09-11 15:41:49,842 [DEBUG] AUTHENTICATE: login failed with status 500, check credentials
2018-09-11 15:41:49,846 [DEBUG] Starting new HTTPS connection (1): simple-note.appspot.com:443
2018-09-11 15:41:50,213 [DEBUG] https://simple-note.appspot.com:443 "POST /api2/data?email=toto%2540dagami.org HTTP/1.1" 401 0
2018-09-11 15:41:50,216 [DEBUG] RESPONSE ERROR: 401 Client Error: Unauthorized for url: https://simple-note.appspot.com/api2/data?email=toto%2540dagami.org
2018-09-11 15:41:50,217 [DEBUG] ERROR: Failed to sync note to server (key=ee6149f9dab14d199e5d4c28c10205fd)

2018-09-11 15:41:50,977 [DEBUG] Starting new HTTPS connection (1): simple-note.appspot.com:443
2018-09-11 15:41:51,335 [DEBUG] https://simple-note.appspot.com:443 "POST /api/login HTTP/1.1" 500 0
2018-09-11 15:41:51,338 [DEBUG] AUTHENTICATE: login failed with status 500, check credentials
2018-09-11 15:41:51,340 [DEBUG] Starting new HTTPS connection (1): simple-note.appspot.com:443
2018-09-11 15:41:51,701 [DEBUG] https://simple-note.appspot.com:443 "GET /api2/index?email=toto%2540dagami.org&length=100 HTTP/1.1" 401 0
2018-09-11 15:41:51,705 [DEBUG] ERROR: Failed to get note list from server
2018-09-11 15:41:51,705 [DEBUG] Saved note to disk (key=efa57d50c0ba4c5c8a179d774dfe5886)
2018-09-11 15:41:51,705 [DEBUG] Full sync completed

It looks like sncli make retries, and some seem to work.

By all my old notes haven't been imported.

I'll do more tests

N0ury commented 6 years ago

Here's an other issue: If I delete a note via the web, and then sync with sncli, the note is not removed with sncli sync. Nothing is logged for this action. But trashing from sncli moves correctly the note to trash

samuelallan72 commented 6 years ago

@nbenm thanks for testing! Yes, I think I need to write up a new syncing algorithm for sncli - the old one isn't going to work with the new api.

samuelallan72 commented 6 years ago

@nbenm Oh I just read through the logs you posted. It seems that it is still the old sncli version since it mentions "simple-note.appspot.com:443" which is the old api.

samuelallan72 commented 6 years ago

I'm working on a new syncing algorithm now. Perhaps refrain from testing the next branch until I've got something somewhat working.

I would also stress that if you want to help test the new parts I'm working on, please use at own risk. Either use an account that doesn't have anything important saved on it, or make a full backup of your notes first. I really appreciate help with testing, but I don't want anyone to lose any notes!

N0ury commented 6 years ago

I understand. I’m using a new account for these tests. Please let me know when I can do tests again.

Le 12 sept. 2018 à 03:50:52, Samuel Walladge notifications@github.com a écrit :

I'm working on a new syncing algorithm now. Perhaps refrain from testing the next branch until I've got something somewhat working.

I would also stress that if you want to help test the new parts I'm working on, please use at own risk. Either use an account that doesn't have anything important saved on it, or make a full backup of your notes first. I don't want anyone to lose any notes!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

samuelallan72 commented 6 years ago

@nbenm the latest commit I added makes it somewhat usable. Feel free to test. See the latest commit message for known issues. :)

PS: please continue conversation about the code/bugs/testing in the PR thread #67. That will leave this thread clear for general discussion. Also it will be easier to line up code/testing discussion with the progress on the code.

insanum / sncli

Migration to Simperium API needed by October 2018 #64