Closed sarpu closed 3 years ago
hi @sarpu i've transferred this issue to the cli repo.
You can configure the URL the API will be available on as outlined in the docs at https://github.com/zooniverse/panoptes-cli#log-in-and-optionally-set-the-api-endpoint
Press enter without typing anything to keep the current value (shown in brackets). You probably don't need to change the endpoint, unless you're running your own copy of the Panoptes API.
Please configure your API end point to be the URL your local copy of panoptes is running on, i assume http://localhost:3000
.
Sorry the error message I pasted is a bit long so it's easy to skip, but I did specify the api endpoint as https://localhost:3000
on the first line.
requests.exceptions.SSLError: HTTPSConnectionPool(host='localhost', port=3000):
Your local version won't be running on https scheme, change your local endpoint URL to http://localhost:3000
as i've outlined above.
This is what I get:
(federica_zooniverse) sarp@sarp:~/projects/Panoptes$ panoptes -e 'http://localhost:3000' configure
username [sarpu]: sarp
password:
Error: Invalid endpoint supplied. Endpoint must be an HTTPS URL.
well - that is a different error, i'm not sure why it's enforcing https for localhost connections but the code seems quite opinionated, https://github.com/zooniverse/panoptes-cli/blob/1e9bf291ad568e9b9bb20eb4a5180fc3e82221fc/panoptes_cli/commands/configure.py#L36-L40
short term there are a few routes you can take:
Point 1 is probaby easiest right now, sorry i don't have a better idea :(
Longer term we can look at adding in SSL support to the API (feel free to open an issue on https://github.com/zooniverse/panoptes) but that still doesn't work well with browsers and the certificate trust model (warnings / aborted connections abound)
I don't think I was thinking about people running the API on localhost when I wrote that check. A quick way around it should be to manually edit ~/.panoptes/config.yaml to change the URL.
@camallen I was about to submit a PR to that effect, but I was having issues running the panoptes-cli
using docker (to test before I submit, per the contributing guidelines). More specifically, making the panoptes-cli
running in docker to talk to the panoptes backend
running in docker. I am finding some blogposts about managing network namespaces/interfaces with docker, but if you have a quick answer to achieve that off the top of your head I'd be grateful.
@adammcmaster I don't think that would be necessary since running panoptes-cli with -e 'localhost:3000'
automatically sets the URL in ~/.panoptes/config.yaml
. I just checked for example, and it is indeed localhost.
So now when I try to connect to panoptes running in docker from the panoptes-cli client on my host and run PANOPTES_DEBUG=true panoptes -e 'http://localhost:3000' project create "My Project" "This is a description of my project"
, I am getting the following error:
(debug_panoptes) sarp@sarp:~/projects$ PANOPTES_DEBUG=true panoptes -e 'http://localhost:3000' project create "My Project" "This is a description of my project"
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:3000
DEBUG:urllib3.connectionpool:http://localhost:3000 "GET /users/sign_in HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:http://localhost:3000 "POST /users/sign_in HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:http://localhost:3000 "GET /users/sign_in HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:http://localhost:3000 "POST /users/sign_in HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:http://localhost:3000 "POST /oauth/token HTTP/1.1" 422 None
Traceback (most recent call last):
File "/home/sarp/.pyenv/versions/debug_panoptes/bin/panoptes", line 11, in <module>
load_entry_point('panoptescli', 'console_scripts', 'panoptes')()
File "/home/sarp/.pyenv/versions/3.6.9/envs/debug_panoptes/lib/python3.6/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/home/sarp/.pyenv/versions/3.6.9/envs/debug_panoptes/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/home/sarp/.pyenv/versions/3.6.9/envs/debug_panoptes/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/sarp/.pyenv/versions/3.6.9/envs/debug_panoptes/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/sarp/.pyenv/versions/3.6.9/envs/debug_panoptes/lib/python3.6/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/sarp/.pyenv/versions/3.6.9/envs/debug_panoptes/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/sarp/projects/panoptes-cli/panoptes_cli/commands/project.py", line 115, in create
project.save()
File "/home/sarp/projects/panoptes-python-client/panoptes_client/panoptes.py", line 815, in save
etag=self.etag
File "/home/sarp/projects/panoptes-python-client/panoptes_client/panoptes.py", line 404, in post
retry=retry,
File "/home/sarp/projects/panoptes-python-client/panoptes_client/panoptes.py", line 267, in json_request
retry=retry,
File "/home/sarp/projects/panoptes-python-client/panoptes_client/panoptes.py", line 191, in http_request
token = self.get_bearer_token()
File "/home/sarp/projects/panoptes-python-client/panoptes_client/panoptes.py", line 551, in get_bearer_token
bearer_data
File "/home/sarp/.pyenv/versions/3.6.9/envs/debug_panoptes/lib/python3.6/site-packages/requests/models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "/home/sarp/.pyenv/versions/3.6.9/lib/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/home/sarp/.pyenv/versions/3.6.9/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/sarp/.pyenv/versions/3.6.9/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The panoptes container output corresponding to this command is as follows:
sidekiq_1 | `Redis#exists(key)` will return an Integer in redis-rb 4.3. `exists?` returns a boolean, you should use it instead. To opt-in to the new behavior now you can set Redis.exists_returns_integer = true. To disable this message and keep the current (boolean) behaviour of 'exists' you can set `Redis.exists_returns_integer = false`, but this option will be removed in 5.0. (/usr/local/bundle/gems/sidekiq-5.2.8/lib/sidekiq/launcher.rb:105:in `block (2 levels) in ❤')
panoptes_1 | [5fe1e66d-cfec-4cf8-b17e-7736af956479]
panoptes_1 | [5fe1e66d-cfec-4cf8-b17e-7736af956479]
panoptes_1 | [5fe1e66d-cfec-4cf8-b17e-7736af956479] Started GET "/users/sign_in" for 172.18.0.1 at 2020-11-23 11:42:38 +0000
panoptes_1 | [5fe1e66d-cfec-4cf8-b17e-7736af956479] Processing by SessionsController#new as JSON
panoptes_1 | [5fe1e66d-cfec-4cf8-b17e-7736af956479] Completed 200 OK in 1ms (Views: 0.2ms | ActiveRecord: 0.0ms)
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86]
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86]
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86] Started POST "/users/sign_in" for 172.18.0.1 at 2020-11-23 11:42:38 +0000
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86] Processing by SessionsController#create as JSON
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86] Parameters: {"authenticity_token"=>"Cnf+TasWgnS/vKo3xzAUAW8ukiJ1iET3fZC52bYOWM2Dg3T5cbl1W9F3k3rwptt6bvJubJZfe/jw7caHGyEcuw==", "user"=>{"login"=>"sarp", "password"=>"[FILTERED]", "remember_me"=>true}}
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86] [primary] User Load (1.0ms) SELECT "users".* FROM "users" WHERE (LOWER("users"."login") = 'sarp' OR "users"."email" = 'sarp') ORDER BY "users"."id" ASC LIMIT 1
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86] [primary] (0.3ms) BEGIN
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86] [primary] SQL (0.7ms) UPDATE "users" SET "last_sign_in_at" = '2020-11-23 11:42:03.030141', "current_sign_in_at" = '2020-11-23 11:42:38.379540', "sign_in_count" = 48, "updated_at" = '2020-11-23 11:42:38.380703' WHERE "users"."id" = 2 [["last_sign_in_at", Mon, 23 Nov 2020 11:42:03 UTC +00:00], ["current_sign_in_at", Mon, 23 Nov 2020 11:42:38 UTC +00:00], ["sign_in_count", 48], ["updated_at", Mon, 23 Nov 2020 11:42:38 UTC +00:00], ["id", 2]]
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86] [primary] (8.4ms) COMMIT
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86] [primary] User Load (0.7ms) SELECT "users".* FROM "users" WHERE "users"."id" = 2 LIMIT 20 OFFSET 0 [["id", 2]]
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86] [primary] Flipper::Adapters::ActiveRecord::Gate Load (1.1ms) SELECT "flipper_gates".* FROM "flipper_gates" WHERE "flipper_gates"."feature_key" = 'cached_serializer' [["feature_key", "cached_serializer"]]
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86] Flipper feature(cached_serializer) enabled? true (3.4ms) [ thing=nil gate_name=boolean ]
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86] [primary] Medium Load (0.5ms) SELECT "media".* FROM "media" WHERE "media"."linked_id" = 2 AND "media"."linked_type" = 'User' AND "media"."type" = 'user_avatar' LIMIT 1 [["linked_id", 2], ["linked_type", "User"], ["type", "user_avatar"]]
panoptes_1 | [b6a58a8a-9fa9-4b06-bd73-ef2f7092fb86] Completed 200 OK in 96ms (Views: 0.6ms | ActiveRecord: 12.7ms)
panoptes_1 | [a299022c-1b7f-4d5f-a3d5-067e7fe1f142]
panoptes_1 | [a299022c-1b7f-4d5f-a3d5-067e7fe1f142]
panoptes_1 | [a299022c-1b7f-4d5f-a3d5-067e7fe1f142] Started GET "/users/sign_in" for 172.18.0.1 at 2020-11-23 11:42:38 +0000
panoptes_1 | [a299022c-1b7f-4d5f-a3d5-067e7fe1f142] Processing by SessionsController#new as JSON
panoptes_1 | [a299022c-1b7f-4d5f-a3d5-067e7fe1f142] Completed 200 OK in 2ms (Views: 0.3ms | ActiveRecord: 0.0ms)
panoptes_1 | [81a06048-5f43-42d7-b18b-6325317349b8]
panoptes_1 | [81a06048-5f43-42d7-b18b-6325317349b8]
panoptes_1 | [81a06048-5f43-42d7-b18b-6325317349b8] Started POST "/users/sign_in" for 172.18.0.1 at 2020-11-23 11:42:38 +0000
panoptes_1 | [81a06048-5f43-42d7-b18b-6325317349b8] Processing by SessionsController#create as JSON
panoptes_1 | [81a06048-5f43-42d7-b18b-6325317349b8] Parameters: {"authenticity_token"=>"sRUkYGcNcPpR8c3z+gKKBJlSszYeDFtngqsPHhCOPnEgwOLTckMDsDupQL7VyEOmP5GgyRGnDcCdnQBn79JDjA==", "user"=>{"login"=>"sarp", "password"=>"[FILTERED]", "remember_me"=>true}}
panoptes_1 | [81a06048-5f43-42d7-b18b-6325317349b8] [primary] User Load (1.0ms) SELECT "users".* FROM "users" WHERE "users"."id" = 2 ORDER BY "users"."id" ASC LIMIT 1 [["id", 2]]
panoptes_1 | [81a06048-5f43-42d7-b18b-6325317349b8] [primary] User Load (1.4ms) SELECT "users".* FROM "users" WHERE "users"."id" = 2 LIMIT 20 OFFSET 0 [["id", 2]]
panoptes_1 | [81a06048-5f43-42d7-b18b-6325317349b8] [primary] Flipper::Adapters::ActiveRecord::Gate Load (1.4ms) SELECT "flipper_gates".* FROM "flipper_gates" WHERE "flipper_gates"."feature_key" = 'cached_serializer' [["feature_key", "cached_serializer"]]
panoptes_1 | [81a06048-5f43-42d7-b18b-6325317349b8] Flipper feature(cached_serializer) enabled? true (4.0ms) [ thing=nil gate_name=boolean ]
panoptes_1 | [81a06048-5f43-42d7-b18b-6325317349b8] Completed 200 OK in 18ms (Views: 1.1ms | ActiveRecord: 3.8ms)
panoptes_1 | [95750035-decd-4bc4-bb5e-6e6365416bd4]
panoptes_1 | [95750035-decd-4bc4-bb5e-6e6365416bd4]
panoptes_1 | [95750035-decd-4bc4-bb5e-6e6365416bd4] Started POST "/oauth/token" for 172.18.0.1 at 2020-11-23 11:42:38 +0000
panoptes_1 | [95750035-decd-4bc4-bb5e-6e6365416bd4] [primary] Doorkeeper::Application Load (1.0ms) SELECT "oauth_applications".* FROM "oauth_applications" WHERE "oauth_applications"."uid" = 'ce310d45f951de68c4cc8ef46ca38cc0a008f607a2026680295757bfef99f43c' ORDER BY "oauth_applications"."id" ASC LIMIT 1 [["uid", "ce310d45f951de68c4cc8ef46ca38cc0a008f607a2026680295757bfef99f43c"]]
sidekiq_1 | `Redis#exists(key)` will return an Integer in redis-rb 4.3. `exists?` returns a boolean, you should use it instead. To opt-in to the new behavior now you can set Redis.exists_returns_integer = true. To disable this message and keep the current (boolean) behaviour of 'exists' you can set `Redis.exists_returns_integer = false`, but this option will be removed in 5.0. (/usr/local/bundle/gems/sidekiq-5.2.8/lib/sidekiq/launcher.rb:105:in `block (2 levels) in ❤')
I don't know what is causing the 422. I tried debugging with pdb a bit, and the JSONDecodeError
is due to empty response body. Below are the request and response:
Request:
{'_body_position': None,
'_cookies': <RequestsCookieJar[Cookie(version=0, name='_Panoptes_session', value='VFRmRmxtQThXUlBzZVRLYzJ3SzBqVllGdmNEOTB4ckxuMDNRRDVNbXJNeHFjQXpGMWpwTno3NkdadTl4TU95YXg3czFkOEp1cUk4NWpCeGpkcExZbzZFd2xOTmJaMzU2cmxBZjkycWMyR0JKc1FlcXhoVDc1bS9WYlVoTkd4VExaTHI5RDNnYWh3QTliaHZKMU1UTEhUS3E4NjNRbko1OXEvVDcvTkdMNWphVWY3SEp3Q1orR2lGMDg5UnB6T0QzNGp0bHJ4amZJZEVRR0NJbEJxakF4WnExTFJKUEMrRzdOZytxallKMlQvND0tLWtGMzZybFlUWEtHdGNqTG9NbGI2WEE9PQ%3D%3D--e535701e418c1c69779bac7c6c2a125b3cc6e4aa', port=None, port_specified=False, domain='localhost.local', domain_specified=False, domain_initial_dot=False, path='/', path_specified=True, secure=False, expires=None, discard=True, comment=None, comment_url=None, rest={'HttpOnly': None}, rfc2109=False), Cookie(version=0, name='remember_user_token', value='W1syXSwiJDJhJDEwJFpYdXkyVi8yN3VOLjhRcU5XTnZkMk8iLCIxNjA2MTMyMjI1LjcwMjk0NyJd--f08ff8ba72d46ee63412888c7ea039e8beb60e23', port=None, port_specified=False, domain='localhost.local', domain_specified=False, domain_initial_dot=False, path='/', path_specified=True, secure=False, expires=1607341825, discard=False, comment=None, comment_url=None, rest={'HttpOnly': None}, rfc2109=False)]>,
'body': 'grant_type=password&client_id=ce310d45f951de68c4cc8ef46ca38cc0a008f607a2026680295757bfef99f43c',
'headers': {'User-Agent': 'python-requests/2.24.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Cookie': '_Panoptes_session=VFRmRmxtQThXUlBzZVRLYzJ3SzBqVllGdmNEOTB4ckxuMDNRRDVNbXJNeHFjQXpGMWpwTno3NkdadTl4TU95YXg3czFkOEp1cUk4NWpCeGpkcExZbzZFd2xOTmJaMzU2cmxBZjkycWMyR0JKc1FlcXhoVDc1bS9WYlVoTkd4VExaTHI5RDNnYWh3QTliaHZKMU1UTEhUS3E4NjNRbko1OXEvVDcvTkdMNWphVWY3SEp3Q1orR2lGMDg5UnB6T0QzNGp0bHJ4amZJZEVRR0NJbEJxakF4WnExTFJKUEMrRzdOZytxallKMlQvND0tLWtGMzZybFlUWEtHdGNqTG9NbGI2WEE9PQ%3D%3D--e535701e418c1c69779bac7c6c2a125b3cc6e4aa; remember_user_token=W1syXSwiJDJhJDEwJFpYdXkyVi8yN3VOLjhRcU5XTnZkMk8iLCIxNjA2MTMyMjI1LjcwMjk0NyJd--f08ff8ba72d46ee63412888c7ea039e8beb60e23', 'Content-Length': '94', 'Content-Type': 'application/x-www-form-urlencoded'},
'hooks': {'response': []},
'method': 'POST',
'url': 'http://localhost:3000/oauth/token'}
Response:
{'_content': b'',
'_content_consumed': True,
'_next': None,
'connection': <requests.adapters.HTTPAdapter object at 0x7f51aca8c588>,
'cookies': <RequestsCookieJar[]>,
'elapsed': datetime.timedelta(0, 0, 27521),
'encoding': 'ISO-8859-1',
'headers': {'X-Frame-Options': 'SAMEORIGIN', 'X-XSS-Protection': '1; mode=block', 'X-Content-Type-Options': 'nosniff', 'Content-Type': 'text/html', 'P3P': 'CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"', 'Cache-Control': 'no-cache', 'X-Request-Id': '87c361a1-6a58-45b2-a598-36763439c045', 'X-Runtime': '0.021135', 'Vary': 'Origin', 'Transfer-Encoding': 'chunked'},
'history': [],
'raw': <urllib3.response.HTTPResponse object at 0x7f51acaac828>,
'reason': 'Unprocessable Entity',
'request': <PreparedRequest [POST]>,
'status_code': 422,
'url': 'http://localhost:3000/oauth/token'}
I think this is related to the underlying python client using the wrong oauth application id for oauth authentication flows.
The underlying library defaults to the production API client id and not the one that your local panoptes API has setup (different database). https://github.com/zooniverse/panoptes-python-client/blob/c87ed294ae94ea1028cb13e60680db1d8cd67bae/panoptes_client/panoptes.py#L70-L77
I believe you can set this via the PANOPTES_CLIENT_ID
env variable https://github.com/zooniverse/panoptes-python-client/blob/c87ed294ae94ea1028cb13e60680db1d8cd67bae/panoptes_client/panoptes.py#L149-L150
That said the cli configuration object would be able to take this value and initialize the underlying python client lib with these values as well, similar to how a different API endpoint is configured.
Can you try using the env variable to match the oauth client you would have received when setting up the panoptes API (point 9 in https://github.com/zooniverse/panoptes#usage)?
Perfect, that worked, thank you!
Hello, I followed the directions to run panoptes locally with docker. Is it possible to connect to it with the
panoptes-cli
client? When I am getting the following errors when I try: