Pagination for "v3/reference/tickers" has multiple bugs

[marco-alban-atomic]

URL "v3/reference/tickers"

Result 1) Make query with params: active=true&sort=ticker&order=asc&limit=1000&exchange=XNYS&apiKey=blah

2) You'll get a cursor in response object. Second pagination call works fine: `/v3/reference/tickers/?cursor=123&apiKey=blah"

3) Third pagination call fails because next_cursor returns: "next_cursor": "https://api.polygon.io:443:443/v3/reference/tickers..." Notice the double :443

Manually parsing the url to just get the cursor and rehydrate what should be the correct url (no :443) makes it so that the same next_cursor is always returned (e.g. data is the same and cursor is ignored, you cannot advance pagination)

Expected Result 3rd pagination call should work

Additional Fix seems to "hard code" url to always be https://api.polygon.io:443 and then build the next url manually. It seems like pagination just doesn't work without adding the port. Relying on the next_url returned from the object itself does not work as it will return :443:443

Desktop (please complete the following information):

• OS: macOS
• curl

[calebcase]

Hi @marco-alban-atomic! Thanks for the bug report. I'm trying to replicate what you are seeing with duplicated port :443:443, but I'm not able to reproduce it. Here's how I tried:

➜ curl -s "https://api.polygon.io/v3/reference/tickers?active=true&sort=ticker&order=asc&limit=1000&exchange=XNYS&apiKey=$POLYGON_API_KEY" | jq -r '.next_url'
https://api.polygon.io:443/v3/reference/tickers?cursor=YWN0aXZlPXRydWUmZGF0ZT0yMDIxLTA1LTA2JmV4Y2hhbmdlPVhOWVMmbGltaXQ9MTAwMCZvcmRlcj1hc2MmcGFnZV9tYXJrZXI9RFJIJTdDNzAzNzRhNWRlNjhmMDU5Yzc0YjJmZDUxODY0YzQ3NTY3NzY0MmZlMWU4ZjFmN2U4ZDA3MjdhNzJjOWFkYmMyYSZzb3J0PXRpY2tlcg
➜ curl -s "https://api.polygon.io:443/v3/reference/tickers?cursor=YWN0aXZlPXRydWUmZGF0ZT0yMDIxLTA1LTA2JmV4Y2hhbmdlPVhOWVMmbGltaXQ9MTAwMCZvcmRlcj1hc2MmcGFnZV9tYXJrZXI9RFJIJTdDNzAzNzRhNWRlNjhmMDU5Yzc0YjJmZDUxODY0YzQ3NTY3NzY0MmZlMWU4ZjFmN2U4ZDA3MjdhNzJjOWFkYmMyYSZzb3J0PXRpY2tlcg&apiKey=$POLYGON_API_KEY" | jq -r '.next_url'
https://api.polygon.io:443/v3/reference/tickers?cursor=YWN0aXZlPXRydWUmZGF0ZT0yMDIxLTA1LTA2JmV4Y2hhbmdlPVhOWVMmbGltaXQ9MTAwMCZvcmRlcj1hc2MmcGFnZV9tYXJrZXI9TFhGUiU3Q2NhZWUyODEzM2JjZDRjMDVmNjczNjA1Y2Y3YzIyMDVhMTY2YjNjNGI1MzdiZDk3ZmE2ZWNjMmM5YmNkNzJmZDQmc29ydD10aWNrZXI
➜ curl -s "https://api.polygon.io:443/v3/reference/tickers?cursor=YWN0aXZlPXRydWUmZGF0ZT0yMDIxLTA1LTA2JmV4Y2hhbmdlPVhOWVMmbGltaXQ9MTAwMCZvcmRlcj1hc2MmcGFnZV9tYXJrZXI9TFhGUiU3Q2NhZWUyODEzM2JjZDRjMDVmNjczNjA1Y2Y3YzIyMDVhMTY2YjNjNGI1MzdiZDk3ZmE2ZWNjMmM5YmNkNzJmZDQmc29ydD10aWNrZXI&apiKey=$POLYGON_API_KEY" | jq -r '.next_url'
https://api.polygon.io:443/v3/reference/tickers?cursor=YWN0aXZlPXRydWUmZGF0ZT0yMDIxLTA1LTA2JmV4Y2hhbmdlPVhOWVMmbGltaXQ9MTAwMCZvcmRlcj1hc2MmcGFnZV9tYXJrZXI9U1BMUCU3Q2M4ZTQ2YTM3ZmExMzg5MDdiYjRhMjU4N2IzOTA4ZjRlODlmMDdiODQyOGNhZjBhNjIyM2QwZTg1ODYwNWI0NDYmc29ydD10aWNrZXI

I went through several pages without seeing it double up the port. But I also noticed you mentioned next_cursor and this API doesn't have that field so I might be looking at the wrong API. Can you try the same curl's above and put the output here?

[marco-alban-atomic]

edited, meant next_url. Definitely the same API. I was able to work around it by building the url manually with some string manipulation to remove extra port. I can still reproduce it. Hopefully this helps, will move on to other things. Feel free to close.

[lueurxax]

Hi, all! I can reproduce this bug with header 'Host'

curl -H "Host: api.polygon.io:443" 'https://api.polygon.io:443/v2/reference/news?cursor=eyJsaW1pdCI6MTAwLCJzb3J0IjoicHVibGlzaGVkX3V0YyIsIm9yZGVyIjoiZGVzY2VuZGluZyIsInRpY2tlciI6eyJlcSI6IkFBUEwifSwicHVibGlzaGVkX3V0YyI6eyJndCI6IjIwMjEtMDUtMTFUMDg6MjU6MDMuODcyWiJ9LCJzZWFyY2hfYWZ0ZXIiOlsxNjIxNDI2NTAwMDAwLG51bGxdfQ&apiKey=$POLYGON_API_KEY' | jq -r .next_url