jamalex / notion-py

Unofficial Python API client for Notion.so
MIT License
4.3k stars 473 forks source link

Issues in client.get_collection_view function? #301

Open pfserra opened 3 years ago

pfserra commented 3 years ago

When I try to get one collection I'm getting the following error: File "C:\Users\pfser\PycharmProjects\HousesNotion\venv\lib\site-packages\notion\client.py", line 165, in get_collection_view collection = self.get_block( File "C:\Users\pfser\PycharmProjects\HousesNotion\venv\lib\site-packages\notion\client.py", line 118, in get_block block = self.get_record_data("block", block_id, force_refresh=force_refresh) File "C:\Users\pfser\PycharmProjects\HousesNotion\venv\lib\site-packages\notion\client.py", line 111, in get_record_data return self._store.get(table, id, force_refresh=force_refresh) File "C:\Users\pfser\PycharmProjects\HousesNotion\venv\lib\site-packages\notion\store.py", line 184, in get self.call_load_page_chunk(id) File "C:\Users\pfser\PycharmProjects\HousesNotion\venv\lib\site-packages\notion\store.py", line 286, in call_load_page_chunk recordmap = self._client.post("loadPageChunk", data).json()["recordMap"] File "C:\Users\pfser\PycharmProjects\HousesNotion\venv\lib\site-packages\notion\client.py", line 209, in post raise HTTPError( requests.exceptions.HTTPError: Invalid input.

Someone is getting the same error or there are any updates in the lib?

gnestor commented 3 years ago

I'm seeing the same error as of 7-10 days ago

davencyw commented 3 years ago

Same!

mixeden commented 3 years ago

Yep, +1 for this

fedeserbin commented 3 years ago

Yes. We are struggling with a few errors since 7-10 days ago.

429 Too Many Requests and Error and 400 error on get_block() call seems to have the same root problem.

Please any help would be really appreciate!!

@jamalex can you please check this?

Thanks 😃

Py0zz1 commented 3 years ago

This error is resolved by modifying 'call_load_page_chunk()' within 'store.py'.

    def call_load_page_chunk(self, page_id):
        if self._client.in_transaction():
            self._pages_to_refresh.append(page_id)
            return

        data = {
            "pageId": page_id,
            "limit": 100,
            "cursor": {"stack": [[{"table": "block", "id": page_id, "index": 0}]]},
            "chunkNumber": 0,
            "verticalColumns": False,
        }
        data = self._client.post("loadPageChunk", data).json()
        self.store_record_map(data)

POINT. Modify 'limit' to 100

pfserra commented 3 years ago

This error is resolved by modifying 'call_load_page_chunk()' within 'store.py'.

    def call_load_page_chunk(self, page_id):
        if self._client.in_transaction():
            self._pages_to_refresh.append(page_id)
            return

        data = {
            "pageId": page_id,
            "limit": 100,
            "cursor": {"stack": [[{"table": "block", "id": page_id, "index": 0}]]},
            "chunkNumber": 0,
            "verticalColumns": False,
        }
        data = self._client.post("loadPageChunk", data).json()
        self.store_record_map(data)

POINT. Modify 'limit' to 100

If I update the call_load_page_chunk I'm having the following error:

Traceback (most recent call last):
  File "C:\Users\pfser\PycharmProjects\HousesNotion\Houses_update.py", line 166, in <module>
    cv = client.get_collection_view(notion_db)
  File "C:\Users\pfser\PycharmProjects\HousesNotion\venv\lib\site-packages\notion\client.py", line 216, in get_collection_view
    collection = self.get_block(
  File "C:\Users\pfser\PycharmProjects\HousesNotion\venv\lib\site-packages\notion\client.py", line 169, in get_block
    block = self.get_record_data("block", block_id, force_refresh=force_refresh)
  File "C:\Users\pfser\PycharmProjects\HousesNotion\venv\lib\site-packages\notion\client.py", line 162, in get_record_data
    return self._store.get(table, id, force_refresh=force_refresh)
  File "C:\Users\pfser\PycharmProjects\HousesNotion\venv\lib\site-packages\notion\store.py", line 184, in get
    self.call_load_page_chunk(id)
  File "C:\Users\pfser\PycharmProjects\HousesNotion\venv\lib\site-packages\notion\store.py", line 285, in call_load_page_chunk
    self.store_record_map(data)
AttributeError: 'RecordStore' object has no attribute 'store_record_map'
jheddings commented 3 years ago

Unfortunately, simply adding a limit to the returns data call in call_load_page_chunk did not resolve the issue when looping through rows of large data sets. I've added a rate limit that is working for 1000+ rows. See pull request above.

I have also tested this with merge #294 and it is working.

This may all just be band-aids while they release the official API.

juliusambros commented 3 years ago

I had this error and figured out it only occurred when I want to load a page that is nested in a folder-like structure. Once I put it back to the top-level the error was gone... this is an unpleasant fix but feasible for now...

ThallyssonKlein commented 3 years ago

Any updates on that? I'm having this problem

alaedinebe commented 3 years ago

Currently experiencing the exact same issue. Any idea how to fix that? Do you think the official api can help? :)

yashgorana commented 3 years ago

Ah, they changed the API again. page_id needs to be provided in an object.

def call_load_page_chunk(self, page_id):

        if self._client.in_transaction():
            self._pages_to_refresh.append(page_id)
            return

        data = {
            "page": {
                "id": page_id
            },
            "limit": 100,
            "cursor": {"stack": []},
            "chunkNumber": 0,
            "verticalColumns": False,
        }

        recordmap = self._client.post("loadPageChunk", data).json()["recordMap"]

        self.store_recordmap(recordmap)