Error: Querying all projects failed. Please try again.

ncvescera commented 11 months ago

Command ols list doesn't work. I am successfully logged in via ols login. This is the output of ols list -v:

⠧ Querying all projectsTraceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/olsync/olsync.py", line 348, in execute_action
    success = action()
              ^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/olsync/olsync.py", line 142, in query_projects
    for index, p in enumerate(sorted(overleaf_client.all_projects(), key=lambda x: x['lastUpdated'], reverse=True)):
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/olsync/olclient.py", line 92, in all_projects
    BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-projects'}).get('content'))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'

💥  Querying all projects
Error: Querying all projects failed. Please try again.

h-takeyeah commented 10 months ago

Hey, I came across this. I'd point out the Overleaf team introduced some changes last year, which may cause this issue. Since I'm an overleaf/overleaf-sync newbie, I'd appreciate experts' help! :heart:

The Overleaf team has been working on a migration project for the last year and completed it.

Before

meta name: ol-projects

json structure

// consists of a project array
[
{"id": "...", "name": "..."},
{"id": "...", "name": "..."},
// ...
]

After

meta name: ol-prefetchedProjectsBlob

json structure

// became a children of an object
{
"totalSize": 5,
"projects": [
  { "id": "...", "name": "..." },
  { "id": "...", "name": "..." },
 // ...
]
}

Git history

Gathered by me after digging the git history of overleaf/overleaf repository and may be incorrect.

Take a look at the commits that introduced these changes into Overleaf (CE edition).

into controller: https://github.com/overleaf/overleaf/commit/a0fabee3b4342204eb83bd68ffa195491c1faff7 on Sep 14, 2022
into view: https://github.com/overleaf/overleaf/commit/cdbf8c1831fe7468e52e8376031811d04c5e6555 on Oct 13, 2022

Two versions of the project list page seem to have been maintained during migration, and the old one that contains ol-project meta was removed at the commit https://github.com/overleaf/overleaf/commit/3e315eada1a99bdd6eee7db48b278118f69ced99 on May 26, 2023. I mentioned the removal of services/web/app/views/project/list.pug.

What can we do?

One patch will be to have OverleafClient.filter_projects() accept new structure dict and rename ol-projects ol-prefetchedProjectsBlob in each caller. https://github.com/moritzgloeckl/overleaf-sync/blob/aa62165eb9eba48f8b8bf3d93358f9feed0bf5a9/olsync/olclient.py#L38-L41 to

    @staticmethod
    def filter_projects(json_content, more_attrs=None):
        more_attrs = more_attrs or {}
+        projects = json_content.get("projects", []) # you have to check json_content is not None
-        for p in json_content:
+        for p in projects:

and https://github.com/moritzgloeckl/overleaf-sync/blob/aa62165eb9eba48f8b8bf3d93358f9feed0bf5a9/olsync/olclient.py#L91-L93 to

        json_content = json.loads(
-            BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-projects'}).get('content'))
+            BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-prefetchedProjectsBlob'}).get('content'))
        return list(OverleafClient.filter_projects(json_content))

My suggestion seems not intuitive because to call filter_project with an object does not follow the standard. In other words, python's built-in filter takes an array. So return list(OverleafClient.filter_projects(json_content).get("projects")) should be better. Anyone has an idea? Thanks.

h-takeyeah commented 10 months ago

One of the folk already fixed this issue.

https://github.com/svennniiii/overleaf-sync

farbod777 commented 10 months ago

Hi sorry, I did not understand how I can fix this error. Again I am sorry as I am a noob on git.

ncvescera commented 10 months ago

Hi sorry, I did not understand how I can fix this error. Again I am sorry as I am a noob on git.

I think you have to use this repo: https://github.com/svennniiii/overleaf-sync Just clone the project wherever you want git clone https://github.com/svennniiii/overleaf-sync and run pip install . inside the folder. This should install the script.

rochamatcomp commented 5 months ago

For PIPX users: pipx install git+https://github.com/svennniiii/overleaf-sync.git

moritzgloeckl / overleaf-sync