Initial backup attempt error: Process completed with exit code 1.

bencf commented 4 years ago

Describe the bug I followed the directions for installing with GitHub Actions, and after running for 8m 6s the action ended in an error: Process completed with exit code 1. (Check failure on line 1 in .github)

To Reproduce Steps to reproduce the behavior: I just followed the directions in the readme for GitHub Actions. I created the three separate secrets.

Expected behavior The backup to complete.

Traceback Process completed with exit code 1. (Check failure on line 1 in .github)

Run roam-to-git --debug notes/ and report what you get. (I don't believe this feature is possible with the GitHub Actions method?)

Please complete the following information:

OS: MacOS
Do you use Github Action? Yes
Do you use multiple Roam databases? Yes
Does roam-to-git use to work for you? When precisely did it stopped to work? No
Does some backup runs are still working? No

Additional context Thank you for any and all help in advance! I will try it locally now, but figured the bug report was at least useful regardless.

jmsidhu commented 4 years ago

I am in the same boat.

Here is a clip from the log


2020-06-22T17:43:25.2989612Z 2020-06-22 17:43:25.298 | DEBUG    | roam_to_git.scrapping:download_rr_archive:76 - Closing browser markdown
2020-06-22T17:43:25.2990533Z 2020-06-22 17:43:25.298 | DEBUG    | roam_to_git.scrapping:download_rr_archive:76 - Closing browser json
2020-06-22T17:43:25.3237598Z 2020-06-22 17:43:25.322 | DEBUG    | roam_to_git.scrapping:download_rr_archive:78 - Closed browser json
2020-06-22T17:43:25.3431589Z 2020-06-22 17:43:25.342 | DEBUG    | roam_to_git.scrapping:download_rr_archive:78 - Closed browser markdown
2020-06-22T17:43:25.3485417Z 2020-06-22 17:43:25.343 | ERROR    | __main__:<module>:11 - An error has been caught in function '<module>', process 'MainProcess' (3043), thread 'MainThread' (140356239140672):
2020-06-22T17:43:25.3485931Z Traceback (most recent call last):
2020-06-22T17:43:25.3486209Z 
2020-06-22T17:43:25.3486755Z > File "/opt/hostedtoolcache/Python/3.8.3/x64/bin/roam-to-git", line 11, in <module>
2020-06-22T17:43:25.3488043Z     load_entry_point('roam-to-git==0.1', 'console_scripts', 'roam-to-git')()
2020-06-22T17:43:25.3488745Z     └ <function load_entry_point at 0x7fa7379505e0>
2020-06-22T17:43:25.3490491Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/__main__.py", line 76, in main
2020-06-22T17:43:25.3491206Z     scrap(markdown_zip_path, json_zip_path, config)
2020-06-22T17:43:25.3491836Z     │     │                  │              └ <roam_to_git.scrapping.Config object at 0x7fa734310850>
2020-06-22T17:43:25.3492398Z     │     │                  └ PosixPath('/tmp/tmpj437vmt5')
2020-06-22T17:43:25.3492937Z     │     └ PosixPath('/tmp/tmpimgcwkjv')
2020-06-22T17:43:25.3493453Z     └ <function scrap at 0x7fa7343175e0>
2020-06-22T17:43:25.3494150Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 252, in scrap
2020-06-22T17:43:25.3494566Z     asyncio.get_event_loop().run_until_complete(asyncio.gather(*tasks))
2020-06-22T17:43:25.3495271Z     │       │                                   │       │       └ [<coroutine object download_rr_archive at 0x7fa734700dc0>, <coroutine object download_rr_archive at 0x7fa734b9fd40>]
2020-06-22T17:43:25.3495886Z     │       │                                   │       └ <function gather at 0x7fa736304310>
2020-06-22T17:43:25.3496511Z     │       │                                   └ <module 'asyncio' from '/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/asyncio/__init__.py'>
2020-06-22T17:43:25.3497058Z     │       └ <built-in function get_event_loop>
2020-06-22T17:43:25.3498070Z     └ <module 'asyncio' from '/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/asyncio/__init__.py'>
2020-06-22T17:43:25.3498543Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
2020-06-22T17:43:25.3498913Z     return future.result()
2020-06-22T17:43:25.3499440Z            │      └ <method 'result' of '_asyncio.Future' objects>
2020-06-22T17:43:25.3500297Z            └ <_GatheringFuture finished exception=AssertionError('All roads leads to Roam, but that one is too long. Try again when Roam s...
2020-06-22T17:43:25.3501418Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 69, in download_rr_archive
2020-06-22T17:43:25.3502746Z     return await _download_rr_archive(document, output_type, output_directory, config)
2020-06-22T17:43:25.3503633Z                  │                    │         │            │                 └ <roam_to_git.scrapping.Config object at 0x7fa734310850>
2020-06-22T17:43:25.3504393Z                  │                    │         │            └ PosixPath('/tmp/tmpj437vmt5')
2020-06-22T17:43:25.3505428Z                  │                    │         └ 'json'
2020-06-22T17:43:25.3506204Z                  │                    └ <pyppeteer.page.Page object at 0x7fa733c0efd0>
2020-06-22T17:43:25.3508987Z                  └ <function _download_rr_archive at 0x7fa7343173a0>
2020-06-22T17:43:25.3509861Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 121, in _download_rr_archive
2020-06-22T17:43:25.3510248Z     assert dot_button is not None, "All roads leads to Roam, but that one is too long. Try " \
2020-06-22T17:43:25.3510703Z            └ None
2020-06-22T17:43:25.3510901Z 
2020-06-22T17:43:25.3511169Z AssertionError: All roads leads to Roam, but that one is too long. Try again when Roam servers are faster.
2020-06-22T17:43:25.3511856Z Traceback (most recent call last):
2020-06-22T17:43:25.3512495Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/bin/roam-to-git", line 11, in <module>
2020-06-22T17:43:25.3512957Z     load_entry_point('roam-to-git==0.1', 'console_scripts', 'roam-to-git')()
2020-06-22T17:43:25.3513504Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/loguru/_logger.py", line 1220, in catch_wrapper
2020-06-22T17:43:25.3513788Z     return function(*args, **kwargs)
2020-06-22T17:43:25.3514496Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/__main__.py", line 76, in main
2020-06-22T17:43:25.3514795Z     scrap(markdown_zip_path, json_zip_path, config)
2020-06-22T17:43:25.3515285Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 252, in scrap
2020-06-22T17:43:25.3515568Z     asyncio.get_event_loop().run_until_complete(asyncio.gather(*tasks))
2020-06-22T17:43:25.3515873Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
2020-06-22T17:43:25.3516122Z     return future.result()
2020-06-22T17:43:25.3516620Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 69, in download_rr_archive
2020-06-22T17:43:25.3516918Z     return await _download_rr_archive(document, output_type, output_directory, config)
2020-06-22T17:43:25.3517878Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 121, in _download_rr_archive
2020-06-22T17:43:25.3518221Z     assert dot_button is not None, "All roads leads to Roam, but that one is too long. Try " \
2020-06-22T17:43:25.3518500Z AssertionError: All roads leads to Roam, but that one is too long. Try again when Roam servers are faster.
2020-06-22T17:43:25.3845430Z ##[error]Process completed with exit code 1.```

tillleinen commented 4 years ago

Had the same issue and for me it was related to the env vars. Have now set up the secrets in a .env file instead of Github Secrets and it works.

jmsidhu commented 4 years ago

Nvrmind. I have moved .env to root dir and the action just ran and complete successfully.

Thank you @tilleinen

Thanks Till. How does the main.yml file need to be modified? What directory should the .env go in?

On Mon, Jun 22, 2020 at 2:17 PM Till Leinen notifications@github.com wrote:

Had the same issue and for me it was related to the env vars. Have now set up the secrets in a .env file instead of Github Secrets and it works.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/MatthieuBizien/roam-to-git/issues/29#issuecomment-647693295, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAYF2DJRDZ5JQA47POAP65DRX6N4TANCNFSM4OEU3R5Q .

MatthieuBizien commented 4 years ago

Thanks @tillleinen for the tip. I would like to try to debug that one though, because versioning password is not good for security. @bencf and @tillleinen, if you have the time and knowledge, could you add https://github.com/marketplace/actions/debugging-with-tmate to your CI steps, and see what are the content of the environment variables?

bencf commented 4 years ago

OK, so I wasn't sure how to get the tmate debugging working, but I found another way to at least get a little more information for logging purposes. Hope it helps!

This line seemed curious given that it's supposed to be using the GH secrets:

2020-06-25T16:12:00.5771483Z 2020-06-25 16:12:00.576 | DEBUG    | roam_to_git.__main__:main:53 - No secret found at /home/runner/work/roam-personal-notes/roam-personal-notes/.env

Here was the full output:

2020-06-25T16:11:59.2399512Z ##[debug]Evaluating: secrets.ROAMRESEARCH_USER
2020-06-25T16:11:59.2399894Z ##[debug]Evaluating Index:
2020-06-25T16:11:59.2400100Z ##[debug]..Evaluating secrets:
2020-06-25T16:11:59.2400536Z ##[debug]..=> Object
2020-06-25T16:11:59.2400839Z ##[debug]..Evaluating String:
2020-06-25T16:11:59.2401058Z ##[debug]..=> 'ROAMRESEARCH_USER'
2020-06-25T16:11:59.2402078Z ##[debug]=> '***'
2020-06-25T16:11:59.2402401Z ##[debug]Result: '***'
2020-06-25T16:11:59.2403206Z ##[debug]Evaluating: secrets.ROAMRESEARCH_PASSWORD
2020-06-25T16:11:59.2403390Z ##[debug]Evaluating Index:
2020-06-25T16:11:59.2403563Z ##[debug]..Evaluating secrets:
2020-06-25T16:11:59.2403744Z ##[debug]..=> Object
2020-06-25T16:11:59.2403924Z ##[debug]..Evaluating String:
2020-06-25T16:11:59.2404104Z ##[debug]..=> 'ROAMRESEARCH_PASSWORD'
2020-06-25T16:11:59.2404416Z ##[debug]=> '***'
2020-06-25T16:11:59.2404611Z ##[debug]Result: '***'
2020-06-25T16:11:59.2404998Z ##[debug]Evaluating: secrets.ROAMRESEARCH_DATABASE
2020-06-25T16:11:59.2405252Z ##[debug]Evaluating Index:
2020-06-25T16:11:59.2405431Z ##[debug]..Evaluating secrets:
2020-06-25T16:11:59.2405623Z ##[debug]..=> Object
2020-06-25T16:11:59.2405801Z ##[debug]..Evaluating String:
2020-06-25T16:11:59.2405979Z ##[debug]..=> 'ROAMRESEARCH_DATABASE'
2020-06-25T16:11:59.2406191Z ##[debug]=> '***'
2020-06-25T16:11:59.2406379Z ##[debug]Result: '***'
2020-06-25T16:11:59.2407127Z ##[debug]Evaluating condition for step: 'Run backup'
2020-06-25T16:11:59.2408480Z ##[debug]Evaluating: success()
2020-06-25T16:11:59.2408785Z ##[debug]Evaluating success:
2020-06-25T16:11:59.2409246Z ##[debug]=> true
2020-06-25T16:11:59.2409590Z ##[debug]Result: true
2020-06-25T16:11:59.2410527Z ##[debug]Starting: Run backup
2020-06-25T16:11:59.2415440Z ##[debug]Loading inputs
2020-06-25T16:11:59.2416095Z ##[debug]Loading env
2020-06-25T16:11:59.2420628Z ##[group]Run roam-to-git --skip-git .
2020-06-25T16:11:59.2420846Z [36;1mroam-to-git --skip-git .[0m
2020-06-25T16:11:59.2462528Z shell: /bin/bash -e {0}
2020-06-25T16:11:59.2462766Z env:
2020-06-25T16:11:59.2463077Z   pythonLocation: /opt/hostedtoolcache/Python/3.8.3/x64
2020-06-25T16:11:59.2463422Z   ROAMRESEARCH_USER: ***
2020-06-25T16:11:59.2463651Z   ROAMRESEARCH_PASSWORD: ***
2020-06-25T16:11:59.2463890Z   ROAMRESEARCH_DATABASE: ***
2020-06-25T16:11:59.2464088Z ##[endgroup]
2020-06-25T16:11:59.2502373Z ##[debug]/bin/bash -e /home/runner/work/_temp/fee2c21a-ca01-4067-8a2a-b2210d8f21de.sh
2020-06-25T16:12:00.5771483Z 2020-06-25 16:12:00.576 | DEBUG    | roam_to_git.__main__:main:53 - No secret found at /home/runner/work/roam-personal-notes/roam-personal-notes/.env
2020-06-25T16:12:00.5791158Z 2020-06-25 16:12:00.577 | DEBUG    | roam_to_git.scrapping:download_rr_archive:55 - Creating browser
2020-06-25T16:12:00.5893484Z [W:pyppeteer.chromium_downloader] start chromium download.
2020-06-25T16:12:00.5894172Z Download may take a few minutes.
2020-06-25T16:12:00.7585854Z 
2020-06-25T16:12:00.8588824Z   0%|          | 0/108773488 [00:00<?, ?it/s]
2020-06-25T16:12:00.9588538Z  13%|█▎        | 14479360/108773488 [00:00<00:00, 144782346.87it/s]
2020-06-25T16:12:01.0589722Z  32%|███▏      | 35051520/108773488 [00:00<00:00, 158894557.03it/s]
2020-06-25T16:12:01.4199609Z  55%|█████▍    | 59504640/108773488 [00:00<00:00, 177546632.12it/s]
2020-06-25T16:12:01.5516028Z  69%|██████▉   | 75499520/108773488 [00:00<00:00, 93350587.82it/s] 
2020-06-25T16:12:01.5780782Z  93%|█████████▎| 100669440/108773488 [00:00<00:00, 110271216.24it/s]
2020-06-25T16:12:01.5782045Z 100%|██████████| 108773488/108773488 [00:00<00:00, 132794043.75it/s]
2020-06-25T16:12:01.5782589Z [W:pyppeteer.chromium_downloader] 
2020-06-25T16:12:01.5783011Z chromium download done.
2020-06-25T16:12:03.5959352Z [W:pyppeteer.chromium_downloader] chromium extracted to: /home/runner/.local/share/pyppeteer/local-chromium/588429
2020-06-25T16:12:04.0049773Z 2020-06-25 16:12:04.002 | DEBUG    | roam_to_git.scrapping:download_rr_archive:55 - Creating browser
2020-06-25T16:12:04.5364150Z 2020-06-25 16:12:04.535 | DEBUG    | roam_to_git.scrapping:_download_rr_archive:92 - Configure downloads to /tmp/tmp52zcg_8u
2020-06-25T16:12:04.5478870Z 2020-06-25 16:12:04.547 | DEBUG    | roam_to_git.scrapping:_download_rr_archive:92 - Configure downloads to /tmp/tmp9_j4lemu
2020-06-25T16:12:04.5705678Z 2020-06-25 16:12:04.570 | DEBUG    | roam_to_git.scrapping:signin:183 - Opening signin page
2020-06-25T16:12:04.5830455Z 2020-06-25 16:12:04.582 | DEBUG    | roam_to_git.scrapping:signin:183 - Opening signin page
2020-06-25T16:12:09.9814466Z 2020-06-25 16:12:09.980 | DEBUG    | roam_to_git.scrapping:signin:187 - Fill email '***'
2020-06-25T16:12:10.0059992Z 2020-06-25 16:12:10.005 | DEBUG    | roam_to_git.scrapping:signin:187 - Fill email '***'
2020-06-25T16:12:10.9707330Z 2020-06-25 16:12:10.970 | DEBUG    | roam_to_git.scrapping:signin:192 - Fill password
2020-06-25T16:12:10.9942766Z 2020-06-25 16:12:10.993 | DEBUG    | roam_to_git.scrapping:signin:192 - Fill password
2020-06-25T16:12:11.9102657Z 2020-06-25 16:12:11.909 | DEBUG    | roam_to_git.scrapping:signin:197 - Click on sign-in
2020-06-25T16:12:11.9439231Z 2020-06-25 16:12:11.943 | DEBUG    | roam_to_git.scrapping:signin:197 - Click on sign-in
2020-06-25T16:12:14.1587486Z 2020-06-25 16:12:14.158 | DEBUG    | roam_to_git.scrapping:go_to_database:207 - Load database from url 'https://roamresearch.com/#/app/***'
2020-06-25T16:12:14.1813095Z 2020-06-25 16:12:14.180 | DEBUG    | roam_to_git.scrapping:go_to_database:207 - Load database from url 'https://roamresearch.com/#/app/***'
2020-06-25T16:12:14.2223446Z 2020-06-25 16:12:14.221 | DEBUG    | roam_to_git.scrapping:_download_rr_archive:102 - Wait for interface to load
2020-06-25T16:12:14.2434514Z 2020-06-25 16:12:14.243 | DEBUG    | roam_to_git.scrapping:_download_rr_archive:102 - Wait for interface to load
2020-06-25T16:18:59.1517033Z 2020-06-25 16:18:59.151 | DEBUG    | roam_to_git.scrapping:download_rr_archive:76 - Closing browser json
2020-06-25T16:18:59.1518356Z 2020-06-25 16:18:59.151 | DEBUG    | roam_to_git.scrapping:download_rr_archive:76 - Closing browser markdown
2020-06-25T16:18:59.1977166Z 2020-06-25 16:18:59.197 | DEBUG    | roam_to_git.scrapping:download_rr_archive:78 - Closed browser json
2020-06-25T16:18:59.1984926Z 2020-06-25 16:18:59.198 | DEBUG    | roam_to_git.scrapping:download_rr_archive:78 - Closed browser markdown
2020-06-25T16:18:59.2040091Z 2020-06-25 16:18:59.198 | ERROR    | __main__:<module>:11 - An error has been caught in function '<module>', process 'MainProcess' (3207), thread 'MainThread' (140536825149248):
2020-06-25T16:18:59.2040762Z Traceback (most recent call last):
2020-06-25T16:18:59.2041129Z 
2020-06-25T16:18:59.2041791Z > File "/opt/hostedtoolcache/Python/3.8.3/x64/bin/roam-to-git", line 11, in <module>
2020-06-25T16:18:59.2042487Z     load_entry_point('roam-to-git==0.1', 'console_scripts', 'roam-to-git')()
2020-06-25T16:18:59.2043294Z     └ <function load_entry_point at 0x7fd14358d5e0>
2020-06-25T16:18:59.2045007Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/__main__.py", line 76, in main
2020-06-25T16:18:59.2045814Z     scrap(markdown_zip_path, json_zip_path, config)
2020-06-25T16:18:59.2046461Z     │     │                  │              └ <roam_to_git.scrapping.Config object at 0x7fd13ff4d8e0>
2020-06-25T16:18:59.2047089Z     │     │                  └ PosixPath('/tmp/tmp9_j4lemu')
2020-06-25T16:18:59.2047621Z     │     └ PosixPath('/tmp/tmp52zcg_8u')
2020-06-25T16:18:59.2048170Z     └ <function scrap at 0x7fd13ff535e0>
2020-06-25T16:18:59.2048900Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 252, in scrap
2020-06-25T16:18:59.2049316Z     asyncio.get_event_loop().run_until_complete(asyncio.gather(*tasks))
2020-06-25T16:18:59.2049996Z     │       │                                   │       │       └ [<coroutine object download_rr_archive at 0x7fd14033ddc0>, <coroutine object download_rr_archive at 0x7fd1405d9d40>]
2020-06-25T16:18:59.2050796Z     │       │                                   │       └ <function gather at 0x7fd141f48310>
2020-06-25T16:18:59.2051472Z     │       │                                   └ <module 'asyncio' from '/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/asyncio/__init__.py'>
2020-06-25T16:18:59.2052017Z     │       └ <built-in function get_event_loop>
2020-06-25T16:18:59.2052644Z     └ <module 'asyncio' from '/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/asyncio/__init__.py'>
2020-06-25T16:18:59.2053119Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
2020-06-25T16:18:59.2053564Z     return future.result()
2020-06-25T16:18:59.2054115Z            │      └ <method 'result' of '_asyncio.Future' objects>
2020-06-25T16:18:59.2054751Z            └ <_GatheringFuture finished exception=AssertionError('All roads leads to Roam, but that one is too long. Try again when Roam s...
2020-06-25T16:18:59.2055450Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 69, in download_rr_archive
2020-06-25T16:18:59.2055957Z     return await _download_rr_archive(document, output_type, output_directory, config)
2020-06-25T16:18:59.2056593Z                  │                    │         │            │                 └ <roam_to_git.scrapping.Config object at 0x7fd13ff4d8e0>
2020-06-25T16:18:59.2057175Z                  │                    │         │            └ PosixPath('/tmp/tmp9_j4lemu')
2020-06-25T16:18:59.2057733Z                  │                    │         └ 'json'
2020-06-25T16:18:59.2058310Z                  │                    └ <pyppeteer.page.Page object at 0x7fd13f84bee0>
2020-06-25T16:18:59.2058919Z                  └ <function _download_rr_archive at 0x7fd13ff533a0>
2020-06-25T16:18:59.2059670Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 121, in _download_rr_archive
2020-06-25T16:18:59.2060793Z     assert dot_button is not None, "All roads leads to Roam, but that one is too long. Try " \
2020-06-25T16:18:59.2061342Z            └ None
2020-06-25T16:18:59.2061611Z 
2020-06-25T16:18:59.2061917Z AssertionError: All roads leads to Roam, but that one is too long. Try again when Roam servers are faster.
2020-06-25T16:18:59.2062269Z Traceback (most recent call last):
2020-06-25T16:18:59.2062855Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/bin/roam-to-git", line 11, in <module>
2020-06-25T16:18:59.2063366Z     load_entry_point('roam-to-git==0.1', 'console_scripts', 'roam-to-git')()
2020-06-25T16:18:59.2063983Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/loguru/_logger.py", line 1220, in catch_wrapper
2020-06-25T16:18:59.2064336Z     return function(*args, **kwargs)
2020-06-25T16:18:59.2064937Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/__main__.py", line 76, in main
2020-06-25T16:18:59.2065272Z     scrap(markdown_zip_path, json_zip_path, config)
2020-06-25T16:18:59.2065841Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 252, in scrap
2020-06-25T16:18:59.2066282Z     asyncio.get_event_loop().run_until_complete(asyncio.gather(*tasks))
2020-06-25T16:18:59.2066737Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
2020-06-25T16:18:59.2176061Z     return future.result()
2020-06-25T16:18:59.2230443Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 69, in download_rr_archive
2020-06-25T16:18:59.2230701Z     return await _download_rr_archive(document, output_type, output_directory, config)
2020-06-25T16:18:59.2231193Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 121, in _download_rr_archive
2020-06-25T16:18:59.2231405Z     assert dot_button is not None, "All roads leads to Roam, but that one is too long. Try " \
2020-06-25T16:18:59.2231814Z AssertionError: All roads leads to Roam, but that one is too long. Try again when Roam servers are faster.
2020-06-25T16:18:59.2452968Z ##[error]Process completed with exit code 1.

tillleinen commented 4 years ago

@MatthieuBizien Thanks for the nudge, have now made Github Secrets work. In the end there was no issue with your script, but I didn't set up the ENV vars correctly. Maybe consider making your docs more explicit or check and warn if ENV vars cannot be read correctly?

jmsidhu commented 4 years ago

Do you have any specific tips @tillleinen to getting the env. variables to work?

tillleinen commented 4 years ago

@jmsidhu At first I didn't realize how Github Secrets work and put the entire content of the .env file into one secret. When the .env way worked, I understood that each secret is one ENV var and needs to have the ENV var as the title (e.g. ROAMRESEARCH_DATABASE) and only the value as it's content.

jmsidhu commented 4 years ago

Thanks. Funny. That is what I tried before without success. Going to try again and double check for title spelling and var content.

On Wed, Jul 1, 2020 at 5:07 PM Till Leinen notifications@github.com wrote:

@jmsidhu https://github.com/jmsidhu At first I didn't realize how Github Secrets work and put the entire content of the .env file into one secret. When the .env way worked, I understood that each secret is one ENV var and needs to have the ENV var as the title (e.g. ROAMRESEARCH_DATABASE) and only the value as it's content.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/MatthieuBizien/roam-to-git/issues/29#issuecomment-652647191, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAYF2DJIXHLDECFMYIHTYSLRZOQSJANCNFSM4OEU3R5Q .

bencf commented 4 years ago

Hi @MatthieuBizien - was this fixed? I believe I was using GitHub Secrets the correct way and still got the error. Is there anything else I can do to get it to work?

Thanks!

MatthieuBizien commented 4 years ago

I misunderstood your answer, sorry! I will keep it open, if you have the time to investigate it would be a great help :)

dpetrouk commented 4 years ago

Problem was fixed after I added ROAMRESEARCH_DATABASE environment variable in the Secrets. Earlier roam-to-git worked without it. Now it seems that database (or graph as it called in roam) should be specified.

dpetrouk commented 4 years ago

This problem appeared around june 9th right before gates of Roam were opened and pricing was shipped (Roam change log on June 10th, 2020).

MatthieuBizien commented 4 years ago

I've added the ROAMRESEARCH_DATABASE as a mandatory environment variable. @bencf was it the fix for you?

bencf commented 4 years ago

Got it working!

I had included the database as an environment variable, so that wasn't the issue.

I hadn't tried in a few days, so maybe there's so just something different now.

But, digging into it more, I think it had to do with the fact that I ended up at some point switching to the Google account login, so my normal password was disabled. So I think it was a login error. (I'm not sure there's any way for your script to work with just a Google login, right?)

I was able to reset the password on Roam for my email address and use that new password and then the backup worked.

Thank you for your willingness to help!

LoveOctocat commented 4 years ago

I also encountered this problem, but after the first configuration was successful, I successfully synchronized three times, after which all synchronization failed.

LoveOctocat commented 4 years ago

Keep making mistakes, three days ago suddenly succeeded several times. But success has been a failure until now.

everruler12 commented 4 years ago

Having same problem with failed backup action. Permission denied: '/app' Not using .env since I put in Secrets section.

.
.
.
2020-08-09 22:32:05.139 | DEBUG    | roam_to_git.scrapping:scrap:253 - Scrapping finished
2020-08-09 22:32:05.178 | DEBUG    | roam_to_git.fs:save_markdowns:50 - Saving markdown to /home/runner/work/roam-backup/roam-backup/markdown
2020-08-09 22:32:05.182 | ERROR    | __main__:<module>:11 - An error has been caught in function '<module>', process 'MainProcess' (3032), thread 'MainThread' (140563180279616):
Traceback (most recent call last):
> File "/opt/hostedtoolcache/Python/3.8.5/x64/bin/roam-to-git", line 11, in <module>
    load_entry_point('roam-to-git==0.1', 'console_scripts', 'roam-to-git')()
    └ <function load_entry_point at 0x7fd766383f70>
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/roam_to_git/__main__.py", line 82, in main
    save_markdowns(git_path / "markdown", raws)
    │              │                      └ {'March 3rd, 2020.md': '- Morning Routine::\n    - Got up:: Woke up 6:00 because dad did, stayed awake, got up 6:50\n    - Dr...
    │              └ PosixPath('/home/runner/work/roam-backup/roam-backup')
    └ <function save_markdowns at 0x7fd764c4a040>
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/roam_to_git/fs.py", line 54, in save_markdowns
    dest.parent.mkdir(parents=True, exist_ok=True)  # Needed if a new directory is used
    │    └ <property object at 0x7fd76601e090>
    └ PosixPath('/app/metaroam.md')
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/pathlib.py", line 1284, in mkdir
    self._accessor.mkdir(self, mode)
    │    │               │     └ 511
    │    │               └ PosixPath('/app')
    │    └ <member '_accessor' of 'Path' objects>
    └ PosixPath('/app')

PermissionError: [Errno 13] Permission denied: '/app'
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.5/x64/bin/roam-to-git", line 11, in <module>
    load_entry_point('roam-to-git==0.1', 'console_scripts', 'roam-to-git')()
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/loguru/_logger.py", line 1149, in catch_wrapper
    return function(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/roam_to_git/__main__.py", line 82, in main
    save_markdowns(git_path / "markdown", raws)
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/roam_to_git/fs.py", line 54, in save_markdowns
    dest.parent.mkdir(parents=True, exist_ok=True)  # Needed if a new directory is used
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/pathlib.py", line 1284, in mkdir
    self._accessor.mkdir(self, mode)
PermissionError: [Errno 13] Permission denied: '/app'
##[error]Process completed with exit code 1.

As you can see, it was able to read "March 3rd, 2020.md" so the login must have worked. This is my first time running it. Do I manually have to create 'formatted', 'json', 'markdown' directories?

Edit: I figured out, I had the url roamresearch.com/#/app/metaroam in my database, and the hashtag caused a page to be created called /app/metaroam and this is what was causing the problem. After I deleted that, and used markdown to make it a link, the problem was solved, and the backup ran fine. So, it seems there's a python issue when one of Roam's pages starts with /

quickfold commented 4 years ago

I'm having the same error but the existing comments haven't helped me solve it. I'm on Win 10. The end of my action log is:

File "/opt/hostedtoolcache/Python/3.8.5/x64/bin/roam-to-git", line 11, in <module>
load_entry_point('roam-to-git==0.1', 'console_scripts', 'roam-to-git')()
File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/loguru/_logger.py", line 1149, in catch_wrapper
return function(*args, **kwargs)
File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/roam_to_git/__main__.py", line 76, in main
  scrap(markdown_zip_path, json_zip_path, config)
File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 252, in scrap
  asyncio.get_event_loop().run_until_complete(asyncio.gather(*tasks))
File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
  return future.result()
File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 69, in download_rr_archive
  return await _download_rr_archive(document, output_type, output_directory, config)
File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 121, in _download_rr_archive
  assert dot_button is not None, "All roads leads to Roam, but that one is too long. Try " \
 | roam_to_git.scrapping:_kill_child_process:215 - Terminate child process [psutil.Process(pid=2886, name='chrome', status='sleeping', started='12:18:06'), psutil.Process(pid=2892, name='chrome', status='sleeping', started='12:18:07'), psutil.Process(pid=2912, name='chrome', status='sleeping', started='12:18:07'), psutil.Process(pid=2923, name='chrome', status='sleeping', started='12:18:07'), psutil.Process(pid=2894, name='chrome', status='sleeping', started='12:18:07'), psutil.Process(pid=2911, name='chrome', status='sleeping', started='12:18:07')]
##[error]Process completed with exit code 1.

Any help appreciated.

MatthieuBizien commented 4 years ago

@quickfold could you create a separate issue, and add the full log? Loguru should add more info.

Closing this issue, as @bencf problem was solved.

MatthieuBizien / roam-to-git

Initial backup attempt error: Process completed with exit code 1. #29