Setting up the v3 endpoint (503 error)

[afunnydev]

Hello! I've been reading most of the issues because I couldn't connect my repository with the v2 endpoint. Figured out it was maybe already overloaded or didn't accept new user's request anymore.

So I found out the about the new Staticman app on Github, connected it to my repo, and set everything for a post request to "https://dev.staticman.net/v3/entry/github/felixdb/inbeat-website/master/comments", but keep getting a 503 from Heroku. I also tried "https://dev.staticman.net/v3/entry/github/felixdb/inbeat-website/master" and tried to post vit Postman.

You can see my repo here and my settings here. Anything I'm doing wrong? You can also test here, the form is hidden with CSS but you should find it easily with inspect tool and unhide it.

Thanks for helping out, this project is awesome and I've been looking forward to try it :)

[idgman]

I'm having the same issue. I noticed that another site who has a currently working Staticman configuration is using the endpoint https://api.staticman.net/v3/... rather than https://dev.staticman.net/v3/... However, I tried changing my own endpoint and it didn't work. We seem to have similar configurations, so the problem is probably somewhere else. When I check the JSON response, all I see is "success: false." I'll keep looking... Any ideas, @eduardoboucas?

[deadlydog]

Yeah, I'm just adding staticman into my jekyll blog and am having the same experience. Wasn't able to get it to work with v2, so trying v3 and just getting back the 500 success: false error message. I've tried both https://dev.staticman.net/v3/entry/github/deadlydog/deadlydog.github.io/master/comments and https://api.staticman.net/v3/entry/github/deadlydog/deadlydog.github.io/master/comments but they both give the same error.

This is the post body that gets sent:

fields%5Bmessage%5D=Testing+comments&fields%5Bname%5D=Dan&fields%5Bemail%5D=deadlydog%40hotmail.com&fields%5Burl%5D=&options%5Bslug%5D=powershell-log-levels-included-in-tfs-2017-and-vsts-build-and-release-logs&fields%5Bhidden%5D=

I do have the Staticman app added to my repo:

It would be super helpful if the json response returned another Message field that told us what is failing on the service side.

[gayanW]

When using api.staticman.net, sometimes I'd get the error:

error: text: "Too many requests in this time frame." nextValidRequestDate: "2019-06-19T09:20:00.638Z"

I switched to dev.staticman.net and now everything seem to work fine. Also make sure that the correct form prams are being sent, because the error responses being sent back from staticman, sometimes are not accurate.

[boratanrikulu]

I have same issue. I've tired both of https://api.staticman.net/v3 and https://dev.staticman.net/v3. And still, I got 500 error.

[sachin10101998]

@gayanW , the static comments on your site (for example this blog post (https://www.linuxdeveloper.space/install-windows-after-linux) are not working anymore.

[VincentTam]

@gayanW Switching to the official development instance is contrary to what the project owner suggests in https://github.com/eduardoboucas/staticman/issues/278#issuecomment-475170669. Anyways, I suggest using alternative instance (i.e. anything but official) to avoid unexpectation service disruption due to unnotified updates in the official instance.

Here's a hosting guide on Data Science Blog. You may also refer to some successful examples running Staticman v3 like

1. zcrc.me: a dev blog on GitLab
2. my demo pages on Framagit
3. my blog using GitLab Pages

P.S. Having recommended an alternative instance, I have to mention that I maintain @staticmanlab, a public GitLab instance which you're free to use. In this case, you won't need to self-host an API instance.

[sachin10101998]

@VincentTam You mentioned that @staticmanlab is a Gitlab instance, will it cause troubles if used with a Github repo?

[VincentTam]

@VincentTam You mentioned that @staticmanlab is a Gitlab instance, will it cause troubles if used with a Github repo?

It's the contrarary: it provides freedom for users, espeicall those suffering from the official instances' API limits, to choose between GitHub & GitLab. This provides convenience for those spending hours on "invitation not found" and/or "too many requests".

For some GitHub Pages served by @staticmanlab, you may click on the bot's user page and consult its activities. You may also see the homologous GitLab user profile. Some of my minimal working demo on GitHub:

1. https://git.io/bjsm18
2. https://git.io/smdemo

P.S. Due to a Framagit user's unsuccessful trial to add Staticman support, I've created another instance served by @staticmanlab1.

[sachin10101998]

@VincentTam You have made lots of changes in your staticman instance compared to the original one. Should I clone your instance and try to host it on Heroku and see if it works?

[VincentTam]

@sachin10101998 From the user profile of @staticmanlab, you can see that my instance works. My deploy branch holds the code for Heroku deployment, so you might try. The reason for such a huge difference is PR 231.

[deadlydog]

Hey @VincentTam I forked and deployed your deploy branch to Heroku by following these instructions, but when trying to post messages to it I just get a 500 response code with a json body of "success": false. Is that branch broken now? Do you know if there's some way I can have it return more error info?

The Heroku logs only show this:

2019-08-02T06:54:01.792329+00:00 heroku[router]: at=info method=POST path="/v3/entry/github/deadlydog/deadlydog.github.io/master/comments?fields%5Bmessage%5D=Testing+comments&fields%5Bname%5D=Dan&fields%5Bemail%5D=deadlydog%40hotmail.com&fields%5Burl%5D=&options%5Bslug%5D=powershell-log-levels-included-in-tfs-2017-and-vsts-build-and-release-logs&fields%5Bhidden%5D=" host=dansblog-staticman.herokuapp.com request_id=8b3e4f93-3a6c-46b0-a477-2ad6abb99aa2 fwd="71.17.152.95" dyno=web.1 connect=1ms service=329ms status=500 bytes=353 protocol=https

And this is the response I get in PostMan.

Should I maybe try using a different branch?

I've been trying to get staticman to work for 2 months now, and it's very frustrating. Jumping from GitHub issue to issue trying to piece together various bits of info is very time consuming. I feel it would be worth updating the official documentation with instructions about the new recommended practice (deploying your own instance instead of using the shared public one), and which branches are stable and should be used.

I appreciate any insight you can give into this issue. Thanks!

[VincentTam]

Hey @VincentTam I forked and deployed your deploy branch to Heroku by following these instructions, but when trying to post messages to it I just get a 500 response code with a json body of "success": false.

The linked instructions are written by @matdoering. Perhaps he can also provide you insights.

I can spot out two errors in your previous message: repo config error and testing error.

Have you read my article about Staticman errors? I've listed different kind of possible errors along with their causes. The 500 error code with simply "success: false" is possibly due to reCAPTCHA error.

Is that branch broken now?

No, it works fine, as you can see from the activities of @staticmanlab.

Do you know if there's some way I can have it return more error info?

As a non-developer for the moment, the only way I know is the most primitive one: console.log(yourvar). You may try running it locally and inject this into wherever you want to extract info.

It would be more helpful to provide more details of your instance and the link of your target repo https://github.com/deadlydog/deadlydog.github.io, so that we can look into it and try reproducing the error, like some important API parameters (githubBaseUrl, gitlabBaseUrl), and the user nane your GitHub bot. (I'm not using the GitHub Apps version since it keeps spamming GitHub repos, as reported in #298, so it doesn't really solve the API limit problem. The project owner believes that Akismet has been disabled on the public instance, and the proposed PR #195 is left unmerged. Other reasons for keeping the GitHub bot are evangelical ones detailed in https://github.com/Pyromaths/pyromaths/issues/26).

Let's take a look at your repo config file: https://github.com/deadlydog/deadlydog.github.io/blob/1b2bf011716c045affe8b0e697e45fb4a44b233e/staticman.yml#L97-L104

  # reCaptcha
  # Register your domain at https://www.google.com/recaptcha/ and choose reCAPTCHA V2
  reCaptcha:
    enabled: true
    siteKey: "6LexzqgUAAAAACjnxAhDpsymtZKy3ymJFRbVNWKU"
    # Encrypt reCaptcha secret key using Staticman /encrypt endpoint
    # For more information, https://staticman.net/docs/encryption
    secret: "6LexzqgUAAAAAJjkczlwT888oWg48CeU8x3MrNlc"

You've enabled reCAPTCHA in your target repo, so the API server is expecting reCAPTCHA parameter from the POST request.

It's clear from your screenshot that this 500 error is due to the lack of reCAPTCHA parameters.

:warning: Encrypt your secret. Otherwise, it won't function properly.

https://github.com/eduardoboucas/staticman/blob/2be29f73fa0688e02770b9f118288b6e0e617b01/controllers/process.js#L31-L36

Encrypted secret should look like the following.

  reCaptcha:
    enabled: false
    # siteKey: "6Lcv8G8UAAAAAEqV1Y-XEPum00C_DxhD6O--qkFo"
    # ENCRYPT reCaptcha secret key using Staticman /encrypt endpoint
    # i.e. https://{STATICMAN API INSTANCE}/v3/encrypt/{your-site-secret}
    # For more information, visit https://staticman.net/docs/encryption
    # secret: "p5uHlH9hCqpMJaGKXdt5MEWFo7K6fX8hoYUwR3aIafOI6rtItLauaDCkGOucysJtrVZy+sHffioGzMsOU64JFDSyPQgrXujegcOHFRXHhD4fOUuBXSvV+OZ8JhSPTGWaRcQcoiGX4pT5hlebLddOl59b6sn6kU1ODQcEbhP83xVLZlaTWOrNrF5Wvy3TMXpH5gyl1tZEORxADAShMYyUbNR7XZYLEg1DfgIBHfIg3cKwdFt7KVLejFGKIiBYRAZDE2JuHItNmzJ2x9JgSK3E+XnShV5tuWpncnyFonJVHGEky/zRfUVLHobDMcJ/u9nlZqE8u47W+833F1WaIYuwNw=="

:warning: Also, please respond to my issue https://github.com/deadlydog/deadlydog.github.io/pull/6 for the sake of your privacy. You've exposed your reCAPTCHA site secret there. Please consider changing key.

The Heroku logs only show this:

2019-08-02T06:54:01.792329+00:00 heroku[router]: at=info method=POST path="/v3/entry/github/deadlydog/deadlydog.github.io/master/comments?fields%5Bmessage%5D=Testing+comments&fields%5Bname%5D=Dan&fields%5Bemail%5D=deadlydog%40hotmail.com&fields%5Burl%5D=&options%5Bslug%5D=powershell-log-levels-included-in-tfs-2017-and-vsts-build-and-release-logs&fields%5Bhidden%5D=" host=dansblog-staticman.herokuapp.com request_id=8b3e4f93-3a6c-46b0-a477-2ad6abb99aa2 fwd="71.17.152.95" dyno=web.1 connect=1ms service=329ms status=500 bytes=353 protocol=https

And this is the response I get in PostMan.

Your test procedures might not be correct, from my experience in https://github.com/eduardoboucas/staticman/pull/219#issuecomment-419678597.

@ ntsim Thank you so much for your patience and help! :grinning: It finally worked on my forked project. If I start the project from scratch, do I need to manually create comments folder?

During my testing last year, the form key and values didn't show up in the POST URL.

Should I maybe try using a different branch?

Yes and no.

• Yes: try my PR #301, which fixed #90. :smile:
• No: in case you get tired of testing, and you want to stick with your working version.

I've been trying to get staticman to work for 2 months now, and it's very frustrating. Jumping from GitHub issue to issue trying to piece together various bits of info is very time consuming.

However, learning from Stack Overflow involves jumping from questions to questions, and research requires reading from different pieces of info. Links to issues are a means for reference and supporting one's ideas. The past experience described in the issues helps other future users. If you feel that jumping from issues to issues are tiring, then just skip reading and learn from your own errors, until you're so tired with errors that you've the patience to read issues :smiling_imp:.

I feel it would be worth updating the official documentation with instructions about the new recommended practice (deploying your own instance instead of using the shared public one), and which branches are stable and should be used.

Documentation requests like #291, #293 are inactive. Recent attempts to update the README in #232 and #256 don't receive response from the project owner. Even #298 doesn't get a direct response (despite @staticmanapp responded with its disappearance). Given the status of these issue, I would better save my effort in testing and writing guides and documentation for other repo. Here's a non-exhausive list of references that provide info about setting up Staticman:

1. :meat_on_bone: Barebone example by Nicolas Tsim, the author of #219 and #231: https://gitlab.com/nstim/test-staticman (requires internal access). The info at the top of #219 is extremely useful. It's the reference for Staticman v3 API endpoints.
2. :notebook: Beautiful Jekyll's README.
3. :book: Minimal Mistake's docs page
4. :1234: My upvoted answer on Math.Meta.SE
5. Docs on my Minimo + Framagit + Staticman demo site
6. Hugo Swift Theme's README
7. README in my improved version of Hugo Theme Huginn
8. And last, but not least, Staticman's outdated docs page

As the name "Lab" in "Staticman Lab" suggest, it's supposed to be a laboratory for testing. That's why I feel no guilt using a Heroku free dyno. That's no obligations about testing, in which we found out the errors, discuss and correct them so that things work.

I appreciate any insight you can give into this issue. Thanks!

Give me :+1: or :heart: if you appreciate it. :loud_sound: Sharing Staticman guides and helping future users in return are even better.

[deadlydog]

Thanks so much for the detailed reply @VincentTam and the Issues/PRs on my personal blog :) I had not seen your blog with staticman errors, and it looks like it does have something to do with the Recaptcha settings. I still have not been able to completely figure it out; if I disable Recaptcha completely then comments work, but when enabled (using an encrypted site key and Recaptcha v2, or v3) it still fails. I don't understand why when I use the exact same request from PostMan to my personal staticman instance that it succeeds when I have my blog settings set to not use Recaptcha, but fails when it is set to use Recaptcha. That is a setting set in my blog GitHub repo (jekyll site), not my heroku staticman instance, so I find it weird that a different client's setting affects the server response. That is, my PostMan request is the exact same all the time, and is making a request to my staticman instance, so why does my blog GitHub setting affect the response returned by the staticman instance? Or does my staticman instance somehow check that setting in my GitHub repo before opening the PR against it?

Thanks again for all your help! At the very least you've helped me get comments working without Recaptcha enabled, and made some good suggestions in your PRs 👍 I truly appreciate it :)

[VincentTam]

I don't understand why when I use the exact same request from PostMan to my personal staticman instance that it succeeds when I have my blog settings set to not use Recaptcha, but fails when it is set to use Recaptcha.

The root-level Staticman config file staticman.yml is used for the communication between the API server and the GitHub/GitLab repo. When reCaptcha.enabled is switched on, the GitHub/GitLab repo only accepts POST requests with the reCATPCHA parameters. I dunno how this can be done in PostMan since I've always used web browsers to reCAPTHCA.

That is a setting set in my blog GitHub repo (jekyll site)

According to https://github.com/eduardoboucas/staticman/issues/264#issuecomment-460446511, Staticman shouldn't care whether GitHub Pages are used or not. See my list of Staticman demo sites on Framagit for Hugo &/or Jekyll examples.

Thanks again for all your help!

Credits to Nicholas Tsim, the author Staticman's GitLab integration, who guided me through testing his PR.

At the very least you've helped me get comments working without Recaptcha enabled.

You may consult my demos on Framagit to get reCAPTCHA works, especially Minimal Mistakes: https://staticman-gitlab-pages.frama.io/minimal-mistakes-demo/post%20formats/post-image-caption/.

[deadlydog]

Thank you for the reply again @VincentTam . Consulting that demo and it's source code helped me get everything working. Essentially I had 2 problems that were preventing me from getting ReCaptcha working:

1. I had to duplicate the reCaptcha siteKey and secret fields from my staticman.yml into my _config.yml. This is because the minimal mistakes theme I'm using looks for them using site.reCaptcha.siteKey and site.reCaptcha.secret, which looks in the _config.yml and not the staticman.yml. Doing this got the ReCaptcha control actually showing up in my comment forms.
2. Even though I'm using my own hosted instance of staticman, I had naively still used the public instance (as per the docs) to encrypt my ReCaptcha secret. This led to my instance of staticman not knowing how to decrypt the secret properly. Once I changed to encrypt the value using my instance of staticman, then the decryption worked properly and everything worked. It took a bit of console line debugging to figure out this was the problem, but eventually found the error message that was getting eaten, instead of being returned back in the web response. Yay!

Thanks so much for your help. I'm going to spend a bit of time to see if I can edit the code to have it return back the actual error message in the web response to make troubleshooting these issues easier for others in the future. Thanks so much for your help!

[VincentTam]

Thanks so much for your help. I'm going to spend a bit of time to see if I can edit the code to have it return back the actual error message in the web response to make troubleshooting these issues easier for others in the future.

░░░░░░░░░░░░▄▄ ░░░░░░░░░░░█░░█ ░░░░░░░░░░░█░░█ ░░░░░░░░░░█░░░█ ░░░░░░░░░█░░░░█ ███████▄▄█░░░░░██████▄ ▓▓▓▓▓▓█░░░░░░░░░░░░░░█ ▓▓▓▓▓▓█░░░░░░░░░░░░░░█ ▓▓▓▓▓▓█░░░░░░░░░░░░░░█ ▓▓▓▓▓▓█░░░░░░░░░░░░░░█ ▓▓▓▓▓▓█░░░░░░░░░░░░░░█ ▓▓▓▓▓▓█████░░░░░░░░░█ ██████▀░░░░▀▀██████▀

[alexwaibel]

Duplicate of #318