search

chiefonboarding / ChiefOnboarding

Free and open-source employee onboarding platform. Onboard new hires through Slack or the web portal.
https://chiefonboarding.com
GNU Affero General Public License v3.0
656 stars 121 forks source link

ChiefOnboarding v2.0 release #152

Closed GDay closed 2 years ago

GDay commented 2 years ago

Alright, so everything is almost ready to be pushed to master. The code has been tested quite well (and has a 93% test coverage), though it could still use some more test cases, especially with the custom integrations and perhaps on the frontend.

If all goes well, then everything will be merged on Thursday!

To summarize (and keep track of what needs to be done):

BREAKING CHANGES:

Implemented for release:

To be done right after release:

GDay commented 2 years ago

Can't merge it yet. There are a few bugs that I need to solve and I am missing the German translations (will be created and merged in the next few days).

woakes070048 commented 2 years ago

@GDay wanted to see how this was going. We are thinking about trying this for our company but we have been waiting for v2 and are not sure when an eta was. Thank

GDay commented 2 years ago

@woakes070048 Yeah, I am doing the best I can. There are a few bugs left in the sequences that I need to fix. The rest is working fine. Should be ready for production early next week.

GDay commented 2 years ago

@woakes070048 This will be merged on Thursday.

woakes070048 commented 2 years ago

@GDay thanks I look forward to trying this out.

MuratDoganer commented 2 years ago

Everyone will be F5'ing today :D

GDay commented 2 years ago

Haha! Will take a few more hours. I am currently running a few more live tests and fixing some minor bugs here and there.

MuratDoganer commented 2 years ago

We all really appreciate it @GDay <3

GDay commented 2 years ago

Well, this was a long day. So, I have just merged everything into master. I did a lot of testing today and most things seem to work fine, but will do more battle-testing in the next few days. I have one issue still standing where courses/resources don't update the modal when it's opened in Slack. It works locally, but not on my staging server, not sure what's going on there, but will give that another go tomorrow.

As always: use at your own risk and create a backup before updating. There are a lot of migrations that will be ran.

I have also updated the main post above with all the breaking change that I noticed.

One of the main features for this release is the webhook/integration option. You can now create your own integrations, like this one: https://integrations.chiefonboarding.com/d/1-asana-account-creation. This can then be used in a sequence. I will be creating more of those integrations with different services. Docs: https://github.com/chiefonboarding/ChiefOnboarding/blob/master/docs/Integrations.md

For some reason the docs workflow crashed, so the site is not up to date yet, but will fix that tomorrow.

Please report any bugs, so I can fix them :)

MuratDoganer commented 2 years ago

Hey @GDay loving v2! It looks so gorgeous!

Ran into a slight issue with the slackbot, Ive updated the url as suggested above, but the bot itself has completely stopped working :D

I am going to create a new test env so I can see if its just that particular bot

GDay commented 2 years ago

@MuratDoganer Oh, that's not good. Are you using the web socket or not?

If you are not using the socket, then see this (as mentioned above):

You can update Slack to use the internal websocket instead of the HTTPS calls webhooks. That way, you remove one open endpoint (technically, you could use the app now with Slack behind a VPN). Websocket is the default. If you don't want to use the websocket, then you will have to add SLACK_USE_SOCKET to False in your environment variables. It will then fall back to the "old" url and webhooks. It's important that you set this variable AFTER all migrations have ran.

If you are using the socket, then you will need to create an app token, see docs: https://github.com/chiefonboarding/ChiefOnboarding/blob/master/docs/Deployment.md#slack-bot

MuratDoganer commented 2 years ago

@GDay Yup done those!

I've just deployed with heroku to test a new integration and it worked fine, so maybe it had some trouble migrating?

GDay commented 2 years ago

@MuratDoganer Strange, in your Heroku logs, does it hit the webhook url when you write something to the bot? That should return a 200 status code.

MuratDoganer commented 2 years ago

@GDay Interesting, no it does not. Nothing comes up in the logs at all

I did use the new token under the SLACK_APP_TOKEN env, sockets are on, maybe ill try falling back to the old way and see what happens

GDay commented 2 years ago

Wait, you are currently using websockets, is that right?

If that's the case, then you should not have the SLACK_USE_SOCKET environment variable. It's on by default, so you don't need it. You will also need: SLACK_APP_TOKEN and SLACK_BOT_TOKEN as environment variables.

But yeah, perhaps good to fall back to webhooks for now until we get down to the issue.

MuratDoganer commented 2 years ago

Ah I was missing the first bit of the SLACK_BOT_TOKEN, i've added it in and I am seeing activity from the bot now in the logs

However trying to run any kind of command leads to this:


2022-07-08T16:10:16.942420+00:00 app[worker.1]: Traceback (most recent call last):
2022-07-08T16:10:16.942421+00:00 app[worker.1]:   File "/app/.heroku/python/lib/python3.9/site-packages/slack_sdk/socket_mode/client.py", line 155, in process_messages```
GDay commented 2 years ago

Hmm.. strange, I see it now as well on Heroku. However, the bot still seems to function fine. I am looking into this.

MuratDoganer commented 2 years ago

So I've reverted back to the old urls, still no slackbot responses but, i do now see the correct message in the logs

[08/Jul/2022:16:22:10 +0000] "POST /api/slack/bot HTTP/1.1" 200 0 "-" "Slackbot 1.0 (+https://api.slack.com/robots)"

GDay commented 2 years ago

Hmm... could you run this please? Go into heroku and click on "more" and then "run console". Then type in "bash" and hit "run", Then do "cd back" and then run "python manage.py shell". That will get us a python shell.

Then type:

from admin.integrations.models import Integration
print(Integration.objects.filter(integration=0).count())

If that shows anything more than 1, then that might be the issue. It will always take the first one. If you have tried to add the slack bot multiple times in the past, then the first one might have an old/invalid token.

MuratDoganer commented 2 years ago

Yes you are correct, I did indeed get back 1

GDay commented 2 years ago

If it got back 1, then that's not the issue. Very strange. I can't seem to replicate it on my side. It looks like Slack recognizes that you sent a message, but for some reason refuses to send one back.

Let's see if you can send a message manually.

Go back into the shell again with the instructions above. Then get your slack user id:

from users.models import User
u = User.objects.get(email="youremail@....com")
print(u.slack_user_id)

Send a slack message:

from slack_bot.utils import Slack
Slack().send_message(channel=u.slack_user_id, text="test message")
MuratDoganer commented 2 years ago

Yeah nothing is coming through, no error message either!

But in the other environment (from new) its working fine, so weird

GDay commented 2 years ago

Hmm... I have found an issue with websockets that may be why it wasn't responding back to you. I have just merged a PR for that. Perhaps you could try it again with the new update?

Also, is it just messages that don't work, or does it also not react to buttons?

GDay commented 2 years ago

Oh, also, it does actually save error messages when sending message (but it doesn't display those yet). It saves notifications. You can get the last one with (within shell):

from organization.models import Notification
note = Notification.objects.filter(notification_type="failed_send_slack_message").last()
print(note.description)
GDay commented 2 years ago

Alright, so at least on Heroku (not sure about other platforms), the websocket requests are incredibly unstable. I am constantly getting requests that take way too long and therefore timeout/crash.

The previous error:

File "/app/.heroku/python/lib/python3.9/site-packages/slack_sdk/socket_mode/client.py", line 155, in process_messages

seems to be related to the python version. With Python 3.8, it works fine. Both 3.9 and 3.10 show the error. Could also be related to Heroku, I am not entirely sure.

On local host (for development), I have never gotten them - it's always been fine. I am not sure why it's slow, perhaps Heroku doesn't like the threading that we need to do to keep the websocket up and alive. I have now set SLACK_USE_SOCKET to False by default and I would recommend using the webhooks for now.

If you want to do a query to Slack directly (using their library), then you can do this (through shell again):

from admin.integrations.models import Integration
from slack_bolt import App as SlackBoltApp
from users.models import User

u = User.objects.get(email="youremail@....com")
integration = Integration.objects.filter(integration=0).first()                                                                                                                                                          
client = slack_sdk.WebClient(token=integration.token)
client.chat_postMessage(channel=u.slack_user_id, text="Whatever you want to say")

If it errors, Slack will raise an exception that will be printed out in your console.

MuratDoganer commented 2 years ago

Okay, getting this:

redirect_uri did not match any configured URIs. Passed URI: https://mycompaniesurlforchief.com

GDay commented 2 years ago

In your Slack app settings: https://api.slack.com/apps.

Go to "OAuth & Permissions" and change the "redirect url" to https://mycompaniesurlforchief.com/admin/integrations/slack (swap the domain with yours). You probably have "api" instead of "admin" in the url right now.

Not sure why you are getting that though, because that url is not used for slack messages, I think.

GDay commented 2 years ago

Yeah, looking through the code, that's not being used with Slack calls. You got that error when you sent a message to the bot through the code above?

MuratDoganer commented 2 years ago

I dont know why, but that did fix it

I can only speculate it wasnt authorising properly?

MuratDoganer commented 2 years ago

Yeah that was the error message I got when I used the code above.

It's odd, but changing /api/ to /admin/ like you suggested worked, I am now getting messages in Slack again

GDay commented 2 years ago

The redirect url is only used when we are installing the app iirc. But hey, if it works for you now, then it should be fine, I guess.

I am pretty sure my bot worked even without changing that url in the slack app. Oh well. Glad it works now for you :)

Also, make sure to update to the latest version, I fixed a few bugs :)

MuratDoganer commented 2 years ago

Updated! Thanks Gday :)

Kuuchuu commented 2 years ago

@GDay Is there a way to verify which version I am currently running? Thanks for all your work on this project!

GDay commented 2 years ago

@Kuuchuu You are welcome! That kinda depends on how you run chiefonboarding. If that's through heroku, then you can just check the commit that is on the heroku server. (or just push it and it will either update or tell you it's up to date).

If you are using Docker, then you are probably still on the old version. Will push a new docker image in a few hours.

GDay commented 2 years ago

@Kuuchuu Docker image has now also been updated. I have not tested migrations, but I quickly ran the image on a server and it works fine, so migrations should also work fine. As always though: make a backup before you update.

If you are using the latest tag of the docker image, then I don't think you can check on what version you actually are. Starting this version (v2.0), there is a version number in the footer of the dashboard. After this week, that will be updated when new versions are deployed. This week is mainly pushing a few more features and bug fixes and at the end of this week, I would consider it to be stable.

MuratDoganer commented 2 years ago

Another bug: In heroku latest build; duplicating a sequence returns a Server Error (500)

GDay commented 2 years ago

@MuratDoganer Thanks for reporting that. I have just fixed it (just merged the PR). I have noticed that there are some performance issues with duplicating very large sequences. It can take several seconds before it's done. I might need to move it to a background task, not sure yet, but duplicating should work now (with a little patience, haha).

MuratDoganer commented 2 years ago

Thanks @GDay you're a real legend :)

If you need a tester I am happy to help haha

MuratDoganer commented 2 years ago

Found another one! Uploading files / pdfs, get an upload failed message (I do have aws s3 settings in my env in heroku)

Does this need a new integration?

GDay commented 2 years ago

Thanks. Nope it doesn't need an integration - only the environment variables.

Small typo caused that: https://github.com/chiefonboarding/ChiefOnboarding/pull/183/files

Merged into master now!

MuratDoganer commented 2 years ago

Updated, checked, same file upload failed

Let me do another test to make sure its not anything else on my side!

GDay commented 2 years ago

You could check the network tab in your browser. When you are uploading a file it should make two requests: one should return a url (presigned url to upload to) and the second request should be the actual uploading to AWS (or equivalent). If any of those two fails, then it's likely one of the environment variables.

MuratDoganer commented 2 years ago

Yup that looks okay, no failures in Network tab, but console is showing this message:

Block «paragraph» skipped because saved data is invalid

GDay commented 2 years ago

Yeah, that's because it failed for some reason (and then the editor doesn't save the block).

So you get this, right?

image

Or maybe it cached some of the javascript? Perhaps try in incognito?

MuratDoganer commented 2 years ago

Hmm, yes but in Incognito I got this:

Failed to load resource: the server responded with a status of 500 (Internal Server Error)

GDay commented 2 years ago

Make sure you have all environment credentials set up: 

AWS_S3_ENDPOINT_URL=https://s3.eu-west-1.amazonaws.com
AWS_ACCESS_KEY_ID=AKIAXXXXXXXX
AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXX
AWS_STORAGE_BUCKET_NAME=bucket-name
AWS_REGION=eu-west-1

If you want a more detailed error then go into shell and run:

from misc.s3 import S3
S3().get_presigned_url("test.txt")
MuratDoganer commented 2 years ago

Yeah thats all good, tested outside chief and is working as expected there - was also working in v1

Ill run that and see what it says one moment

MuratDoganer commented 2 years ago

No error messages in shell, the plot thickens!

GDay commented 2 years ago

I have no clue. It's so weird. I have tested it in Linux (firefox) and on Windows (chrome and firefox) and all seem to work just fine on my demo server. Tested in templates as well as in the sequence.

Could you try a different file? And perhaps in a different place in the app (shouldn't matter, but who knows)?

MuratDoganer commented 2 years ago

Yep tried different files at different sizes, pdfs, images, etc none work

I am thinking I might just spin it up in Render and see how I get on there (we are testing after all!)