Open smoya opened 2 years ago
Hope I'm not butting in too much but in case you guys want a quick and easy solution for this (until you develop your own), you can use https://thumbsmith.com/
(Full disclaimer - I'm the founder ๐ฌ )
Usually this is more for websites that can't easily deploy services like you guys tho (eg: wordpress sites)
@smoya Thanks for that awesome idea! I understand implementation of that but the main problem in current solution (how Studio works) is that we have full SPA application, so everything is loaded and resolved in the runtime in the JS, so even if we update og:meta
in the head of the page the web crawlers (like in fb for og:meta
) will have already read this metadata and will not wait for JS to load and execute. I don't know how to get around this problem. One of the possible solutions could be to intercept the server-side request and check if the queryer is e.g. a crawler and return him the appropriate html with the appropriate metadata, but for this we should have a simple server.
@loteoo Hi and thanks! :) It may sound stupid, but in your solution there is a support for SPA applications that would change this metadata at runtime (see my comment above)?
Hi @magicmatatjahu, my pleasure! Unfortunately as you mentioned the meta tags really needs to be in the initial HTML for the crawlers to pick it up. You will have to do SSR, or wait for something we had actually planned on our roadmap similar to this. (But you will have to share a custom URL, not the original URL)
@magicmatatjahu as we are hosting the Studio in Netlify, we can make use of prerendering which will prerender pages when those are requested by crawlers. See https://docs.netlify.com/site-deploys/post-processing/prerendering/
EDIT: I tested this creating a new site from Studio in my personal Netlify account. Just by enabling prerendring and adding a simple js script, we can make it work:
<script>
const params = new URLSearchParams(window.location.search)
if (params.has('base64')) {
document.querySelector('meta[property="og:image"]').setAttribute("content", "https://example.com/?base64=" + params.get('base64'));
}
</script>
Source: https://github.com/smoya/studio/blob/master/public/index.html#L32-L37 replace example.com with the preview image service URL we would deploy
curl "https://hopeful-liskov-2a657c.netlify.app/?_escaped_fragment_=&base64=BASE64HERE" | grep og:image
...
<meta property="og:image" content="https://example.com/?
base64=BASE64HERE">
@smoya https://www.youtube.com/watch?v=9CS7j5I6aOc Can I say that we are in home? ๐
@smoya So, do you wanna handle that feature and create such a lambda for Netlify? I don't know where we should keep the source code of this lambda, in the Studio or as a new repo in the organization?
@smoya So, do you wanna handle that feature and create such a lambda for Netlify? I don't know where we should keep the source code of this lambda, in the Studio or as a new repo in the organization?
The pros of having it in this repository is that it will be always in sync with the code and everything will be handled by Netlify at the deploy level. The cons is that we will be adding more code to this repository that is not really needed for running Studio. I still need to cleanup the code so I would say if the final code is not so big, I would advocate for adding it to this repository.
WDYT?
@smoya then we have it in this repo :)
@mcturco I would like to ask you if you could help me with the design part for this.
@smoya yes, I can help out with a design for this! Are there any limitations as far as layout/styling goes? I will use your example images to reference what content will be included, but was just wondering how it would be implemented since I see the open graph image is gathering meta information and not using HTML (unless I am incorrect?)
@smoya yes, I can help out with a design for this! Are there any limitations as far as layout/styling goes? I will use your example images to reference what content will be included, but was just wondering how it would be implemented since I see the open graph image is gathering meta information and not using HTML (unless I am incorrect?)
Those images I attached are generated from HTML. What we need is to design those cards in HTML (CSS, TS, whatever). The HTML used for those images is located here. I guess we could ask @magicmatatjahu or any other user from the community to help with that part, but still the design is needed. The minimum size would be 1200 x 630 (as recommended for high-res displays).
About the data we can display on it, I'm up to suggestions. I thought on:
Anything you all think we could add/remove from it? (we should try to avoid overloading the image, so some could be drop if needed).
@smoya sounds good! yeah just wanted to make sure that I can apply some of the new styles that we have been using as part of the brand refresh. Cool! I can get to work on that ๐
Hi all! Sorry for the delay on my part for this issue. Going to add this back onto my list as we are working towards launching the new brand stuff. I will be using the new logo/colors/typography for this open graph ๐
This issue has been automatically marked as stale because it has not had recent activity :sleeping:
It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.
There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.
Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.
Thank you for your patience :heart:
This issue has been automatically marked as stale because it has not had recent activity :sleeping:
It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.
There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.
Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.
Thank you for your patience :heart:
still relevant
Hello everyone ๐,
I was looking for some issues to contribute and help out, and found this one related to SEO ๐. @smoya @mcturco Is there anything code related that we can create a PR, even just a draft PR to start with? From what I understood the code you have on your fork @smoya could just go to this PR. and we'd deploy that Netlify function.
Would love your feedback, if there is anything I can help let me know ๐
@BOLT04 It's hard to say where to start because we need to test the Netlify preview feature, write a lambda function that would generate such images (using this vercel-og project and changing it a bit for our use case) and then connect it together. It's hard for me to write where to start and how difficult it is. However, if you want we can discuss it :)
one question @magicmatatjahu, could we start development using a sort of mock design? Then when @mcturco has the final version for this open graph preview, we change the code to use that?
@BOLT04 Yeah, we can mock "preview image" in the development time and at the end change it. Sergio exactly made it in this way, he focus on logic and make mock of preview image https://github.com/asyncapi/studio/issues/224#issue-1097178110 We can even reuse his code - but code has 1 year so probably we need adjust it to the latest "standard" of netlify :) Do you wanna handle it? btw. sorry for delay in response!
This issue has been automatically marked as stale because it has not had recent activity :sleeping:
It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.
There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.
Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.
Thank you for your patience :heart:
still relevant
@smoya , Looks interesting, i would like to work on this under the mentorship program if it gets selected.
This issue has been automatically marked as stale because it has not had recent activity :sleeping:
It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.
There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.
Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.
Thank you for your patience :heart:
Still relevant
Thanks for creating a new pitch ๐ฅณ. You can now create or link existing scopes. You can create new scopes in two different ways:
Option 1
See this example
Option 2
related to #ISSUE_BET_NUMBER
See this example
A sample implementation, I had built for one of my projects can be found here.
A preview can be found here with this generated image
I added a small sequence diagram in the description of this issue showing the big picture of a request made to retrieve the open graph image. Also a small comment about image caching.
Hey everyone hey @smoya , I am Ashmit Jagtap, from Indian Institute of information Technology, Pune. I am looking forward to contribute to this project as a part of GSOC 2024. I may be a bit late but it turns out, well. Looking forward to collabrate :)
Hey @smoya I had been reading more about the issue and looking at ways we can approach it. Are there any prerequisits that need to be gone through or any qualification task for this project?
Hi @smoya and community, I'm Claudia Ma, preparing myself for contributing to this issue in GSoC 24. As a step 1, I am working on understanding the issue by reproducing the current progress of generating the preview image through og-image.
Currently I encountered several questions during reproducing, and I listed as below. I was wondering if I have some misunderstanding towards the logic of calling og services. If there is anything wrong in my reproduction logic, can you guide me through it so that I can at least reproduce the current progress? Appreicate any insights and guidance!
1) For og-image service:
I forked @smoya 's version, the only modification I made is in package.json
where I made node version to 16.x
(in purpose of letting the site run locally, since using nvm use v14
does not work on my laptop. )
After running vercel dev
at localhost:3000
, I found the service have several errors as below.
And the site is not 100% working:
I was wondering if those are the expected perforance since you've made some modifications, or it should work correctly.
2) For AsyncAPI Studio:
og:image
tag in studio/public/index.html
. Here is my modification, where the service url is localhost:3000
(og-image).localhost:3001
. 3) For deployment:
Thank you so much for reading my questions! Sorry for making it little bit long as a comment, I will also dig deeper into it at the same time.
Hello @smoya
Hello! I'm Aditya Singh, a Computer Science undergraduate from India with a passion for open-source contributions. Previously, I had the opportunity to participate in GSOC 23 at Rocket.chat, which was an enriching experience. This season, I'm eager to channel my skills and knowledge into contributing to AsyncAPI. The project's requirement matches my tech stack, and I'm excited about the potential impact I can make by contributing to JSON Schema. I'm looking forward to making some worthy contributions๐
Async API Open Graph This project envisions enhancing the social media sharing experience for AsyncAPI To create a img preview -
Hang in there, soon I will try to present a prototype ๐
FYI, I updated the description of the issue with a detailed list of work to do and prerequisites. Also added clarifications about deprecated og-image package and suggested using a newest tool.
@smoya Hello! I'm Athul and I'm looking forward to contribute to this issue. I've been contributing to Modelina recently. I went through the description, prerequisites and the todo. I'll come up with a MVP soon. Looking forward to making a meaningful contribution to the project!
FYI, I updated the description of the issue with a detailed list of work to do and prerequisites. Also added clarifications about deprecated og-image package and suggested using a newest tool.
heyy @smoya that's great, BTW I tried to implement a supermini working model, whenever someone pastes the link a preview appears
@RegretfulWinter i had the some of the same errors that you mentioned.
The third error that you mentioned, I had the exact same issue.
This is because apps/studio-next
is being deployed by default, and we need apps/studio
to be deployed. Once, i configured apps/studio
to be deployed, I got the desired output.
I've created this basic page that follows the Open Graph protocol. It includes a basic JavaScript event handler and is hosted on Netlify. It's fascinating to observe how it renders differently on various platforms. Try It
Thank you @smoya and @Athul0491! ๐ Thanks to your help and sharing, I've solved all the questions I mentioned last time. Updating that my first version of MVP has come alive on Netlify that covers basic workable functions mentioned before, as well as several issues I've discovered during development.
1. MVP:
/api/og?base64=<encoded AsyncAPI>
to call the HTTP GET api
for the picture. Will improve the design and preview picture design later:https://github.com/asyncapi/studio/assets/20860329/b5b5eaac-ecbf-456e-be05-f3a7de2fa6fb
2. Current issues for the implementation: 1) Preview is not stable - sometimes the local cache will delay the real-time effect. Probably there is better way of implementing the API and cache management (e.g. Middleware) 2) Currently the speed of preview picture generation on Studio side is slow. Would research more on the cause and how to solve it. 3) Although the service can update preview pictures as AsyncAPI have changed, the script in Studio's index.html cannot update real time. 4) Need to improve the design.
P.S. Several other issues that may worth discussion : 1) URL shortening for super long AsyncAPI documents: it's hard for people to copy-paste even in the preview (apps like Slack will remind you the link is too long). Would be great if the preview link can shorten. 2) Adding preview design templates
Sincerely Appreciate any feedbacks and discussions!
Hello :wave: @smoya, After going through the description, prerequisites and the todo, I came up with a MVP.
This is an HTTP API that generates an Open Graph image based on:
To generate an Open Graph image using the URL pointing to raw AsyncAPI document:
Go to /api/og?docurl={link_of_your_doc}
To generate an Open Graph image using base64 encoded AsyncAPI document:
Go to /api/og?base64={base64_encoded_doc}
Example:
Dynamic OpenGraph image
<head>
tag of public/index.html
. This works for now on my hosted Studio, but I am not sure if it is a concrete solution.
Preview:
Thoughts: react-helmet also does not seem to be a viable option as it does not provide support for web crawlers and it does not provide good compatibility with opengraph data. It is also considered to be deprecated as it is not currently being maintained.
Ideas:
Any suggestions and insights are highly appreciated
Cheers, Athul :smile:
@RegretfulWinter
Preview is not stable - sometimes the local cache will delay the real-time effect. Probably there is better way of implementing the API and cache management (e.g. Middleware) What do you mean by "local cache"?
Currently the speed of preview picture generation on Studio side is slow. Would research more on the cause and how to solve it.
I assume you enabled Netlify's pre-rendering. I did test sharing a link through your deployed Studio instance and indeed it is so slow that Slack didn't render the preview image of the link ๐ค
I sent an email to Slack to understand better what's their timeout when expanding links. However do not expect a quick answer.
I recommend you to debug your code and see where it is spending so much time. If it ends up being the parsing part of it, unfortunately there is no so much to do there . I'm not that familiar with NextJS but I would tell you to check if that's penalizing the request time.
Although the service can update preview pictures as AsyncAPI have changed, the script in Studio's index.html cannot update real time.
You can click on the share button in Studio and if there are changes, a new base64 will be generated.
URL shortening for super long AsyncAPI documents: it's hard for people to copy-paste even in the preview (apps like Slack will remind you the link is too long). Would be great if the preview link can shorten.
There was an issue (now closed as won't do) related to this https://github.com/asyncapi/studio/issues/127
FYI: I added the following statement to the description of the issue:
In case the image can't be generated due to whatever reason, the default AsyncAPI Studio should be served instead: https://studio.asyncapi.com/img/meta-studio-og-image.jpeg
@Athul0491
To make the OpenGraph image show dynamically based on such parameters, I tried added a script in the
tag of public/index.html. This works for now on my hosted Studio, but I am not sure if it is a concrete solution.
In the description of the issue there is a mention to Netlify's prerendering and the reason why you need a solution like that.
react-helmet also does not seem to be a viable option as it does not provide support for web crawlers and it does not provide good compatibility with opengraph data. It is also considered to be deprecated as it is not currently being maintained.
Do we really need a framework for this simple app? What are the pros/cons on this side?
The speed of preview picture generation on Studio would be slow unless caching is applied
Still generating seems slow, so the issue seems more on the first request, meaning caching won't help here. I commented the same to another contributor in https://github.com/asyncapi/studio/issues/224#issuecomment-1988363145
@smoya The shared link through my deployed Studio instance does render the preview on Slack. I'm attaching the screenshot below.
Screenshot:
@smoya The shared link through my deployed Studio instance does render the preview on Slack. I'm attaching the screenshot below. Screenshot:
Can you share the link to your Studio?
Can you share the link to your Studio?
I guess i forgot to add the link to my deployed studio instance :sweat_smile: Here's the link
Here's the link
Oh, I see now it works slow, but it works. ๐ Thanks for sharing. Anyway, I encourage you to debug where the time is spent and see how can be improved.
Oh, I see now it works slow, but it works. ๐ Thanks for sharing. Anyway, I encourage you to debug where the time is spent and see how can be improved.
I do agree that it is slow ๐
I'll see if i can optimize my code and also check if any part is taking too much time and how it can be improved. Thanks!
FYI, I got an answer from Slack support, and they answered the following:
We are expecting the data used for unfurls to be available within 5 seconds of the request being made.
Obviously this is just Slack, how the rest of services set their timeouts is unknown. But I believe 5 seconds is already a lot, so we should aim to have a lower timeout IMHO. Unless the bottleneck is the Parser, obviously. Just to clarify, it is 5 seconds of timeout to fetch the image.
I did test sharing a link through your deployed Studio instance and indeed it is so slow that Slack didn't render the preview image of the link ๐ค
Thank you for the feedback @smoya . Sorry for the late reply. I tested again today and see it's not producing the same result when I commented. Will debug today and see how I can streamline my logic of the code.
OpenGraph Studio issue
Reason/Context
Thanks to the
?url=<url-of-file>
and?base64=<base64-encoded-doc>
query param, Studio can load most of files (yes, not all of them, see https://github.com/asyncapi/studio/issues/127). I expect users will use that to share their AsyncAPI docs.Whenever a link to Studio (with or without those query params) is pasted into social media (Twitter, Linkedin, Facebook, Slack...), the preview image is this one:
It is a great pic, however it says nothing about the file being shared.
What if we could dynamically generate the preview image based on the file being shared? For example, the title, description and some stats could be shown.
I created a POC based on https://github.com/vercel/og-image (deprecated atm), available in my fork (It's just a POC) which is a server that generates dynamic images for being used on Open Graph image meta tags. This works by generating a dynamic HTML, making an screenshot of it through headless Chromium, and serving the resulting image.
The server accepts a
?base64=<base64-encoded-doc>
query param, and generates an image that contains the AsyncAPI doc Title, Description, number of servers, channels and messages.Despite the horrible design, the service is able to generate the following:
Based on the following AsyncAPI doc:
See
```yaml asyncapi: '2.2.0' info: title: Account Service version: 1.0.0 description: This service is in charge of processing user signups channels: user/signedup: subscribe: message: $ref: '#/components/messages/UserSignedUp' components: messages: UserSignedUp: payload: type: object properties: displayName: type: string description: Name of the user email: type: string format: email description: Email of the user ``` [Open in Studio](https://studio.asyncapi.com/?base64=YXN5bmNhcGk6ICcyLjIuMCcKaW5mbzoKICB0aXRsZTogQWNjb3VudCBTZXJ2aWNlCiAgdmVyc2lvbjogMS4wLjAKICBkZXNjcmlwdGlvbjogVGhpcyBzZXJ2aWNlIGlzIGluIGNoYXJnZSBvZiBwcm9jZXNzaW5nIHVzZXIgc2lnbnVwcwpjaGFubmVsczoKICB1c2VyL3NpZ25lZHVwOgogICAgc3Vic2NyaWJlOgogICAgICBtZXNzYWdlOgogICAgICAgICRyZWY6ICcjL2NvbXBvbmVudHMvbWVzc2FnZXMvVXNlclNpZ25lZFVwJwpjb21wb25lbnRzOgogIG1lc3NhZ2VzOgogICAgVXNlclNpZ25lZFVwOgogICAgICBwYXlsb2FkOgogICAgICAgIHR5cGU6IG9iamVjdAogICAgICAgIHByb3BlcnRpZXM6CiAgICAgICAgICBkaXNwbGF5TmFtZToKICAgICAgICAgICAgdHlwZTogc3RyaW5nCiAgICAgICAgICAgIGRlc2NyaXB0aW9uOiBOYW1lIG9mIHRoZSB1c2VyCiAgICAgICAgICBlbWFpbDoKICAgICAgICAgICAgdHlwZTogc3RyaW5nCiAgICAgICAgICAgIGZvcm1hdDogZW1haWwKICAgICAgICAgICAgZGVzY3JpcHRpb246IEVtYWlsIG9mIHRoZSB1c2Vy)Studio will need to modify the
og:image
tag so it points to this new service.The preview image would then look like (note that https://shaggy-stingray-56.loca.lt/ was a local tunnel to my localhost serving a simple html with the
og-image
tag):By the way, all of this could run on serverless functions such as the Netlify functions (which are AWS Lambda) available in free tier :)
Description
Here is a sequence diagram showing the big picture of the flow a request made by an Open Graph crawler (crawlers used for querying the open graph image whenever you share a link) will follow:
Note that, as explained in this comment, we would need to configure pre-rendering in Netlify for doing the og:image content URL replacement on each request made by a crawler.
Alternatively, whatever technology we use (for example NextJS), the flow for rendering the Studio page would be something like the following:
In case the image can't be generated due to whatever reason, the default AsyncAPI Studio should be served instead: https://studio.asyncapi.com/img/meta-studio-og-image.jpeg
What you will need to do
Note that the design of the Open Graph image card is also part of this task. Ask @Mayaleeeee for help on this (Thanks! ๐ ).
Prerequisites
Work to do
Create then a new service that exposes an HTTP API that generates an Open Graph image based on few query params (use the names you want, the following are just suggestions):
Some hints:
og:image
meta tag content to point to the new service URL including thedoc_url
ordoc_base64
query param with the right content. That will be the trick that will make the OpenGraph image shown dynamically based on such parameters.Investigate about caching. If hosted as a Netlify function, I believe we could just trust in cached responses. See https://docs.netlify.com/platform/caching/#supported-cache-control-headers. Otherwise, we could give a try to Netlify Blobs and store each generated image using the base64 hash (or a reproducible and atomic hash) so every new request first check if that image is already generated and in the case it is, serve the blob directly (not 100% if this use case can be supported, but I guess it is).
Anyway, more investigation on how to implement the service should be taken, so please do not take my words here as the right way to do it as I didnโt spent time on it when I created this issue.
GSoC 2024
This issue got accepted as part of the GSoC 2024. @helios2003 is assigned as mentee.
We are using the following read-only Project board to track the current status of it's work: https://github.com/orgs/asyncapi/projects/49/views/1