search

10up / classifai

Supercharge WordPress Content Workflows and Engagement with Artificial Intelligence.
https://classifaiplugin.com
GNU General Public License v2.0
581 stars 52 forks source link

Auto-populate missing meta tags and descriptions #159

Closed jeffpaul closed 1 year ago

jeffpaul commented 4 years ago

Is your enhancement related to a problem? Please describe. For publishers with a high volume of posts created, it's easy to neglect meta tags and descriptions. We currently recommend setting a systematic fallback based on content type and title but it would be a lot better to have something contextual - ideally based on target keywords from the title and then from the content and setting rules (e.g., "no more than 60 words"). This would be a huge value add for large publishers who usually end up having to manually update meta data or utilize a sub-optimal systematic solution.

Describe the solution you'd like

Designs n/a

Describe alternatives you've considered n/a

Additional context n/a

jeffpaul commented 4 years ago

I did some research on Watson and Azure and didn't immediately see an API from either that would provide a text summary. I did, however, find the following items to consider:

The Watson code pattern is probably most worth continuing to review to see if it's a feasible option, but otherwise I didn't find something that would appear to easily help solve this problem.

jeffpaul commented 4 years ago

Related news in terms of integrating meta tags/descriptions with Yoast.

jeffpaul commented 4 years ago

OpenAI's Summarization seems like a potential option, but its in private beta currently. I've joined the waitlist to see about getting access for testing, but not a great option until that's public and ideally with a free tier.

jeffpaul commented 4 years ago

Sounds like OpenAI will launch GPT-3 publicly tomorrow (?) with pricing tiers that include a free level:

Explore: Free tier: 100K tokens or a three-month trial, whichever you use up first. Create: $100 per month for 2M tokens, plus 8 cents for every additional 1k tokens. Build: $400 per month for 10M tokens, plus 6 cents for every additional 1k tokens. Scale: Contact OpenAI for pricing.

jeffpaul commented 4 years ago

Given that this is still unverified as feasible with GPT-3 and no existing place to store this in core, this feature isn't something we'll build in the near term and may not be something we ever work on. However if someone reading this has a strong use case and idea on how/where to store in core, then we can look to come back to this later.

jeffpaul commented 2 years ago

Now that OpenAI has opened up to the public, an integration point here is feasible. We will need to explore what sort of auth process its API uses but seems like we could very feasibly look into automating the generation of meta descriptions and post excerpts.

dkotter commented 1 year ago

I really like the idea of integrating with some sort of summarization feature that could be used to auto-generate post excerpts (which could then be shown on the home page, archive pages, section landing pages, etc). It's a fairly common design pattern to show a 2-3 sentence summary of an article on those pages so automating that could be useful.

I've looked through the various links already in this thread and unfortunately most of them 404'ing now. In doing some of my own research, I looked at Azure, Watson and OpenAI.

Azure

For Azure, they do have a Summarization feature we could look to use. This is currently marked as a preview feature, so it could change or be removed all together. It is part of the Cognitive Services - Language API, which we haven't integrated with yet, so this would need to be a new Provider we add.

IBM Watson

For IBM Watson, they also have a Summarization feature as part of their NLU API. We already integrate with that API for our content classification, so I was able to quickly test this feature out.

This feature is marked as experimental and in my testing, doesn't give great results. If you read the link above, it mentions:

The summarization feature analyzes sets of sentences from a summarized document, and then chooses the set of sentences that best combines 3 objectives

And that was what I was seeing. Basically it always took the first 2-3 sentences of an article for the summary, which isn't really accurate. But because this is a feature of the NLU API, we could integrate with this really quickly (I could have a PR up in a couple hours if we decide this is worth adding).

OpenAI

The last thing I looked at was OpenAI. They don't have a specific summarization feature but they do showcase an example of their Completions API that will summarize an article. This seems to do what we want, though we haven't yet integrated with OpenAI and they have fairly strict limits around usage (no real free tier) and the amount of text you can send in one request (most articles will probably be too long to send in full).

Azure does now have an option to use OpenAI through their platform, though you have to apply for access. Doesn't appear they have a free tier either but may be worth considering since we already integrate with Azure.

Conclusion

At the moment, the IBM Watson Summarization feature would be the easiest to integrate with but doesn't seem to work well enough to be used in real-world scenarios.

The Azure Summarization feature seems decent but it's also marked as in a preview state so there is risk in building on that now. My suggestion for next steps is to utilize the Language Studio feature to test this out and see what types of summaries it provides to ensure things look good before proceeding.

I think it's probably worth building in OpenAI as a Provider at some point, even if we don't plan to utilize any of their features right away. That's not necessarily applicable to this issue but once that code is in place, does make it easier to test out those features (though the no free tier makes that harder).

cadic commented 1 year ago

@dkotter an update for OpenAI. From their own website, they provide a limited free access:

Start for free Start experimenting with $18 in free credit that can be used during your first 3 months.

jeffpaul commented 1 year ago

I would say the document abstractive summarization from Azure might be best as we already have an integration with Azure a free tier is available. Here's the quickstart guide for that implementation. Sample Azure Language access keys and endpoint are documented here.

dkotter commented 1 year ago

From their own website, they provide a limited free access

OpenAI does have a free credit they provide that expires in 3 months, the problem is once those 3 months pass or that credit is used, you are either stuck paying for it or creating new accounts to continue testing/building, whereas both Watson and Azure provide pretty generous free tiers. So not impossible to build on top of OpenAI, just makes it a little harder.

I would say the document abstractive summarization from Azure might be best

I agree but looking at this again, the document extractive summarization feature is publicly available whereas the document abstractive summarization feature is not. The former functions the same as the IBM Watson summarization feature, where it just pulls out a few sentences from the source content. The latter is what we want, where it tries to summarize the entire content.

There's a whole application process you have to go through to gain the proper access to use the abstractive summarization and from reading the requirements there, doesn't seem like we'll be able to get approved. So at the moment, the two easiest options to integrate with (IBM Watson and Azure) aren't as full featured as we probably want.

So I'm torn at the moment on if it's worth integrating with either of those or if we just wait until access is open up. I'm also leaning towards adding an OpenAI integration, which would allow us to get a summarization feature in place and then also opens up the possibility to more easily add other OpenAI features.

jeffpaul commented 1 year ago

I concur, OpenAI is most ideal if not for their time/money cap. I've got a contact there who I can reach out to in hopes we might get slightly better API access for development purposes and keep the team posted if that becomes more viable for longer term development on our end.

dkotter commented 1 year ago

I've opened a PR that integrates with OpenAI's ChatGPT service to automatically generate a summary of content and store that as an excerpt. Looks like Max has a WIP of integrating with the Azure Language provider which I think is fine to have multiple options here.

One thing I found out while doing this work, OpenAI has reduced their free limits to only $5 instead of $18 and accounts are tied to both email and phone number, so not easy to create new accounts for testing purposes. On the plus side, I only used like 5 cents of my credit while building this feature so it's not super expensive to build on