Vimeo OEmbed blocked?

[FransdeJong]

There is something going on with Vimeo OEmbed on Cloud. If I try to select a Vimeo video it takes a long time and after what seems to be a timeout it returns this response: {"OEmbedStatus":1,"SupportsDimensions":true,"Markup":null}

Locally I get this response: {"OEmbedStatus":2,"SupportsDimensions":true,"Markup":"<iframe src=\"https://player.vimeo.com/video/70591644?app_id=122963\" width=\"360\" height=\"203\" frameborder=\"0\" allow=\"autoplay; fullscreen; picture-in-picture\" allowfullscreen title=\"Test Video for Embed\">"}

This happens on all project I tried it on. It looks like cloud doesn't allow calls to Vimeo? (and maybe other services?)

[hjaltedaniel-umbraco]

Hi Frans,

Thank you for raising this issue 👍

This is an issue we have seen before - and it comes from Vimeo having implemented a ban on the Umbraco Cloud public IPs.

What seems to be happening is that;

1. A random site on Umbraco Cloud makes way too many requests to the Vimeo API
2. Vimeo implements a 24 hour ban on the IP the site currently has
3. The ban is into effect for all Cloud sites currently on that IP This repeats every time the random site exceeds Vimeos allowed requests.

There is unfortunately not much we can do about this. Obviously, it's less than ideal that a spammy user can ruin it for everyone else. We do not know who it is so we have no way of reaching out to the spammy user(s). From a Cloud perspective the outgoing IPs are needed, so that our customers with payment providers can give a known IP that can be used for their setup. And in general for IP restriction needs.

My best recommendation is that you get in contact with Vimeo and search for alternative solutions to the built-in embed provider. Another option can maybe be to implement a macro to insert Vimeo videos?

Feel free to reach out to support if anything is unclear 😅

[FransdeJong]

I understand what you are saying but this I fid this answer hard to accept... If I host Umbraco on a hosting solution designed specially for Umbraco I loose functionality and have to spend extra time to fix embedding for my customers? On every other hosting platform this works just fine...

I understand this is a issue you cannot solve straight away but I think it is in Umbraco's best interest if you contact Vimeo and find a solution for cloud as a platform.

If we keep this mindset it's a question of time before another part of Umbraco breaks on cloud?

Can you please reopen this issue I can't seem to reopen it.

[dawoe]

Hi @hjaltedaniel-umbraco

I just got of a phone call with a customer who is running in to the exact same issue.

On the umbraco cloud page it says this

Let’s just start with getting this one out of the way: the core CMS is the same no matter how you host it (Cloud or not).

A little bit strange you think with that line in mind, that you ask customers of the hosting platform provided by Umbraco HQ, to create a work around for core functionality because it does not work on their platform ?

I agree with @FransdeJong that this workaround should be created by HQ or that HQ contacts vimeo to fix the problem.

And also from your response you suspect that is caused by a "spammy" user. Than you should try to pinpoint this user and work on a solution with them. Now all your other customers suffer from problems caused by one user.

Dave

[c9mb]

I've hosted on other platforms where they were able to use network logging to trace excessive requests to a service, and isolate the culprit.

[FransdeJong]

I've hosted on other platforms where they were able to use network logging to trace excessive requests to a service, and isolate the culprit.

If Cloud grows the normal requests added together will mean a ban anyway so I'm curious if this will be a futureproof solution.

[dawoe]

@hjaltedaniel-umbraco

I was looking at vimeo docs to see what the rate limit is on the oembed endpoint. But I can't seem to find any info on that : https://developer.vimeo.com/api/oembed https://developer.vimeo.com/api/oembed/videos

I am also looking at the Rate Limits : https://developer.vimeo.com/guidelines/rate-limiting But does seem to apply to the API access only. And according to the first link OEmbed is seperate from the Vimeo API. So I assume that rate limits don't apply there.

Dave

[nul800sebastiaan]

Hey all, I don't know if this was the actual reason for Vimeo but before we go any further:

A little bit strange you think with that line in mind, that you ask customers of the hosting platform provided by Umbraco HQ, to create a work around for core functionality because it does not work on their platform ?

I imagine more hosting providers with shared IPs will run into this problem if Vimeo is indeed blocking IPs.

I was looking at vimeo docs to see what the rate limit is on the oembed endpoint.

I can not imagine that any service like this would work without rate limiting, just because they haven't documented them, it doesn't mean they don't exist.

Has anyone been able to actually confirm this is the problem?

[FransdeJong]

Has anyone been able to actually confirm this is the problem?

Hi Sebastiaan,

Thanks for helping out!

I don't think it's possible for me to confirm this is the case. These were the words of Hjalte in his reply. When I look at the log I see this error on Cloud:

This gives me no information about why it isn't working but it just isn't. What I do know is the following:

• I see this behavior on all sites I test on Cloud.
• I don't see this behavior on the same sites locally
• I don't see this behavior on other sites on our own server

The only way for me to debug this issue is to add logging to UmbracoCMS build a custom DLL and push that to a live environment. That seems a bit radical (and frankly I'm afraid to do something like that 😁) for something that is probably a platform issue since more people run into this issue.

What I can do is help you find the root of this issue and hopefully find a way to make our and your customers happy again.

Please let me know how we can help you debug this issue.

Frans

[nul800sebastiaan]

I've just been reading up on this and clearly we're not the only ones affected by this: https://meta.discourse.org/t/vimeo-embed-not-working-on-my-site-due-to-vimeo-server-ip-blacklisting/126479/56

Ultimately, I am afraid that oEmbed is a wonderful but naive technology and IMO, I don't see it existing for much longer as it's an excellent way for people to get quality content for free to put in their spam sites. We already saw that Instagram abandoned oEmbed for presumably the same reasons. It's impossible to ban anonymous users nicely so they have to resort to IP bans which is unfortunate. So in the end, the only stable solution I see is to have an API authenticated datatype for things that can now be embedded using oEmbed.

Until that time, I'm sure we can reach out to Vimeo and see if we can get unbanned, but it will be a game of whack-a-mole and it's unpredictable to know if we get banned again.

It seems unfeasible for us to analyze all the outgoing traffic to see patterns and take action before Vimeo does. If this is something we could do then I am sure we won't be able to make that happen soon either (and then we're back to making sure we can embed using proper API keys).

[nul800sebastiaan]

And yes, it seems like this is the problem at the moment, same error as described in that discourse thread.

This happens if I try to do a request in Kudu:

[FransdeJong]

Thanks for the extensive answer. This is something I can go back to my client with. I would suggest adding a disclaimer to this page in the documentation warning about the possibilities of ip blocking on cloud services for example cloud? https://our.umbraco.com/documentation/extending/Embedded-Media-Provider/

[c9mb]

@nul800sebastiaan - I believe that the abandoned Instragram OEmbed issue refers to their legacy OEmbed API, which has since been replaced by their new OEmbed API.

OEmbed is a documented and recommended mode to access Vimeo data, and should work. https://vimeo.com/api/oembed.xml?url=https://vimeo.com/65107797

[nul800sebastiaan]

@c9mbundy Correct, for Instagram, see the discussion in https://github.com/umbraco/Umbraco-CMS/issues/9155.

Vimeo oEmbed does work, until Vimeo hands out an IP ban, which is what happened here.

As an update, we've contacted them to get the block removed, will update here when we get a reply on that.

[nul800sebastiaan]

We got an answer back from Vimeo, linking to https://developer.vimeo.com/api/authentication#table-2

So it wasn't even people using oEmbed too much. Not sure I understand this IP ban.. Presumably to generate a client_credential token you, need to provide a username/password or something to identify you? Why ban a whole IP when you could ban the single person trying to generate a token? I haven't looked into their API, but it seems like poor API design/rate limiting design on their part.

[nul800sebastiaan]

FYI: we're following up on this but seeing Vimeo's replies to other hosting companies, I suspect we won't get a satisfactory answer for this.

[dawoe]

Hi @nul800sebastiaan

Could you figure out which site is generating the client_credential and contact the owner of that site.

Dave

[nul800sebastiaan]

We don't have that ability at the moment. And I would also say that it's up to Vimeo to figure that out (they didn't want to tell us either).

[umbrabot]

Hiya @FransdeJong,

Just wanted to let you know that we noticed that this issue got a bit stale and might not be relevant any more.

We will close this issue for now but we're happy to open it up again if you think it's still relevant (for example: it's a feature request that's not yet implemented, or it's a bug that's not yet been fixed).

To open it this issue up again, you can write @umbrabot still relevant in a new comment as the first line. It would be super helpful for us if on the next line you could let us know why you think it's still relevant.

For example:

@umbrabot still relevant This bug can still be reproduced in version x.y.z

This will reopen the issue in the next few hours.

Thanks, from your friendly Umbraco GitHub bot :robot: :slightly_smiling_face: