search

migueesc123 / YoutubeAnalytics

A Microsoft Data Connector or Power Query Connector for YoutubeAnalytics data
MIT License
61 stars 21 forks source link

Error accessing Table: YoutubeAnalytics.Videos #4

Closed michael534 closed 7 years ago

michael534 commented 7 years ago

Wow! You have done a fantastic job not only creating this connector, also providing really clear instructions. I am all the way everything works. Almost. There is just one table I cannot access via the connector. Sadly it is the one table that really matters. For some reason YoutubeAnalytics.Video returns this error:

DataSource.Error: Web.Contents failed to get contents from 'https://www.googleapis.com/youtube/v3/search?part=snippet,id&order=date&maxResults=50&forMine=true&type=video' (400): Bad Request Details: DataSourceKind=YoutubeAnalytics DataSourcePath=YoutubeAnalytics Url=https://www.googleapis.com/youtube/v3/search?part=snippet,id&order=date&maxResults=50&forMine=true&type=video

I am using Power BI Desktop Version: 2.51.4885.701 64-bit (October 2017)

I am attaching a screenshot of the error too. Any ideas?

Many thanks Michael

connector error 15oct17

migueesc123 commented 7 years ago

Hey! Thanks for trying out the connector. This is extremely weird! I'm unable to replicate the issue and to be even more precise, the DailyVideoStats function relies on that Videos function as well, so if the Videos function didn't work then the DailyVideoStats shouldn't have worked either.

Can you confirm that all the other functions are working and loading to your data model inside Power BI Desktop? I'd recommend that you check your WebApp.json file and that it looks almost exactly like this: {"web": {"client_id":"something.apps.googleusercontent.com", "project_id":"youtube-analytics-169304", "auth_uri":"https://accounts.google.com/o/oauth2/auth", "token_uri":"https://accounts.google.com/o/oauth2/token", "auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs", "client_secret":"Miguel", "redirect_uris":["https://oauth.powerbi.com/views/oauthredirect.html"]}}

michael534 commented 7 years ago

Thanks for getting back so fast on this. I verified my JSON exactly similar to yours. I have now resolved, and it is my error. I did not realize there are two levels of access on the youtube login I was using. I picked the team access which does not have full access rights. Now using the second of the two user options, which must deeper access to the analytics. Really appreciate you help on this!

migueesc123 commented 7 years ago

glad it's working for you now! Let me know what you think about the connector so far :)

michael534 commented 7 years ago

Miguel, your connector does a good job, allowing me to build my own dashboards rather than rely on the youtube analytics portal. My main reason for wanting this though is currently not possible with the connector either. Possibly Google deliberately guards this info: do the APIs offer a way to access the additional audience demographic data too? One really cool Power BI dashboard I look forward to creating one day is one that would allow me to see who is watching (and how far they watch) based on age, gender, education, role, salary etc.

Cheers, Michael

Edited by Miguel - Removed PII

migueesc123 commented 7 years ago

I haven't found a way to get the detailed info by age and gender. The API kept telling me that the combination wasn't accepted. I couldn't even use a videoID filter when trying to use the age or gender dimension. I'm a complete newbie when it comes to using the YouTube API, so perhaps there is a way that I just haven't found out just yet.

michael534 commented 7 years ago

I greatly appreciate what you have accomplished. Without you, I wouldn’t even be at this point! Thank you.

Interesting that they make access so difficult, when they offer so much else. I wonder if it gets any easier with the new Creator Studio currently in beta. Even with the current system, I think the data must be there - companies like this one http://www.demographicspro.com/ seem to have found a way to access it.

Best wishes to you.

And thanks Michael

*Edited by Miguel - reason: hiding PII

migueesc123 commented 7 years ago

I think that those companies are basically providing the same info that I get from my Power BI Template. Basically a combination of age/gender viewer percentage that can have multiple filters like:

but you can't mix it with metrics like views, ads, estimatedrevenue or similar so you can't really get the full picture. From the YouTube Creator Studio you are also unable to answer questions like:

If you somehow manage to find someone that does show that info, please let me know. I'd love to find out how to do that and get an answer to the previous questions.

michael534 commented 7 years ago

I think you are correct. They are demonstrating additional demographics analysis but this is on other platforms, not Youtube.

I am finding useful ways of working with your current connector. The cross correlation between charts means PowerBI provides a far richer and easier way to drill into the data.

Two issues arising as I use it

  1. Is there any way to set up a scheduled refresh? Presently all attempts to refresh the data from with PowerBI seem to fail. As a test I have created the simplest report I can, just a single table generated using fx daily subscribers, and just a 1 week date range. The data loads just fine on Power BI desktop, and publishes to PowerBI online. After that any attempt to do a refresh fails. Autorefresh fails with “You can't schedule refresh for this dataset because one or more sources currently don't support refresh.” And Manual refresh fails with “Unable to refresh the model (id=3621628) because it references an unsupported data source.”

  2. Using the fx functions your text notes that some field are optional. For example fx Daily Subscribers notes: “function (optional StartDate as a nullable date, optional EndDate as a nullable date) as table”. If I omit either parameter though, the function returns an error – e.g. Reason: Parameter Issue. Message: “EndDate can not be null.”. Detail EndDate = null

If these questions are becoming too much, please do say. I can work with what I have.

Many thanks Michael

From: Miguel Escobar [mailto:notifications@github.com] Sent: Thursday, October 19, 2017 7:27 AM To: migueesc123/YoutubeAnalytics YoutubeAnalytics@noreply.github.com Cc: Michael Emanuel michael@michaelemanuel.net; State change state_change@noreply.github.com Subject: Re: [migueesc123/YoutubeAnalytics] Error accessing Table: YoutubeAnalytics.Videos (#4)

I think that those companies are basically providing the same info that I get from my Power BI Template. Basically a combination of age/gender viewer percentage that can have multiple filters like:

but you can't mix it with metrics like views, ads, estimatedrevenue or similar so you can't really get the full picture. From the YouTube Creator Studio you are also unable to answer questions like:

If you somehow manage to find someone that does show that info, please let me know. I'd love to find out how to do that and get an answer to the previous questions.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHubhttps://github.com/migueesc123/YoutubeAnalytics/issues/4#issuecomment-337925250, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfTNHgOzKSS5lO9ZN3VP4oEPinD_fCUBks5st1w8gaJpZM4P51-E.

migueesc123 commented 7 years ago

More info about the Custom Connectors Roadmap and when they'll be able to refresh through the Data Gateway here.

For now, most people use 3rd party tools to refresh the file locally and then publish it to the cloud.

Sadly, those 2 parameters had to be added as optional, but they are actually required because you can't have a mix of required and optional parameters in a function. Furthermore, in the Custom Connector scenario, in order to bypass some by-design limitations, I had to put some parameters as optional in order to improve the discovery of these functions. So, these parameters are required (start date and end date), but in some cases the rest of the parameters are actually completely optional like the GETData functions

michael534 commented 7 years ago

Thank you! I guessed the answer may be the gateway. But when I looked up what I could find on it, it didn’t seem to be a supported function. Glad to see Microsoft is planning to add it.

Makes sense that the end date isn’t optional. It was just the wording around the explanation that confused me. I spent a little time in the advanced editor trying ways to implement a null field, finally concluding I couldn’t. 😊

Many thanks

Michael

From: Miguel Escobar [mailto:notifications@github.com] Sent: Saturday, October 21, 2017 6:52 AM To: migueesc123/YoutubeAnalytics YoutubeAnalytics@noreply.github.com Cc: Michael Emanuel michael@michaelemanuel.net; State change state_change@noreply.github.com Subject: Re: [migueesc123/YoutubeAnalytics] Error accessing Table: YoutubeAnalytics.Videos (#4)

More info about the Custom Connectors Roadmap and when they'll be able to refresh through the Data Gateway herehttps://github.com/Microsoft/DataConnectors.

For now, most people use 3rd party tools to refresh the file locally and then publish it to the cloud.

Sadly, those 2 parameters had to be added as optional, but they are actually required because you can't have a mix of required and optional parameters in a function. Furthermore, in the Custom Connector scenario, in order to bypass some by-design limitations, I had to put some parameters as optional in order to improve the discovery of these functions. So, these parameters are required (start date and end date), but in some cases the rest of the parameters are actually completely optional like the GETData functions

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHubhttps://github.com/migueesc123/YoutubeAnalytics/issues/4#issuecomment-338397796, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfTNHq8gWC1WXOq-_wvnCgQ3wkJgygVRks5sufb0gaJpZM4P51-E.

damnpat commented 6 years ago

Hello, I am trying to set up the custom data connector with Power BI and follow very carefully your detailed explanations. Thanks a lot for this!

When I try to get the data from Youtube, I got the following error message on the data preview.

> DataSource.Error: Web.Contents failed to get contents from 'https://www.googleapis.com/youtube/v3/search?part=snippet,id&order=date&maxResults=50&forMine=true&type=video' (400): Bad Request
> Details:
>     DataSourceKind=YoutubeAnalytics
>     DataSourcePath=YoutubeAnalytics
>     Url=https://www.googleapis.com/youtube/v3/search?part=snippet,id&order=date&maxResults=50&forMine=true&type=video

I am defined as a user of the channel with no restrictions, I was wondering if the problem came from this Also, I got my email adress as Channel name in the preview and I am not sure it is normal? How can I connect Power BI with this youtube account and not with my google account?

Thanks a lot for your help on this! Cheers,

Damien

migueesc123 commented 6 years ago

Hey @damnpat

Currently, the custom connector is created in a way where it reads the authenticated user and it grabs the main youtube channel of that authenticated user. I might add support for other scenarios in the future, but the only supported scenario at this point is where the owner of the channel is the one authenticated.

damnpat commented 6 years ago

Hello @migueesc123

Thanks a lot for your answer. Have you experimented the same kind of situation with a Content Manager account accessible to many different google accounts? Will it be a way to authenticate the content manager account instead of my own google account?

migueesc123 commented 6 years ago

Hey @damnpat

I believe the Youtube Analytics API let's you do it, but the custom connector that I created has a limited scope at this moment as mentioned in the previous reply. I haven't tested anything with a Content Manager as I wanted to keep things as simple as possible, and adding support for accessing data through a CM would require big changes to the whole code an UI of the Custom Connector. (mainly to input the channel_id or channel name that you're trying to connect to)