search

memen45 / SubMusic

Sync music and podcasts to your Garmin watch from your own SubSonic or Ampache server
GNU General Public License v3.0
122 stars 13 forks source link

Media Error Occured on fetching playlist #1

Closed tarioch closed 4 years ago

tarioch commented 4 years ago

I just installed the app today and when I'm trying to sync I'm getting the error "Media Error Occured" when it's trying to fetch the playlist.

My setup is

server address: http://myhost.com:44 username: myuser api key: mypassword

I'm running airsonic which works very well with all other subsonic clients: https://airsonic.github.io/

One thing that I found a bit weird is that on the config page, it shows "alpha 0.0.1" but as I installed today I think it should be 0.0.3 .

I'm using a vivoactive 4

memen45 commented 4 years ago

Thank you for trying the app and providing feedback!

As far as I know, it is not possible to use other ports than the default http and https ports. The "Media Error Occured" throws due to the ":"in the URL. If you can make your server available over http or https, it should start working. I have only tested with HTTPS (port 443) though, and I have seen issues with others trying to connect using HTTP, so let me know if http works too.

With regards to your second remark: you are completely right! I fixed it locally, but didn't push an update for it yet.

tarioch commented 4 years ago

Thanks for your quick feedback. I now tried with http://my.host.com (I only need the one with port when I'm outside my wlan and as the sync happens only in the wlan anyway it's good enough) and I'm still getting the media error.

tarioch commented 4 years ago

Is there any way to add some more logging/debugging to it to figure out what's failing? Also have you thought about wheter you want to keep the app closed source or making it open source?

memen45 commented 4 years ago

Are you able to get a response if you use "http://my.host.com/rest/ping" in the browser? The browser should show some XML response if the URL is working (right click -> view source). If that does work, "http://my.host.com" as URL should work as well and not throw an error,

If the error remains, there should be something wrong in the code. If you are willing to help debug, there should be a file called CIQ_LOG.YAML in /GARMIN/APPS/LOGS on the device. It should provide a cause of error and a stack trace that could help me solve it.

tarioch commented 4 years ago

ping request works, log has exception in it


---
Error: Unhandled Exception
Time: 2020-06-10T11:07:19Z
Part-Number: 006-B3225-00
Firmware-Version: '4.70'
Language-Code: eng
ConnectIQ-Version: 3.1.8
Store-Id: 600bd75f-6ccf-4ca5-bc7a-0a4fcfdcf794
Store-Version: 2
Filename: A6A85331
Appname: SubMusic
Stack: 
  - pc: 0x100018df
---
Error: Unhandled Exception
Time: 2020-06-10T11:07:19Z
Part-Number: 006-B3225-00
Firmware-Version: '4.70'
Language-Code: eng
ConnectIQ-Version: 3.1.8
Store-Id: 600bd75f-6ccf-4ca5-bc7a-0a4fcfdcf794
Store-Version: 2
Filename: A6A85331
Appname: SubMusic
Stack: 
  - pc: 0x100018df
tarioch commented 4 years ago

Response for the ping rest call (after adding required params), so it was

http://my.host.com/rest/ping?u=myuser&p=mypassword&c=foo&v=1.15

<subsonic-response xmlns="http://subsonic.org/restapi" status="ok" version="1.15.0"/>
memen45 commented 4 years ago

Hmm, unfortunately the stack trace is not too comprehensive. The URL seems fine though. I do see however that I missed the required parameters "c" and "v", as Nextcloud does not require them. If you leave the c and v parameters, do you get a subsonic-response - status="failed"?

I am not planning on keeping the app closed source at all. I wanted to clean up one part of the code before publishing as it is only functional, but not good code (still too much based on some example code). As soon as I have time to rewrite it, I will publish the code in this repository.

tarioch commented 4 years ago

I think that's it, it fails if c or v are missing. According to this http://www.subsonic.org/pages/api.jsp they are mandatory.

Thanks again for this app and your very fast responses.

memen45 commented 4 years ago

Thanks a lot for your feedback and help. Just now I uploaded version 0.0.4 to the app store. I hope this solves the issues and allows you to connect to your server.

Please let me know if it does, as I can then update the description and notify some other users about this.

tarioch commented 4 years ago

Thanks, I got further with this. Now I see my playlists. But unfortunately when I try to sync it, it fails again with the media error.


---
Error: Unexpected Type Error
Details: 'Failed invoking <symbol>'
Time: 2020-06-10T13:17:43Z
Part-Number: 006-B3225-00
Firmware-Version: '4.70'
Language-Code: eng
ConnectIQ-Version: 3.1.8
Store-Id: 600bd75f-6ccf-4ca5-bc7a-0a4fcfdcf794
Store-Version: 3
Filename: A6A85331
Appname: SubMusic
Stack: 
  - pc: 0x10001a0a
---
Error: Unexpected Type Error
Details: 'Failed invoking <symbol>'
Time: 2020-06-10T13:17:43Z
Part-Number: 006-B3225-00
Firmware-Version: '4.70'
Language-Code: eng
ConnectIQ-Version: 3.1.8
Store-Id: 600bd75f-6ccf-4ca5-bc7a-0a4fcfdcf794
Store-Version: 3
Filename: A6A85331
Appname: SubMusic
Stack: 
  - pc: 0x10001a0a
---
Error: Unexpected Type Error
Details: 'Failed invoking <symbol>'
Time: 2020-06-10T13:18:07Z
Part-Number: 006-B3225-00
Firmware-Version: '4.70'
Language-Code: eng
ConnectIQ-Version: 3.1.8
Store-Id: 600bd75f-6ccf-4ca5-bc7a-0a4fcfdcf794
Store-Version: 3
Filename: A6A85331
Appname: SubMusic
Stack: 
  - pc: 0x10001a0a
---
Error: Unexpected Type Error
Details: 'Failed invoking <symbol>'
Time: 2020-06-10T13:18:07Z
Part-Number: 006-B3225-00
Firmware-Version: '4.70'
Language-Code: eng
ConnectIQ-Version: 3.1.8
Store-Id: 600bd75f-6ccf-4ca5-bc7a-0a4fcfdcf794
Store-Version: 3
Filename: A6A85331
Appname: SubMusic
Stack: 
  - pc: 0x10001a0a
memen45 commented 4 years ago

Great! That was the hardest part. I think I figured out the subsequent problem. I fixed it and pushed it to the store (version 0.0.5).

Thanks a lot!

tarioch commented 4 years ago

Unfortunately still fails with 0.0.6


---
Error: Unexpected Type Error
Details: 'Failed invoking <symbol>'
Time: 2020-06-10T14:34:51Z
Part-Number: 006-B3225-00
Firmware-Version: '4.70'
Language-Code: eng
ConnectIQ-Version: 3.1.8
Store-Id: 600bd75f-6ccf-4ca5-bc7a-0a4fcfdcf794
Store-Version: 5
Filename: A6A85331
Appname: SubMusic
Stack: 
  - pc: 0x10001a3b
---
Error: Unexpected Type Error
Details: 'Failed invoking <symbol>'
Time: 2020-06-10T14:34:51Z
Part-Number: 006-B3225-00
Firmware-Version: '4.70'
Language-Code: eng
ConnectIQ-Version: 3.1.8
Store-Id: 600bd75f-6ccf-4ca5-bc7a-0a4fcfdcf794
Store-Version: 5
Filename: A6A85331
Appname: SubMusic
Stack: 
  - pc: 0x10001a3b
memen45 commented 4 years ago

Sorry, I am unable to reproduce the error.

tarioch commented 4 years ago

Sure, sent you a message over the garmin iq page.

memen45 commented 4 years ago

Thanks! I get an SECURE_CONNECTION_REQUIRED response. In the simulator I can disable HTTPS requirements, which allows it to work. However, I do not think that this is possible on the device.

I will improve the error reporting in the following version (0.0.7). However, the problem resides with Connect IQ and the choice to only support HTTPS connections, unfortunately. This would require setting up a Let's Encrypt service on your machine (which is free and quite simple fortunately).

tarioch commented 4 years ago

Thanks a lot, a bit weird that the other requests work on http. Anyway I switched to https and unfortunately still the same problem, sent you a mail with credentials.

tarioch commented 4 years ago 
Error: Unexpected Type Error
Details: 'Failed invoking <symbol>'
Time: 2020-06-10T17:40:34Z
Part-Number: 006-B3225-00
Firmware-Version: '4.70'
Language-Code: eng
ConnectIQ-Version: 3.1.8
Store-Id: 600bd75f-6ccf-4ca5-bc7a-0a4fcfdcf794
Store-Version: 6
Filename: 8ED2EFC8
Appname: SubMusic
Stack: 
  - pc: 0x10001a6a
memen45 commented 4 years ago

Testing against your server I get a

NETWORK_RESPONSE_TOO_LARGE = -402 | Serialized response was too large.

I have checked the Subsonic API, but I cannot find a way to limit the number of results per request. On sync, it tries to retrieve the playlist in a single request. So the response will be an array of all songs including all metadata. I do not know the exact maximum size of a response.

Do you know if Subsonic supports paged responses?

memen45 commented 4 years ago

Unfortunately still fails with 0.0.6

Was the fetching playlists working with this version already?

tarioch commented 4 years ago

Fetching of the list of playlists works fine (I see my playlists and can select the right one, but it fails when I then try so sync one of the playlists).

I now created a simpler playlist with just one song in it, but the same problem happens.

According to this: http://www.subsonic.org/pages/api.jsp several of the apis support pagination, but I don't think the playlist one does.

tarioch commented 4 years ago

sorry, not sure what happened the last time, now it works with synchronizing the single entry playlist

memen45 commented 4 years ago

This is remarkable. I thought I could reproduce the last error, and it still does not work for me:

INVALID_HTTP_BODY_IN_NETWORK_RESPONSE = -400

When I load the download URL in the browser: On your server:

Whereas on my server (Nextcloud + music app):

I am not sure how to proceed, since this seems a server side implementation difference. Does the Subsonic API even specify this response Content-Type?

tarioch commented 4 years ago

I don't think it's specified. What I noticed is that /download gives Content-Type: application/x-download but if you use /stream it gives Content-Type: audio/mpeg

I think stream might be the better one anyway as this avoids having to deal with unsupported audio formats.

memen45 commented 4 years ago

Yes I just discovered that too. I assumed stream method was for retrieving parts of songs, but it isnt. I will test with the /stream method and update if it works correctly!

Will be version 0.0.8. Hoping that this finally solves it!

tarioch commented 4 years ago

Saw that you pushed 0.0.8, now I'm getting a 402 even on only selecting a 1 entry list

tarioch commented 4 years ago

Uninstalled app and reinstalled. A couple more interesting points

memen45 commented 4 years ago

Haha, you found the exact reason I did not publish the code yet. The keeping track of playlists requires a rewrite to make it more robust.

I have fixed the former bug, allowing to remove playlists even when they are not yet completely synced. However, the UNABLE_TO_PROCESS_MEDIA is weird. The song element "suffix" is used to determine the media encoding, so if that element is not one of the four supported suffixes, it might throw such an error. Did you verify the suffix is one of ADTS, MP3, M4A or WAV.

tarioch commented 4 years ago

I might have an idea what's going on. The file has m4a, but as you're accessing it with the stream api, it's actually converted to mp3.

I think I also found another issue, it has problems with file names with special characters (e.g. &). I think now that you use the stream api, it would make more sense to not use the original filename (which I'm guessing you're doing right now) and instead use maybe the id and add .mp3. That way it's save from special characters and should solve the encoding issue as well. If needed you could even set the parameter format=mp3 on the stream call to be more explicit.

memen45 commented 4 years ago

The app is accessing by id in either case, so titles should not be the issue. If stream always returns an mp3, then I have to adapt the code indeed, however I will have to test this on Nextcloud as well, as otherwise it may break.

tarioch commented 4 years ago

Makes sense, was just guessing here as I see issues with files that have a path (not title) like Klaus Badelt &amp; Hans Zimmer/Pirates of the Caribbean_ Soundtrack Treasures Collection/1-01 Klaus Badelt - Fog Bound.mp3

But if always the id (which is numeric) is used, then this shouldn't be an issue.

memen45 commented 4 years ago

I am just testing with Nextcloud and it returns Content-Type: audio/mp4, I can reproduce your -1005 error. Also in the browser this does not work ("No video with supported format and MIME type found"). The format=mp3 parameter does give the same Content-Type. The best I can do here is just skip over a song if downloading fails. That way the rest of the sync will still work, instead of cancelling as it is now.

Does adding the format=mp3 return a Content-Type: audio/mpeg for you? If so, I might add that parameter to ensure mp3 is tried if the transcoding is supported by the server.

tarioch commented 4 years ago

On my server if I use the stream url (independent of if I aadd the format=mp3 param or not) I'm getting Content-Type: audio/mpeg and the file is converted. If I use download, I get a different file than if I use stream, the stream gives me an mp3 converted file.

tarioch commented 4 years ago

After another couple of reinstalls and avoiding different playlists and files which are problematic (as you said, path does actually not make any troubles, I guess that was just caused by earlier issues). I was now able to sync a playlist with 27 songs in it :) Adding one more makes it fail. Are your playlists so short or does the nextcloud impl add less metadata?

Anyway, I think this is great progress (from not able to connect, to able to sync 27 songs in a playlist). Thank you again very much.

Do you see any way to work around the issue with the response size of the getPlaylist?

memen45 commented 4 years ago

Indeed I only tested with ~10 songs per playlist. Unfortunately the response limit is very low, somewhere around 16kB or 32kB. There is no simple way around this currently.

I have made an update again:

Will upload it shortly (Version 0.0.9).

tarioch commented 4 years ago

Sound's great (do you never sleep? ;)). Is there a way to "stream" the response? As there isn't too much info needed from that response if there would be a way to work on the response stream, that might solve the issue.

tarioch commented 4 years ago

Started a thread on the forum to see if anyone has an idea, if we could stream it, I think it would be possible to handle it https://forums.garmin.com/developer/connect-iq/f/discussion/229268/way-to-receive-web-request-response-as-a-stream

memen45 commented 4 years ago

I don't think that is possible. Also other options like bluetooth etc are not possible due to the same memory limitation.

The most feasible option would be to add support for Ampache API. Looks much better and allows paged requests on almost all methods. Will look into this when I have more time.

Thanks a lot for the efforts on investigating! I will close this issue as the initial problem is fixed.

tarioch commented 4 years ago

Can you tell me the necessary stuff that you need on that getPlaylist call? I'm thinking about creating a small proxy that will do the real request and then strip out all the unnecessary stuff, that way it should reduce the response quite a bit, so I can also sync larger playlists.

memen45 commented 4 years ago

The metadata should be delivered on downloading the song itself. Up to now, only the playlist name is shown to the user. Therefore for the getPlaylist request, the only thing required of each of the songs are the "id" elements. If you strip all the other elements from the song objects, I think it will take a while before you hit the limit again!

tarioch commented 4 years ago

If anyone is interested, I started creating it https://github.com/tarioch/subsonic-api-proxy still testing a bit more, I also want to something to have all the files as mp3, as in my setup the stream api will always return mp3 independent of the filename or metadata of the file, because of the on the fly conversion.