Problem with folder name in non-western characters

wiserain commented 5 years ago

I have downloaded one youtube video using the exactly same format as in Readme.md via youtube-dl, which includes a Korean channel name and Korean video name, like below.

/volume1/video/yotube/디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw]/NA - 발라드가수의비극 feat.이해리 - [zwKSZczv3LU].mp4

The log said

2019-06-03 10:56:49,830 (7f2c827fc700) :  CRITICAL (model:192) - Exception serializing TV_Show with guid 'com.plexapp.agents.youtube://youtube|UCCaYq4ZNPHP9v_0vL2xjrWw|디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw]?lang=xn' (most recent call last):
  File "/usr/lib/plexmediaserver/Resources/Plug-ins-005053bb5/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/modelling/model.py", line 190, in _write
    self._serialize(os.path.join(self._storage_path, subdir))
  File "/usr/lib/plexmediaserver/Resources/Plug-ins-005053bb5/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/modelling/model.py", line 215, in _serialize
    el.set('id', self._id)
  File "lxml.etree.pyx", line 699, in lxml.etree._Element.set (lxml-2.3/src/lxml/lxml.etree.c:34531)
  File "apihelpers.pxi", line 563, in lxml.etree._setAttributeValue (lxml-2.3/src/lxml/lxml.etree.c:15781)
  File "apihelpers.pxi", line 1366, in lxml.etree._utf8 (lxml-2.3/src/lxml/lxml.etree.c:22211)
ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

I found a related issue reporting a limitation in a folder name, https://github.com/ZeroQI/YouTube-Agent.bundle/issues/14 , however, I don't see it is closely related because in my case it doen't have any special characters except non-western ones.

After I changed the folder name from 디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw] to test [UCCaYq4ZNPHP9v_0vL2xjrWw] by replacing channel name with "test", it successfully updated metainfo.

Is this expected not to work with CJK characters? or am I missing something else?

Thanks for your work.

ZeroQI commented 5 years ago

All code file involved are not my agent ones but Plex ones. At least I would need the last output of my agent to know where it is failing, as there is a chance it is OS specific...

wiserain commented 5 years ago

Thanks for your feedback. Below is the full log.

2019-06-04 04:57:01,781 (7faa2e7fc700) :  DEBUG (runtime:717) - Handling request GET /:/plugins/com.plexapp.agents.youtube/messaging/function/X0FnZW50S2l0OlVwZGF0ZU1ldGFkYXRh/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKMTAKczIKeG5zNApsYW5nYjFzNQpmb3JjZWIwczgKcGVyaW9kaWNzNQoxMzk1NHM0CmRiaWRpMApzNwp2ZXJzaW9ubnMxMApwYXJlbnRHVUlEbnM4CnBhcmVudElEczcKVFZfU2hvd3MxMAptZWRpYV90eXBlczExMgpjb20ucGxleGFwcC5hZ2VudHMueW91dHViZTovL3lvdXR1YmV8VUNDYVlxNFpOUEhQOXZfMHZMMnhqcld3fOuUlOyXkOydtOyXlOyngCBbVUNDYVlxNFpOUEhQOXZfMHZMMnhqcld3XT9sYW5nPXhuczQKZ3VpZHM3NQp5b3V0dWJlfFVDQ2FZcTRaTlBIUDl2XzB2TDJ4anJXd3zrlJTsl5DsnbTsl5Tsp4AgW1VDQ2FZcTRaTlBIUDl2XzB2TDJ4anJXd11zMgppZHIwCg__
2019-06-04 04:57:01,782 (7faa2e7fc700) :  DEBUG (runtime:49) - Received packed state data (80 bytes)
2019-06-04 04:57:01,785 (7faa2e7fc700) :  DEBUG (runtime:814) - Found route matching /:/plugins/com.plexapp.agents.youtube/messaging/function/X0FnZW50S2l0OlVwZGF0ZU1ldGFkYXRh/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKMTAKczIKeG5zNApsYW5nYjFzNQpmb3JjZWIwczgKcGVyaW9kaWNzNQoxMzk1NHM0CmRiaWRpMApzNwp2ZXJzaW9ubnMxMApwYXJlbnRHVUlEbnM4CnBhcmVudElEczcKVFZfU2hvd3MxMAptZWRpYV90eXBlczExMgpjb20ucGxleGFwcC5hZ2VudHMueW91dHViZTovL3lvdXR1YmV8VUNDYVlxNFpOUEhQOXZfMHZMMnhqcld3fOuUlOyXkOydtOyXlOyngCBbVUNDYVlxNFpOUEhQOXZfMHZMMnhqcld3XT9sYW5nPXhuczQKZ3VpZHM3NQp5b3V0dWJlfFVDQ2FZcTRaTlBIUDl2XzB2TDJ4anJXd3zrlJTsl5DsnbTsl5Tsp4AgW1VDQ2FZcTRaTlBIUDl2XzB2TDJ4anJXd11zMgppZHIwCg__
2019-06-04 04:57:01,785 (7faa2e7fc700) :  DEBUG (model:32) - Loading model with GUID com.plexapp.agents.youtube://youtube|UCCaYq4ZNPHP9v_0vL2xjrWw|디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw]?lang=xn
2019-06-04 04:57:01,786 (7faa2e7fc700) :  DEBUG (model:234) - Deserializing from /config/Library/Application Support/Plex Media Server/Metadata/TV Shows/3/35f3ce85c87bdc18c6aa31391b1c742f18e9abb.bundle/Contents/com.plexapp.agents.youtube/Info.xml
2019-06-04 04:57:01,786 (7faa2e7fc700) :  ERROR (model:236) - Unable to deserialize object at /config/Library/Application Support/Plex Media Server/Metadata/TV Shows/3/35f3ce85c87bdc18c6aa31391b1c742f18e9abb.bundle/Contents/com.plexapp.agents.youtube
2019-06-04 04:57:01,786 (7faa2e7fc700) :  DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/library/metadata/13954/tree'
2019-06-04 04:57:01,791 (7faa2e7fc700) :  INFO (__init__:469) - =============================================================================================================================================================
2019-06-04 04:57:01,792 (7faa2e7fc700) :  INFO (__init__:469) - update() - guid: UCCaYq4ZNPHP9v_0vL2xjrWw, dir: 디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw], metadata.id: youtube|UCCaYq4ZNPHP9v_0vL2xjrWw|디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw]
2019-06-04 04:57:01,792 (7faa2e7fc700) :  INFO (__init__:60) - /volume1/video/yotube/디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw]/NA - 발라드가수의비극 feat.이해리 - [zwKSZczv3LU].mp4
2019-06-04 04:57:01,792 (7faa2e7fc700) :  INFO (__init__:72) - [!] Library access denied
2019-06-04 04:57:01,792 (7faa2e7fc700) :  INFO (__init__:75) - [!] ASS root scanner file present: "/config/Library/Application Support/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/_Logs/_root_.scanner.log"
2019-06-04 04:57:01,792 (7faa2e7fc700) :  INFO (__init__:225) - [ ] dir:        "/volume1/video/yotube/디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw]"
2019-06-04 04:57:01,793 (7faa2e7fc700) :  INFO (__init__:226) - [ ] library:    ""
2019-06-04 04:57:01,793 (7faa2e7fc700) :  INFO (__init__:227) - [ ] root:       ""
2019-06-04 04:57:01,793 (7faa2e7fc700) :  INFO (__init__:228) - [ ] path:       "_unknown_folder"
2019-06-04 04:57:01,796 (7faa2e7fc700) :  INFO (__init__:349) - after
2019-06-04 04:57:01,798 (7faa2d7fa700) :  DEBUG (runtime:717) - Handling request GET /:/plugins/com.plexapp.agents.youtube/messaging/function/X0FnZW50S2l0OlVwZGF0ZU1ldGFkYXRh/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKMTAKczIKeG5zNApsYW5nYjFzNQpmb3JjZWIwczgKcGVyaW9kaWNzNQoxMzk1OHM0CmRiaWRpMApzNwp2ZXJzaW9ubnMxMApwYXJlbnRHVUlEbnM4CnBhcmVudElEczcKVFZfU2hvd3MxMAptZWRpYV90eXBlczEwOQpjb20ucGxleGFwcC5hZ2VudHMueW91dHViZTovL3lvdXR1YmV8VUNnc2ZmUzdNZktMNllVM3JfVTNFLWFBfOyKueyasOyVhOu5oCBbVUNnc2ZmUzdNZktMNllVM3JfVTNFLWFBXT9sYW5nPXhuczQKZ3VpZHM3Mgp5b3V0dWJlfFVDZ3NmZlM3TWZLTDZZVTNyX1UzRS1hQXzsirnsmrDslYTruaAgW1VDZ3NmZlM3TWZLTDZZVTNyX1UzRS1hQV1zMgppZHIwCg__
2019-06-04 04:57:01,799 (7faa2d7fa700) :  DEBUG (runtime:49) - Received packed state data (80 bytes)
2019-06-04 04:57:01,800 (7faa2d7fa700) :  DEBUG (runtime:814) - Found route matching /:/plugins/com.plexapp.agents.youtube/messaging/function/X0FnZW50S2l0OlVwZGF0ZU1ldGFkYXRh/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKMTAKczIKeG5zNApsYW5nYjFzNQpmb3JjZWIwczgKcGVyaW9kaWNzNQoxMzk1OHM0CmRiaWRpMApzNwp2ZXJzaW9ubnMxMApwYXJlbnRHVUlEbnM4CnBhcmVudElEczcKVFZfU2hvd3MxMAptZWRpYV90eXBlczEwOQpjb20ucGxleGFwcC5hZ2VudHMueW91dHViZTovL3lvdXR1YmV8VUNnc2ZmUzdNZktMNllVM3JfVTNFLWFBfOyKueyasOyVhOu5oCBbVUNnc2ZmUzdNZktMNllVM3JfVTNFLWFBXT9sYW5nPXhuczQKZ3VpZHM3Mgp5b3V0dWJlfFVDZ3NmZlM3TWZLTDZZVTNyX1UzRS1hQXzsirnsmrDslYTruaAgW1VDZ3NmZlM3TWZLTDZZVTNyX1UzRS1hQV1zMgppZHIwCg__
2019-06-04 04:57:01,802 (7faa2d7fa700) :  DEBUG (model:32) - Loading model with GUID com.plexapp.agents.youtube://youtube|UCgsffS7MfKL6YU3r_U3E-aA|승우아빠 [UCgsffS7MfKL6YU3r_U3E-aA]?lang=xn
2019-06-04 04:57:01,803 (7faa2d7fa700) :  DEBUG (model:234) - Deserializing from /config/Library/Application Support/Plex Media Server/Metadata/TV Shows/d/80ed034afaed8e0052e38f5062152177c2a371c.bundle/Contents/com.plexapp.agents.youtube/Info.xml
2019-06-04 04:57:01,803 (7faa2d7fa700) :  ERROR (model:236) - Unable to deserialize object at /config/Library/Application Support/Plex Media Server/Metadata/TV Shows/d/80ed034afaed8e0052e38f5062152177c2a371c.bundle/Contents/com.plexapp.agents.youtube
2019-06-04 04:57:01,804 (7faa2d7fa700) :  DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/library/metadata/13958/tree'
2019-06-04 04:57:01,806 (7faa2e7fc700) :  DEBUG (networking:161) - Fetching 'https://www.googleapis.com/youtube/v3/channels?part=snippet%2CcontentDetails%2Cstatistics%2CbrandingSettings&id=UCCaYq4ZNPHP9v_0vL2xjrWw&key=AIzaSyC2q8yjciNdlYRNdvwbb7NEcDxBkv1Cass' from the HTTP cache
2019-06-04 04:57:01,808 (7faa2e7fc700) :  INFO (__init__:360) - [?] json_channel_details: ['kind', 'statistics', 'contentDetails', 'brandingSettings', 'snippet', 'etag', 'id']
2019-06-04 04:57:01,808 (7faa2e7fc700) :  INFO (__init__:361) - [ ] title:       "디에이엔지"
2019-06-04 04:57:01,808 (7faa2e7fc700) :  INFO (__init__:370) - [ ] summary:     ""
2019-06-04 04:57:01,813 (7faa2e7fc700) :  INFO (__init__:381) - [ ] role:        디에이엔지
2019-06-04 04:57:01,813 (7faa2e7fc700) :  INFO (__init__:385) - [ ] art:       
2019-06-04 04:57:01,813 (7faa2e7fc700) :  INFO (__init__:388) - [ ] banners:   
2019-06-04 04:57:01,814 (7faa2e7fc700) :  INFO (__init__:391) - [X] posters:   https://yt3.ggpht.com/a/AGF-l79cqf8W2ygOaGMFDjNGV9FAAIDNO0AnJubJLg=s240-mo-c-c0xffffffff-rj-k-no
2019-06-04 04:57:01,816 (7faa2d7fa700) :  INFO (__init__:469) - =============================================================================================================================================================
2019-06-04 04:57:01,816 (7faa2d7fa700) :  INFO (__init__:469) - update() - guid: UCgsffS7MfKL6YU3r_U3E-aA, dir: 승우아빠 [UCgsffS7MfKL6YU3r_U3E-aA], metadata.id: youtube|UCgsffS7MfKL6YU3r_U3E-aA|승우아빠 [UCgsffS7MfKL6YU3r_U3E-aA]
2019-06-04 04:57:01,816 (7faa2e7fc700) :  DEBUG (networking:166) - Requesting 'https://yt3.ggpht.com/a/AGF-l79cqf8W2ygOaGMFDjNGV9FAAIDNO0AnJubJLg=s240-mo-c-c0xffffffff-rj-k-no'
2019-06-04 04:57:01,817 (7faa2d7fa700) :  INFO (__init__:60) - /volume1/video/yotube/승우아빠 [UCgsffS7MfKL6YU3r_U3E-aA]/NA - 항정살은 튀겨먹을때 가장 맛있다 - [2zyRda-LBkE].mp4
2019-06-04 04:57:01,818 (7faa2d7fa700) :  INFO (__init__:72) - [!] Library access denied
2019-06-04 04:57:01,818 (7faa2d7fa700) :  INFO (__init__:75) - [!] ASS root scanner file present: "/config/Library/Application Support/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/_Logs/_root_.scanner.log"
2019-06-04 04:57:01,818 (7faa2d7fa700) :  INFO (__init__:225) - [ ] dir:        "/volume1/video/yotube/승우아빠 [UCgsffS7MfKL6YU3r_U3E-aA]"
2019-06-04 04:57:01,819 (7faa2d7fa700) :  INFO (__init__:226) - [ ] library:    ""
2019-06-04 04:57:01,819 (7faa2d7fa700) :  INFO (__init__:227) - [ ] root:       ""
2019-06-04 04:57:01,819 (7faa2d7fa700) :  INFO (__init__:228) - [ ] path:       "_unknown_folder"
2019-06-04 04:57:01,824 (7faa2d7fa700) :  INFO (__init__:349) - after
2019-06-04 04:57:01,825 (7faa2d7fa700) :  DEBUG (networking:161) - Fetching 'https://www.googleapis.com/youtube/v3/channels?part=snippet%2CcontentDetails%2Cstatistics%2CbrandingSettings&id=UCgsffS7MfKL6YU3r_U3E-aA&key=AIzaSyC2q8yjciNdlYRNdvwbb7NEcDxBkv1Cass' from the HTTP cache
2019-06-04 04:57:01,827 (7faa2d7fa700) :  INFO (__init__:360) - [?] json_channel_details: ['kind', 'statistics', 'contentDetails', 'brandingSettings', 'snippet', 'etag', 'id']
2019-06-04 04:57:01,827 (7faa2d7fa700) :  INFO (__init__:361) - [ ] title:       "승우아빠"
2019-06-04 04:57:01,828 (7faa2d7fa700) :  INFO (__init__:372) - [ ] country: KR
2019-06-04 04:57:01,833 (7faa2d7fa700) :  INFO (__init__:381) - [ ] role:        승우아빠
2019-06-04 04:57:01,833 (7faa2d7fa700) :  INFO (__init__:384) - [X] art:       https://yt3.ggpht.com/uQeMrzE91wD6vBX49dJh8yYkN7AdhHRk-fg4dM-0Y1rcUWQEfpATECsyBdYbJhdoutEk1Wxvhw=w854-fcrop64=1,00000000ffffffff-nd-c0xffffffff-rj-k-no
2019-06-04 04:57:01,834 (7faa2d7fa700) :  DEBUG (networking:166) - Requesting 'https://yt3.ggpht.com/uQeMrzE91wD6vBX49dJh8yYkN7AdhHRk-fg4dM-0Y1rcUWQEfpATECsyBdYbJhdoutEk1Wxvhw=w854-fcrop64=1,00000000ffffffff-nd-c0xffffffff-rj-k-no'
2019-06-04 04:57:02,063 (7faa2e7fc700) :  DEBUG (networking:198) - Not caching 'https://yt3.ggpht.com/a/AGF-l79cqf8W2ygOaGMFDjNGV9FAAIDNO0AnJubJLg=s240-mo-c-c0xffffffff-rj-k-no' (content type 'image/jpeg' not cacheable in Agent plug-ins)
2019-06-04 04:57:02,064 (7faa2e7fc700) :  INFO (__init__:404) - =============================================================================================================================================================
2019-06-04 04:57:02,064 (7faa2e7fc700) :  INFO (__init__:405) - Season: 2019
2019-06-04 04:57:02,064 (7faa2e7fc700) :  INFO (__init__:413) - metadata.seasons[2019].episodes[  1] "NA - 발라드가수의비극 feat.이해리 - [zwKSZczv3LU].mp4"
2019-06-04 04:57:02,065 (7faa2e7fc700) :  INFO (__init__:436) - # videoId [zwKSZczv3LU] not in Playlist/channel item list so loading video_details
2019-06-04 04:57:02,066 (7faa2e7fc700) :  DEBUG (networking:161) - Fetching 'https://www.googleapis.com/youtube/v3/videos?part=snippet,contentDetails,statistics&id=zwKSZczv3LU&key=AIzaSyC2q8yjciNdlYRNdvwbb7NEcDxBkv1Cass' from the HTTP cache
2019-06-04 04:57:02,067 (7faa2e7fc700) :  DEBUG (networking:166) - Requesting 'https://i.ytimg.com/vi/zwKSZczv3LU/sddefault.jpg'
2019-06-04 04:57:02,089 (7faa2d7fa700) :  DEBUG (networking:198) - Not caching 'https://yt3.ggpht.com/uQeMrzE91wD6vBX49dJh8yYkN7AdhHRk-fg4dM-0Y1rcUWQEfpATECsyBdYbJhdoutEk1Wxvhw=w854-fcrop64=1,00000000ffffffff-nd-c0xffffffff-rj-k-no' (content type 'image/jpeg' not cacheable in Agent plug-ins)
2019-06-04 04:57:02,089 (7faa2d7fa700) :  INFO (__init__:387) - [X] banners:   https://yt3.ggpht.com/uQeMrzE91wD6vBX49dJh8yYkN7AdhHRk-fg4dM-0Y1rcUWQEfpATECsyBdYbJhdoutEk1Wxvhw=w854-fcrop64=1,00000000ffffffff-nd-c0xffffffff-rj-k-no
2019-06-04 04:57:02,090 (7faa2d7fa700) :  DEBUG (networking:166) - Requesting 'https://yt3.ggpht.com/uQeMrzE91wD6vBX49dJh8yYkN7AdhHRk-fg4dM-0Y1rcUWQEfpATECsyBdYbJhdoutEk1Wxvhw=w854-fcrop64=1,00000000ffffffff-nd-c0xffffffff-rj-k-no'
2019-06-04 04:57:02,304 (7faa2e7fc700) :  DEBUG (networking:198) - Not caching 'https://i.ytimg.com/vi/zwKSZczv3LU/sddefault.jpg' (content type 'image/jpeg' not cacheable in Agent plug-ins)
2019-06-04 04:57:02,304 (7faa2e7fc700) :  INFO (__init__:447) - [ ] title:    "발라드가수의비극 feat.이해리"
2019-06-04 04:57:02,304 (7faa2e7fc700) :  INFO (__init__:448) - [ ] summary:  "160923 부여백제문화재 다비치"
2019-06-04 04:57:02,304 (7faa2e7fc700) :  INFO (__init__:449) - [ ] date:     "2016-09-24T04:58:43.000Z"
2019-06-04 04:57:02,305 (7faa2e7fc700) :  INFO (__init__:451) - [ ] rating:   "9.0"
2019-06-04 04:57:02,305 (7faa2e7fc700) :  INFO (__init__:452) - [ ] thumbs:   "https://i.ytimg.com/vi/zwKSZczv3LU/sddefault.jpg"
2019-06-04 04:57:02,305 (7faa2e7fc700) :  INFO (__init__:454) - [ ] duration: "PT41S"->"None"
2019-06-04 04:57:02,305 (7faa2e7fc700) :  INFO (__init__:459) - [ ] director: "디에이엔지"
2019-06-04 04:57:02,305 (7faa2e7fc700) :  INFO (__init__:465) - [ ] posters: https://i.ytimg.com/vi/zwKSZczv3LU/sddefault.jpg
2019-06-04 04:57:02,305 (7faa2e7fc700) :  INFO (__init__:467) - [ ] genres:   "[]"
2019-06-04 04:57:02,306 (7faa2e7fc700) :  INFO (__init__:473) - =============================================================================================================================================================
2019-06-04 04:57:02,306 (7faa2e7fc700) :  CRITICAL (model:192) - Exception serializing TV_Show with guid 'com.plexapp.agents.youtube://youtube|UCCaYq4ZNPHP9v_0vL2xjrWw|디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw]?lang=xn' (most recent call last):
  File "/usr/lib/plexmediaserver/Resources/Plug-ins-005053bb5/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/modelling/model.py", line 190, in _write
    self._serialize(os.path.join(self._storage_path, subdir))
  File "/usr/lib/plexmediaserver/Resources/Plug-ins-005053bb5/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/modelling/model.py", line 215, in _serialize
    el.set('id', self._id)
  File "lxml.etree.pyx", line 699, in lxml.etree._Element.set (lxml-2.3/src/lxml/lxml.etree.c:34531)
  File "apihelpers.pxi", line 563, in lxml.etree._setAttributeValue (lxml-2.3/src/lxml/lxml.etree.c:15781)
  File "apihelpers.pxi", line 1366, in lxml.etree._utf8 (lxml-2.3/src/lxml/lxml.etree.c:22211)
ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

2019-06-04 04:57:02,307 (7faa2e7fc700) :  DEBUG (runtime:88) - Sending packed state data (107 bytes)
2019-06-04 04:57:02,307 (7faa2e7fc700) :  DEBUG (runtime:924) - Response: [200] str, 16 bytes
2019-06-04 04:57:02,339 (7faa2d7fa700) :  DEBUG (networking:198) - Not caching 'https://yt3.ggpht.com/uQeMrzE91wD6vBX49dJh8yYkN7AdhHRk-fg4dM-0Y1rcUWQEfpATECsyBdYbJhdoutEk1Wxvhw=w854-fcrop64=1,00000000ffffffff-nd-c0xffffffff-rj-k-no' (content type 'image/jpeg' not cacheable in Agent plug-ins)
2019-06-04 04:57:02,339 (7faa2d7fa700) :  INFO (__init__:391) - [X] posters:   https://yt3.ggpht.com/a/AGF-l7-Rp7vbkppdgdeMW_kWCpqk9vkB5NCUzs7NWw=s240-mo-c-c0xffffffff-rj-k-no
2019-06-04 04:57:02,340 (7faa2d7fa700) :  DEBUG (networking:166) - Requesting 'https://yt3.ggpht.com/a/AGF-l7-Rp7vbkppdgdeMW_kWCpqk9vkB5NCUzs7NWw=s240-mo-c-c0xffffffff-rj-k-no'
2019-06-04 04:57:02,567 (7faa2d7fa700) :  DEBUG (networking:198) - Not caching 'https://yt3.ggpht.com/a/AGF-l7-Rp7vbkppdgdeMW_kWCpqk9vkB5NCUzs7NWw=s240-mo-c-c0xffffffff-rj-k-no' (content type 'image/jpeg' not cacheable in Agent plug-ins)
2019-06-04 04:57:02,568 (7faa2d7fa700) :  INFO (__init__:404) - =============================================================================================================================================================
2019-06-04 04:57:02,568 (7faa2d7fa700) :  INFO (__init__:405) - Season: 2019
2019-06-04 04:57:02,568 (7faa2d7fa700) :  INFO (__init__:413) - metadata.seasons[2019].episodes[  1] "NA - 항정살은 튀겨먹을때 가장 맛있다 - [2zyRda-LBkE].mp4"
2019-06-04 04:57:02,568 (7faa2d7fa700) :  INFO (__init__:436) - # videoId [2zyRda-LBkE] not in Playlist/channel item list so loading video_details
2019-06-04 04:57:02,570 (7faa2d7fa700) :  DEBUG (networking:161) - Fetching 'https://www.googleapis.com/youtube/v3/videos?part=snippet,contentDetails,statistics&id=2zyRda-LBkE&key=AIzaSyC2q8yjciNdlYRNdvwbb7NEcDxBkv1Cass' from the HTTP cache
2019-06-04 04:57:02,572 (7faa2d7fa700) :  DEBUG (networking:166) - Requesting 'https://i.ytimg.com/vi/2zyRda-LBkE/sddefault.jpg'
2019-06-04 04:57:02,874 (7faa2d7fa700) :  DEBUG (networking:198) - Not caching 'https://i.ytimg.com/vi/2zyRda-LBkE/sddefault.jpg' (content type 'image/jpeg' not cacheable in Agent plug-ins)
2019-06-04 04:57:02,874 (7faa2d7fa700) :  INFO (__init__:447) - [ ] title:    "항정살은 튀겨먹을때 가장 맛있다"
2019-06-04 04:57:02,875 (7faa2d7fa700) :  INFO (__init__:448) - [ ] summary:  "#승우아빠 #남편세끼 #항정살 #닭강정? #탕수육?. korean food easy recipe. . 오늘은 항정살로 강정을 만들어 보았습니다. 양념치킨 소스를 만들어서 멋진 맛동산을 만들어 보았습니다. 탕수육이나 닭강정, 양념치킨 같은것은 너무 식상하잖아요. 항정살 강정 한번 만들어 드시면 매력에 빠지실겁니다. . 왜냐구요...? . 이게 업로드 되기전에 저는 두번을 더 만들어 먹었거든요 . . . 0. 재료. 튀김옷 - 항정살 500g / 감자전분 / 계란 흰자. 소스(양념치킨맛) . - 물3큰 / 다진마늘1큰 / 올리고당3큰 / 케찹3큰 / 간장2큰 / 굴소스1큰 / 참기름 1작 . . 1. 항정살을 넉넉한양의 전분과 함께 비닐팩에 넣고 잘 섞어줍니다. 2. 전체적으로 골고루 코팅이 잘 되었는지 확인합니다. 3. 다른 비닐팩에 흰자를 넉넉히 넣고 (4개) 항정살을 섞어줍니다. 4. 계란 흰자가 잘 묻어있는지 확인합니다. 5. 첫 번째 비닐팩에 감자 전분을 다시 넣어주고 골고루 섞어줍니다. 6. 모든 면에 전분이 잘 묻어 있고 덩어리진 것이 없는지 확인합니다. 7. 참기름 또는 5분간 센 불에서 예열한 기름에 항정살을 넣어줍니다. 8. 뚜껑을 닫고 김이 날 때까지 튀겨줍니다 (8~10분). 9. 소스재료를 모두 섞어주고 끓어오를 때까지 가열합니다. 10. 소스가 끓으면 튀김을 넣고 골고루 섞어줍니다. 11. 완성. . ✔️ 재료난이도 별점시스템 안내. . 0.5 집에 다 있는 재료. 1.0 편의점에서 구입 가능. 1.5 편의점에서 3가지 이상 구입. 2.0 동네마트 구입 가능. 2.5 동네마트에서 3가지 이상구입. 3.0 대형마트 구입가능. 3.5 대형마트에서 3가지 이상구입. 4.0 인터넷으로만 구입가능. 4.5 인터넷으로 3가지 이상구입. 5.0 재료비 5만원 이상. . ✔️ 요리레벨 별점시스템. . 0.5 그냥 대충 넣고 조리해도 됨. 1.0 계량스푼 필요. 1.5 계량스푼 + 타이머. 2.0 칼질 해야함. 2.5 칼질 해야함 + 계량스푼. 3.0 칼질 해야함 + 계량스푼 + 타이머. 3.5 불조절 필요 + 계량스푼 + 타이머. 4.0 불조절 필요 + 칼질 해야함 + 계량스푼  + 타이머. 4.5 불조절 필요 + 재료 특수손질 + 계량스푼 + 타이머. 5.0 그냥 시켜먹는게 좋음, 관상용 레시피. . ✔️ 승우아빠 사용도구들. . 편수냄비 - 그린팬 우드비 에디션 편수 16cm. 양수냄비 - 휘슬러 더 크레스트 프로노보 20cm. 후라이팬 - 하얀색: 그린팬 우드비 에디션 28cm 후라이팬.                    빨간색: 휘슬러 센소레드 28cm 후라이팬.                    스텐팬: 휘슬러 뉴 크리스피 프리미엄 26cm 후라이팬. 궁중웍 - 하얀색 : 그린팬 우드비 에디션 28cm 웍.                빨간색 : 휘슬러 센소레드 28cm 웍. 냄비뚜껑 -  휘슬러 28cm 유리뚜껑. 브루스타 - 무인양품 MUJI RK-2 . 식칼 - MAC 프로페셔널 셰프나이프 8인치. 넓은식칼 - MAC 프로페셔널 일본식 야채칼 6.5인치. 다용도칼 - MAC 프로페셔널 유틸리티 나이프 5인치. 과도 - MAC 프로페셔널 페링 나이프 3.25인치. 뼈칼 - MAC 셰프시리즈 보닝 나이프 6인치 (휘는것).            MAC 재패니즈 시리즈 보닝 나이프 6인치 (롱기누스). 슬라이서 - MAC 울티메이트 시리즈 슬라이서 10.25인치. . ✔️ 트위치 생방송 https://www.twitch.tv/swab85. ✔️ 공식카페 https://cafe.naver.com/swab. ✔️ 인스타그램 https://www.instagram.com/mokjinhwa. ✔️ 블로그 https://blog.naver.com/qfinder. ✔️ 문의 mokjinhwa@gmail.com"
2019-06-04 04:57:02,875 (7faa2d7fa700) :  INFO (__init__:449) - [ ] date:     "2019-02-22T02:00:01.000Z"
2019-06-04 04:57:02,876 (7faa2d7fa700) :  INFO (__init__:451) - [ ] rating:   "9.0"
2019-06-04 04:57:02,876 (7faa2d7fa700) :  INFO (__init__:452) - [ ] thumbs:   "https://i.ytimg.com/vi/2zyRda-LBkE/sddefault.jpg"
2019-06-04 04:57:02,876 (7faa2d7fa700) :  INFO (__init__:454) - [ ] duration: "PT10M16S"->"None"
2019-06-04 04:57:02,876 (7faa2d7fa700) :  INFO (__init__:459) - [ ] director: "승우아빠"
2019-06-04 04:57:02,877 (7faa2d7fa700) :  INFO (__init__:465) - [ ] posters: https://i.ytimg.com/vi/2zyRda-LBkE/sddefault.jpg
2019-06-04 04:57:02,877 (7faa2d7fa700) :  INFO (__init__:467) - [ ] genres:   "[]"
2019-06-04 04:57:02,878 (7faa2d7fa700) :  INFO (__init__:473) - =============================================================================================================================================================
2019-06-04 04:57:02,878 (7faa2d7fa700) :  CRITICAL (model:192) - Exception serializing TV_Show with guid 'com.plexapp.agents.youtube://youtube|UCgsffS7MfKL6YU3r_U3E-aA|승우아빠 [UCgsffS7MfKL6YU3r_U3E-aA]?lang=xn' (most recent call last):
  File "/usr/lib/plexmediaserver/Resources/Plug-ins-005053bb5/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/modelling/model.py", line 190, in _write
    self._serialize(os.path.join(self._storage_path, subdir))
  File "/usr/lib/plexmediaserver/Resources/Plug-ins-005053bb5/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/modelling/model.py", line 215, in _serialize
    el.set('id', self._id)
  File "lxml.etree.pyx", line 699, in lxml.etree._Element.set (lxml-2.3/src/lxml/lxml.etree.c:34531)
  File "apihelpers.pxi", line 563, in lxml.etree._setAttributeValue (lxml-2.3/src/lxml/lxml.etree.c:15781)
  File "apihelpers.pxi", line 1366, in lxml.etree._utf8 (lxml-2.3/src/lxml/lxml.etree.c:22211)
ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

2019-06-04 04:57:02,879 (7faa2d7fa700) :  DEBUG (runtime:88) - Sending packed state data (107 bytes)
2019-06-04 04:57:02,879 (7faa2d7fa700) :  DEBUG (runtime:924) - Response: [200] str, 16 bytes

And com.plexapp.system.log

2019-06-04 04:57:01,771 (7fad4effd700) :  DEBUG (runtime:717) - Handling request GET /system/agents/update?mediaType=2&force=1&guid=com%2Eplexapp%2Eagents%2Eyoutube%3A%2F%2Fyoutube%7CUCCaYq4ZNPHP9v_0vL2xjrWw%7C%EB%94%94%EC%97%90%EC%9D%B4%EC%97%94%EC%A7%80%20[UCCaYq4ZNPHP9v_0vL2xjrWw]%3Flang%3Dxn&id=13954
2019-06-04 04:57:01,772 (7fad4effd700) :  DEBUG (runtime:814) - Found route matching /system/agents/update
2019-06-04 04:57:01,773 (7fad4effd700) :  DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/:/plugins'
2019-06-04 04:57:01,775 (7fad4effd700) :  DEBUG (agentservice:322) - Adding com.plexapp.agents.youtube://youtube|UCCaYq4ZNPHP9v_0vL2xjrWw|디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw]?lang=xn to the update list
2019-06-04 04:57:01,776 (7fad4ffff700) :  INFO (agentservice:382) - Preparing metadata for TV_Show in com.plexapp.agents.youtube (youtube|UCCaYq4ZNPHP9v_0vL2xjrWw|디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw])
2019-06-04 04:57:01,776 (7fad4effd700) :  DEBUG (runtime:924) - Response: [200] InitiateUpdateResponse, 161 bytes
2019-06-04 04:57:01,777 (7fad4ffff700) :  DEBUG (runtime:1117) - Created a thread named 'notify_processing'
2019-06-04 04:57:01,777 (7fad2ffff700) :  DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/:/metadata/processing?id=13954'
2019-06-04 04:57:01,778 (7fad4ffff700) :  DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/:/plugins/com.plexapp.agents.youtube/messaging/function/X0FnZW50S2l0OlVwZGF0ZU1ldGFkYXRh/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKMTAKczIKeG5zNApsYW5nYjFzNQpmb3JjZWIwczgKcGVyaW9kaWNzNQoxMzk1NHM0CmRiaWRpMApzNwp2ZXJzaW9ubnMxMApwYXJlbnRHVUlEbnM4CnBhcmVudElEczcKVFZfU2hvd3MxMAptZWRpYV90eXBlczExMgpjb20ucGxleGFwcC5hZ2VudHMueW91dHViZTovL3lvdXR1YmV8VUNDYVlxNFpOUEhQOXZfMHZMMnhqcld3fOuUlOyXkOydtOyXlOyngCBbVUNDYVlxNFpOUEhQOXZfMHZMMnhqcld3XT9sYW5nPXhuczQKZ3VpZHM3NQp5b3V0dWJlfFVDQ2FZcTRaTlBIUDl2XzB2TDJ4anJXd3zrlJTsl5DsnbTsl5Tsp4AgW1VDQ2FZcTRaTlBIUDl2XzB2TDJ4anJXd11zMgppZHIwCg__'
2019-06-04 04:57:01,783 (7fad2ffff700) :  DEBUG (runtime:717) - Handling request GET /system/agents/update?mediaType=2&force=1&guid=com%2Eplexapp%2Eagents%2Eyoutube%3A%2F%2Fyoutube%7CUCgsffS7MfKL6YU3r_U3E-aA%7C%EC%8A%B9%EC%9A%B0%EC%95%84%EB%B9%A0%20[UCgsffS7MfKL6YU3r_U3E-aA]%3Flang%3Dxn&id=13958
2019-06-04 04:57:01,784 (7fad2ffff700) :  DEBUG (runtime:814) - Found route matching /system/agents/update
2019-06-04 04:57:01,784 (7fad2ffff700) :  DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/:/plugins'
2019-06-04 04:57:01,787 (7fad2ffff700) :  DEBUG (agentservice:322) - Adding com.plexapp.agents.youtube://youtube|UCgsffS7MfKL6YU3r_U3E-aA|승우아빠 [UCgsffS7MfKL6YU3r_U3E-aA]?lang=xn to the update list
2019-06-04 04:57:01,788 (7fad2ffff700) :  DEBUG (runtime:924) - Response: [200] InitiateUpdateResponse, 161 bytes
2019-06-04 04:57:01,789 (7fad6ec4f700) :  INFO (agentservice:382) - Preparing metadata for TV_Show in com.plexapp.agents.youtube (youtube|UCgsffS7MfKL6YU3r_U3E-aA|승우아빠 [UCgsffS7MfKL6YU3r_U3E-aA])
2019-06-04 04:57:01,789 (7fad6ec4f700) :  DEBUG (runtime:1117) - Created a thread named 'notify_processing'
2019-06-04 04:57:01,789 (7fad2ffff700) :  DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/:/metadata/processing?id=13958'
2019-06-04 04:57:01,793 (7fad6ec4f700) :  DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/:/plugins/com.plexapp.agents.youtube/messaging/function/X0FnZW50S2l0OlVwZGF0ZU1ldGFkYXRh/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKMTAKczIKeG5zNApsYW5nYjFzNQpmb3JjZWIwczgKcGVyaW9kaWNzNQoxMzk1OHM0CmRiaWRpMApzNwp2ZXJzaW9ubnMxMApwYXJlbnRHVUlEbnM4CnBhcmVudElEczcKVFZfU2hvd3MxMAptZWRpYV90eXBlczEwOQpjb20ucGxleGFwcC5hZ2VudHMueW91dHViZTovL3lvdXR1YmV8VUNnc2ZmUzdNZktMNllVM3JfVTNFLWFBfOyKueyasOyVhOu5oCBbVUNnc2ZmUzdNZktMNllVM3JfVTNFLWFBXT9sYW5nPXhuczQKZ3VpZHM3Mgp5b3V0dWJlfFVDZ3NmZlM3TWZLTDZZVTNyX1UzRS1hQXzsirnsmrDslYTruaAgW1VDZ3NmZlM3TWZLTDZZVTNyX1UzRS1hQV1zMgppZHIwCg__'
2019-06-04 04:57:02,309 (7fad4ffff700) :  DEBUG (model:32) - Loading model with GUID com.plexapp.agents.youtube://youtube|UCCaYq4ZNPHP9v_0vL2xjrWw|디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw]?lang=xn
2019-06-04 04:57:02,309 (7fad4ffff700) :  DEBUG (model:234) - Deserializing from /config/Library/Application Support/Plex Media Server/Metadata/TV Shows/3/35f3ce85c87bdc18c6aa31391b1c742f18e9abb.bundle/Contents/com.plexapp.agents.youtube/Info.xml
2019-06-04 04:57:02,309 (7fad4ffff700) :  ERROR (model:236) - Unable to deserialize object at /config/Library/Application Support/Plex Media Server/Metadata/TV Shows/3/35f3ce85c87bdc18c6aa31391b1c742f18e9abb.bundle/Contents/com.plexapp.agents.youtube
2019-06-04 04:57:02,311 (7fad4ffff700) :  DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/library/metadata/13954/tree'
2019-06-04 04:57:02,315 (7fad4ffff700) :  DEBUG (sandbox:19) - >>> Agent order: []
2019-06-04 04:57:02,315 (7fad4ffff700) :  DEBUG (agentservice:638) - Combining subtitles for media bundle at '/config/Library/Application Support/Plex Media Server/Media/localhost/f/44891549d7797b84dc6096f33efeae7c349036c.bundle'
2019-06-04 04:57:02,316 (7fad4ffff700) :  DEBUG (sandbox:19) - >>> Creating combined XML file from {}
2019-06-04 04:57:02,316 (7fad4ffff700) :  DEBUG (sandbox:19) - Combined XML data: <?xml version='1.0' encoding='utf8'?>
<Subtitles/>

2019-06-04 04:57:02,316 (7fad4ffff700) :  INFO (agentservice:560) - Metadata preparation for com.plexapp.agents.youtube://youtube|UCCaYq4ZNPHP9v_0vL2xjrWw|디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw]?lang=xn complete
2019-06-04 04:57:02,316 (7fad2ffff700) :  INFO (agentservice:728) - Bundle with guid com.plexapp.agents.youtube://youtube|UCCaYq4ZNPHP9v_0vL2xjrWw|디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw]?lang=xn has been updated - notifying the media server
2019-06-04 04:57:02,317 (7fad4ffff700) :  DEBUG (runtime:1117) - Created a thread named 'notify_thread'
2019-06-04 04:57:02,317 (7fad2ffff700) :  DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/:/metadata/notify?guid=com.plexapp.agents.youtube%3A%2F%2Fyoutube%7CUCCaYq4ZNPHP9v_0vL2xjrWw%7C%EB%94%94%EC%97%90%EC%9D%B4%EC%97%94%EC%A7%80+%5BUCCaYq4ZNPHP9v_0vL2xjrWw%5D%3Flang%3Dxn&path=TV+Shows%2F3%2F35f3ce85c87bdc18c6aa31391b1c742f18e9abb.bundle&force=1&queueSize=1&id=13954&success=1&async=1'
2019-06-04 04:57:02,317 (7fad4ffff700) :  DEBUG (agentservice:597) - Removing com.plexapp.agents.youtube://youtube|UCCaYq4ZNPHP9v_0vL2xjrWw|디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw]?lang=xn from the update list
2019-06-04 04:57:02,881 (7fad6ec4f700) :  DEBUG (model:32) - Loading model with GUID com.plexapp.agents.youtube://youtube|UCgsffS7MfKL6YU3r_U3E-aA|승우아빠 [UCgsffS7MfKL6YU3r_U3E-aA]?lang=xn
2019-06-04 04:57:02,881 (7fad6ec4f700) :  DEBUG (model:234) - Deserializing from /config/Library/Application Support/Plex Media Server/Metadata/TV Shows/d/80ed034afaed8e0052e38f5062152177c2a371c.bundle/Contents/com.plexapp.agents.youtube/Info.xml
2019-06-04 04:57:02,882 (7fad6ec4f700) :  ERROR (model:236) - Unable to deserialize object at /config/Library/Application Support/Plex Media Server/Metadata/TV Shows/d/80ed034afaed8e0052e38f5062152177c2a371c.bundle/Contents/com.plexapp.agents.youtube
2019-06-04 04:57:02,884 (7fad6ec4f700) :  DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/library/metadata/13958/tree'
2019-06-04 04:57:02,888 (7fad6ec4f700) :  DEBUG (sandbox:19) - >>> Agent order: []
2019-06-04 04:57:02,889 (7fad6ec4f700) :  DEBUG (agentservice:638) - Combining subtitles for media bundle at '/config/Library/Application Support/Plex Media Server/Media/localhost/e/4e046d2eda76f99054779df96634d8eef244d56.bundle'
2019-06-04 04:57:02,889 (7fad6ec4f700) :  DEBUG (sandbox:19) - >>> Creating combined XML file from {}
2019-06-04 04:57:02,889 (7fad6ec4f700) :  DEBUG (sandbox:19) - Combined XML data: <?xml version='1.0' encoding='utf8'?>
<Subtitles/>

2019-06-04 04:57:02,889 (7fad6ec4f700) :  INFO (agentservice:560) - Metadata preparation for com.plexapp.agents.youtube://youtube|UCgsffS7MfKL6YU3r_U3E-aA|승우아빠 [UCgsffS7MfKL6YU3r_U3E-aA]?lang=xn complete
2019-06-04 04:57:02,889 (7fad2ffff700) :  INFO (agentservice:728) - Bundle with guid com.plexapp.agents.youtube://youtube|UCgsffS7MfKL6YU3r_U3E-aA|승우아빠 [UCgsffS7MfKL6YU3r_U3E-aA]?lang=xn has been updated - notifying the media server
2019-06-04 04:57:02,890 (7fad6ec4f700) :  DEBUG (runtime:1117) - Created a thread named 'notify_thread'
2019-06-04 04:57:02,890 (7fad6ec4f700) :  DEBUG (agentservice:597) - Removing com.plexapp.agents.youtube://youtube|UCgsffS7MfKL6YU3r_U3E-aA|승우아빠 [UCgsffS7MfKL6YU3r_U3E-aA]?lang=xn from the update list
2019-06-04 04:57:02,890 (7fad2ffff700) :  DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/:/metadata/notify?guid=com.plexapp.agents.youtube%3A%2F%2Fyoutube%7CUCgsffS7MfKL6YU3r_U3E-aA%7C%EC%8A%B9%EC%9A%B0%EC%95%84%EB%B9%A0+%5BUCgsffS7MfKL6YU3r_U3E-aA%5D%3Flang%3Dxn&path=TV+Shows%2Fd%2F80ed034afaed8e0052e38f5062152177c2a371c.bundle&force=1&queueSize=0&id=13958&success=1&async=1'

ZeroQI commented 5 years ago

still think it is Plex related Still happening on latest Plex version ?

wiserain commented 5 years ago

Still the same in the latest plex media server. Non-western character in dir (ok in filename) causes

Exception serializing TV_Show with guid 'com.plexapp.agents.youtube://youtube|UCCaYq4ZNPHP9v_0vL2xjrWw|디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw]?lang=xn'

Not sure what exactly the serialization is though. I am suspicious it is somehow related with metadata.id or some guid things (these are made from dirname), so tried to debug but failed. :)

Would you please test video https://www.youtube.com/watch?v=zwKSZczv3LU if you don't mind.

ZeroQI commented 5 years ago

Reproduced

Plex throw an error after the end of my code
replaced series and ep name, same, so one field string cause issues...
japanese is supported, i believe it may be characters that doesn't follow stric encoding rules... Do other korean channels work?

ZeroQI commented 4 years ago

Reproduced again. Show title have chars/format causing issues Tried the following treatment to the string to no avail

Dict(json_playlist_details, 'snippet', 'title').encode("iso-8859-1").decode("utf-8")
unicodedata.normalize('NFKD', unicode(Dict(json_playlist_details, 'snippet', 'title'))).encode('ASCII', 'ignore')

I am not good enough in Python to solve this issue i am afraid...

harroguk commented 4 years ago

I have reproduced this error with any video from the following channel https://www.youtube.com/user/testedcom %(uploader)s (From youtube-dl) returns as "Adam Savage’s Tested" The ’ is offending character (remove it and it works) and appears in the logs as I:\TV\YouTube\Adam Savageâ€™s Tested [UCiDJtJKMICpb9B1qf7qjEOA]\Adam Savageâ€™s Tested - s2019e111012302 - Hands-On with Google Stadia and Technology Deep Dive! [eouqoIur5c4].mp4

As you point out, it looks like Plex files that can't handle them rather than anything in the bundle.

2020-01-08 15:01:15,700 (2e38) :  CRITICAL (core:574) - Exception serializing TV_Show with guid 'com.plexapp.agents.youtube://youtube|UCiDJtJKMICpb9B1qf7qjEOA|Adam Savageâ€™s Tested [UCiDJtJKMICpb9B1qf7qjEOA]?lang=xn' (most recent call last):
  File "C:\InstalledPrograms\PlexMediaServer\Resources\Plug-ins-ac2afe5f8\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\modelling\model.py", line 190, in _write
    self._serialize(os.path.join(self._storage_path, subdir))
  File "C:\InstalledPrograms\PlexMediaServer\Resources\Plug-ins-ac2afe5f8\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\modelling\model.py", line 215, in _serialize
    el.set('id', self._id)
  File "lxml.etree.pyx", line 699, in lxml.etree._Element.set (lxml-2.3\src\lxml\lxml.etree.c:34531)
  File "apihelpers.pxi", line 563, in lxml.etree._setAttributeValue (lxml-2.3\src\lxml\lxml.etree.c:15781)
  File "apihelpers.pxi", line 1366, in lxml.etree._utf8 (lxml-2.3\src\lxml\lxml.etree.c:22211)
ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

ZeroQI commented 4 years ago

i wonder if i could replace in the logs manually with a normal apostrophe, and if hte error would disapear too...

harroguk commented 4 years ago

i wonder if i could replace in the logs manually with a normal apostrophe, and if hte error would disapear too...

I would guess so. Once I renamed the directory and filename to remove the ’ then parsing happened successfully.

ZeroQI commented 4 years ago

in line 42 in function: def filterInvalidXMLChars(string) try to add: if '’' in string: string = string.replace('’',"'")

Please report if it fixes the issue and will add code to the master

wiserain commented 4 years ago

in line 42 in function: def filterInvalidXMLChars(string) try to add: if '’' in string: string = string.replace('’',"'")

Please report if it fixes the issue and will add code to the master

No luck with my case.

ZeroQI commented 4 years ago

You do use the "Absolute Series Scanner" as scanner? it is meant to filter those out normally...

https://stackoverflow.com/questions/2477452/%C3%A2%E2%82%AC-showing-on-page-instead-of t's a ’ (RIGHT SINGLE QUOTATION MARK - U+2019) character which has been encoded as CP-1252 instead of UTF-8. If you check the encodings table, then you see that this character is in UTF-8 composed of bytes 0xE2, 0x80 and 0x99. If you check the CP-1252 code page layout, then you'll see that each of those bytes stand for the individual characters â, € and ™. => Use UTF-8 instead of CP-1252 to read write, store, and display the characters.

So encoding issue and the character is changed somehow

bad_encoding_dict = {
  'â€“': "—",
  "â€”": "–,
  "â€˜": "‘",
  "â€™": "’",
  "â€œ": "“",
  "â€": "”"
}

in line 42 in function: def filterInvalidXMLChars(string) try to add the first line: (the others are development notes)

if 'â' in string: string = string.replace('â€™',"'") #POC, if works will code the second one in the master
if 'â' in string: for string in bad_encoding_dict: string = string.replace(string, bad_encoding_dict[string])
` if 'â' in string: string = string.encode("Windows-1252").force_encoding("UTF-8") #Not sure the function force_encoding exists in python 2 nor is loaded by default on python 3

garbled1 commented 4 years ago

I'm seeing this too: 2020-07-19 15:41:48,752 (7fb8d57fa700) : CRITICAL (model:192) - Exception serializing TV_Show with guid 'com.plexapp.agents.youtube://youtube|UUSFaYYQzNMLo2U6rSNLpghg|Torbjörn Åhman-[UUSFaYYQzNMLo2U6rSNLpghg]?lang=xn' (most recent call last): File "/usr/lib/plexmediaserver/Resources/Plug-ins-79e214ead/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/modelling/model.py", line 190, in _write self._serialize(os.path.join(self._storage_path, subdir)) File "/usr/lib/plexmediaserver/Resources/Plug-ins-79e214ead/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/modelling/model.py", line 215, in _serialize el.set('id', self._id) File "lxml.etree.pyx", line 699, in lxml.etree._Element.set (lxml-2.3/src/lxml/lxml.etree.c:34531) File "apihelpers.pxi", line 563, in lxml.etree._setAttributeValue (lxml-2.3/src/lxml/lxml.etree.c:15781) File "apihelpers.pxi", line 1366, in lxml.etree._utf8 (lxml-2.3/src/lxml/lxml.etree.c:22211) ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

ZeroQI commented 4 years ago

I am sorry but i have no idea how to fix. Ideas welcome

harroguk commented 4 years ago

My work around was simply to drop any special (non-unicode) characters when I name my files. Because the agent matches only based on the youtubeID as long as the filename doesn't contain anything bizarre it seemed to parse with no problems.

Personally I would put this as a "won't fix" people can just name their files nicely/differently :p

garbled1 commented 4 years ago

I did this on another project once.. there is a little python trick to convert unicode stuff to near-ascii equivilent.. could you use something like that and pass everything through it?

ZeroQI commented 4 years ago

Do you mean something like line 40-43

#Based on an answer by John Machin on Stack Overflow http://stackoverflow.com/questions/8733233/filtering-out-certain-bytes-in-python
def filterInvalidXMLChars(string):
  def isValidXMLChar(char):  c = ord(char);  return 0x20 <= c <= 0xD7FF or 0xE000 <= c <= 0xFFFD or 0x10000 <= c <= 0x10FFFF or c in (0x9, 0xA, 0xD)
  return filter(isValidXMLChar, string)

And called line 333 metadata.title = filterInvalidXMLChars(Dict(json_playlist_details, 'snippet', 'title')); Log.Info('[ ] title: "{}"'.format(metadata.title))

garbled1 commented 4 years ago

Ahh, found what I was doing in my other code.. foo.encode('ascii', 'ignore') which I think just drops the bad chars.

wiserain commented 4 years ago

I found a temporary workaround for this issue by removing a folder name from a metadata id as in https://github.com/wiserain/YouTube-Agent.bundle/commit/43f4828fca7d987fa63f6719ce7df11daf3c91d7

For the example in my original thread, metadata id with a folder name youtube|UCCaYq4ZNPHP9v_0vL2xjrWw|디에이엔지 [UCCaYq4ZNPHP9v_0vL2xjrWw] could be updated to UCCaYq4ZNPHP9v_0vL2xjrWw so it can process with a valid channel id.

I don't know what a proper way to fix is, but hope this help finding it.

ZeroQI commented 4 years ago

Knowing the weird character in id cause the issue is key to solve the issue, thank you...

Some users might want to split in different series the videos from the same channel. I want this functionality working, therefore the GUID need to be unique per series, not per channel
The folder was added to make the GUID unique, as if two items have the same GUID, they will be merged in Plex
We could use any string as long as it makes the GUID unique, so changing from 'dir' to 'dir.UUID()' lines 127, 144, 158, 167 would do the trick. If somebody that can test and report, would include in master code

wiserain commented 4 years ago

@ZeroQI Unable to use dir.UUID() because of error 'str' object has no attribute 'UUID'. Instead, I tried with uuid.uuid3(uuid.NAMESPACE_URL, dir).hex and verified it works. If anyone has better solutions, please let me know so I can test.

harroguk commented 3 years ago

Just found an option in youtube-dl that resolves this issue

--restrict-filenames
Restrict filenames to only ASCII characters, and avoid "&" and spaces in filenames

makes the filenames slightly messy but matching is fixed.

ZeroQI commented 3 years ago

Thanks for the update. Not good enough with code pages to solve propertly

ZeroQI commented 3 years ago

A user solved these type of issues on HAMA, so writing it here for future reference to solve:

https://github.com/ZeroQI/Absolute-Series-Scanner/issues/335

When setup logging, force the file encoding to be utf-8

if backup_count:  Handler = logging.handlers.RotatingFileHandler(log_file, maxBytes=10*1024*1024, backupCount=backup_count, encoding='utf-8')
else:             Handler = logging.FileHandler                 (log_file, mode=mode, encoding='utf-8'

When calling Log.info, force the format string to be unicode Log.info(u'id: {}'.format(id))
At the beginning of Scan, sanitize all input arguments related to file path, and decode them to unicode, to ensure consistency

No need for .decode('utf-8') Other commands used:


tv_show = Media.Episode(show.encode('utf-8'), season, epn, title.encode('utf-8'), year)
else:  tv_show.parts.append(file.encode(sys.getfilesystemencoding()))

### Make sure the path is unicode, if it is not, decode using OS filesystem's encoding ###
def sanitize_path(p):
  if not isinstance(p, unicode):  return p.decode(sys.getfilesystemencoding())
  return p

  ### Sanitize all path
  if path:  path = sanitize_path(path)
  if root:  root = sanitize_path(root)
  files = [sanitize_path(p) for p in files]
  dirs = [sanitize_path(p) for p in dirs]```

ZeroQI commented 3 years ago

After full rewrite to reduce length and handle foreign codepage better

2021-03-11 00:18:52,667 (2d74) :  INFO (logkit:16) - C:\Users\benja\Videos\_Plex3\디에이엔지 [youtube2-UCCaYq4ZNPHP9v_0vL2xjrWw]\NA - 발라드가수의비극 feat.이해리 - [zwKSZczv3LU].mp4
2021-03-11 00:18:52,667 (2d74) :  INFO (logkit:16) - =============================================================================================================================================================
2021-03-11 00:18:52,667 (2d74) :  INFO (logkit:16) - Search() - dir: C:\Users\benja\Videos\_Plex3\디에이엔지 [youtube2-UCCaYq4ZNPHP9v_0vL2xjrWw], filename: C:\Users\benja\Videos\_Plex3\ëìì´ìì§ [youtube2-UCCaYq4ZNPHP9v_0vL2xjrWw]\NA - ë°ë¼ëê°ììë¹ê·¹ feat.ì´í´ë¦¬ - [zwKSZczv3LU].mp4, displayname: 디에이엔지 [youtube2-UCCaYq4ZNPHP9v_0vL2xjrWw], Prefs['metadata_source']
2021-03-11 00:18:52,667 (2d74) :  INFO (logkit:16) - search() - YouTube ID not found - regex: "PLAYLIST"
2021-03-11 00:18:52,668 (2d74) :  INFO (logkit:16) - search() - YouTube ID found - regex: CHANNEL, youtube ID: "UCCaYq4ZNPHP9v_0vL2xjrWw"
2021-03-11 00:18:52,668 (2d74) :  INFO (logkit:16) - =============================================================================================================================================================
2021-03-11 00:18:52,670 (2d74) :  DEBUG (runtime:88) - Sending packed state data (107 bytes)
2021-03-11 00:18:52,670 (2d74) :  DEBUG (runtime:924) - Response: [200] str, 788 bytes
2021-03-11 00:19:06,275 (113c) :  DEBUG (runtime:717) - Handling request GET /:/plugins/com.plexapp.agents.youtube/messaging/function/X0FnZW50S2l0OlNlYXJjaA__/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoyCmRpY3QKZGljdAo2CnMyCnhuczQKbGFuZ2IxczYKbWFudWFsYjFzNwpwcmltYXJ5aTAKczcKdmVyc2lvbnIxCnM2Cmt3YXJnc3M3ClRWX1Nob3dzMTAKbWVkaWFfdHlwZTMKbnM0CnllYXJzNQoxODk3N3MyCmlkczUwCuuUlOyXkOydtOyXlOyngCBbeW91dHViZS1VQ0NhWXE0Wk5QSFA5dl8wdkwyeGpyV3ddczQKc2hvd3IwCg__
2021-03-11 00:19:06,276 (113c) :  DEBUG (runtime:49) - Received packed state data (80 bytes)
2021-03-11 00:19:06,278 (113c) :  DEBUG (runtime:814) - Found route matching /:/plugins/com.plexapp.agents.youtube/messaging/function/X0FnZW50S2l0OlNlYXJjaA__/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoyCmRpY3QKZGljdAo2CnMyCnhuczQKbGFuZ2IxczYKbWFudWFsYjFzNwpwcmltYXJ5aTAKczcKdmVyc2lvbnIxCnM2Cmt3YXJnc3M3ClRWX1Nob3dzMTAKbWVkaWFfdHlwZTMKbnM0CnllYXJzNQoxODk3N3MyCmlkczUwCuuUlOyXkOydtOyXlOyngCBbeW91dHViZS1VQ0NhWXE0Wk5QSFA5dl8wdkwyeGpyV3ddczQKc2hvd3IwCg__
2021-03-11 00:19:06,278 (113c) :  INFO (agentkit:961) - Searching for matches for {'show': '\xeb\x94\x94\xec\x97\x90\xec\x9d\xb4\xec\x97\x94\xec\xa7\x80 [youtube-UCCaYq4ZNPHP9v_0vL2xjrWw]', 'id': '18977', 'year': None}
2021-03-11 00:19:06,279 (113c) :  DEBUG (networking:143) - Requesting 'http://127.0.0.1:32400/library/metadata/18977/tree'
2021-03-11 00:19:06,289 (113c) :  CRITICAL (core:574) - Exception in the search function of agent named 'YouTubeSeries', called with keyword arguments {'show': '\xeb\x94\x94\xec\x97\x90\xec\x9d\xb4\xec\x97\x94\xec\xa7\x80 [youtube-UCCaYq4ZNPHP9v_0vL2xjrWw]', 'id': '18977', 'year': None} (most recent call last):
  File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-d8c4875dd\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\api\agentkit.py", line 1011, in _search
    agent.search(*f_args, **f_kwargs)
  File "C:\Users\benja\AppData\Local\Plex Media Server\Plug-ins\YouTube-Agent.bundle\Contents\Code\__init__.py", line 510, in search
    def search (self, results,  media, lang, manual):  Search (results,  media, lang, manual, False)
  File "C:\Users\benja\AppData\Local\Plex Media Server\Plug-ins\YouTube-Agent.bundle\Contents\Code\__init__.py", line 97, in Search
    filename    = urllib2.unquote(sanitize_path(os.path.basename(media.items[0].parts[0].file if movie else media.filename)))  #urllib2.unquote() filename    = os.path.basename(media.items[0].parts[0].file) if movie else os.path.splitext(os.path.basename(media.filename))[0]
  File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-d8c4875dd\Framework.bundle\Contents\Resources\Versions\2\Python\subsystem\ospathfix.py", line 250, in os_path_basename
    return os.path._basename(uni(path))
  File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-d8c4875dd\Framework.bundle\Contents\Resources\Versions\2\Python\subsystem\ospathfix.py", line 51, in uni
    return unicode(s.decode('utf-8'))
AttributeError: 'NoneType' object has no attribute 'decode'

Coded to sort all issues and back to

File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-d8c4875dd\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\modelling\model.py", line 190, in _write
    self._serialize(os.path.join(self._storage_path, subdir))
ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

your foldername is not unicode seemingly, however tried and it pass tests https://apps.timwhitlock.info/unicode/inspect?s=%EB%94%94%EC%97%90%EC%9D%B4%EC%97%94%EC%A7%80+%5Byoutube2-UCCaYq4ZNPHP9v_0vL2xjrWw%5D#block-UAC00

The code crashing is Plex framework, it happen after the last agent output, I cannot fix, i did try... A bug report need to be raised to Plex

harroguk commented 3 years ago

Is it possible to handle the crash in the agent side of things and out put a clean error message "non-unicode characters detected in file/path" or some such so that people can at least see the cause themselves?

Obviously those of us in the know purge offending characters now but others may not.

ZeroQI commented 3 years ago

i have a function to make unicode called sanitize_path() which return unicode (most compatible) I can't catch any error in my code...

After tests,

the characters "디에이엔지" in the series folder name cause issues, and if i put an ascii series title manually in the code: same thing, so it is the folder name itself, not my use of it
went to google translate and translated it and copies in the series foldername: same issue

reddragonguy commented 3 years ago

regarding the issue in #79 with the "TM" symbol, can the offending symbol be removed from the title before it is sent to Plex?

harroguk commented 3 years ago

regarding the issue in #79 with the "TM" symbol, can the offending symbol be removed from the title before it is sent to Plex?

removing specific special characters is not really a viable solution. It would be much cleaner if you removed the special characters yourself via --restrict-filenames when you downloaded.

reddragonguy commented 3 years ago

Fair enough. It was something I was using but thought I wouldn't need any more after the recent changes. I'll go back to using it.

reddragonguy commented 3 years ago

I think i stopped using it because channels like Key & Peele were showing up as Key_Peele or something like that because the folder name was used as channel name. hopefully, with the recent changes, the channel name that's sent from ASS is updated to the channel name from the channel id.

ZeroQI / YouTube-Agent.bundle

Problem with folder name in non-western characters #28